Experiencing Emotions in Video-Mediated Psychological Counselling Versus to Face-to-Face Settings

Round 1

Reviewer 1 Report

The present manuscript (ms) reports results from a multi-part role-play experiment on how negative and positive emotions are experienced in video-based compared to face-to-face counseling. When asked about emotions experienced in their talk, both client and counsellor reported stronger negative emotions in the video-based format (relative to face-to-face), but stronger positive emotions in the face-to-face format (relative to video-based). The topic is important and it offers interesting implications for counseling that – given the current pandemic situation – is potentially of great interest. I do, however, have a major concern in regard to confounds in the experiment: It cannot be ruled out that the results stem from other factors, most importantly unbalanced positive and negative counseling topics, instead of the format. Unless the authors can rule these out by providing more details, I fear the ms does not provide clear indications that practitioners can consider in their work.

I elaborate on this point and on other issues below and I hope that the authors consider my feedback constructive despite the criticism.

 

Format may be confounded by topics

• “How does using video technology influence the (emotional) experience of health communication?” (p. 7) I do not think this question can be answered adequately by what you report. My major concern lies in that we cannot determine if it is indeed the format or the topics that evoked negative or positive emotions differentially. Perhaps you can address this criticism by just adding some clarifying comments to your ms. Strictly speaking, you ought to have employed a standardized set-up with fixed topics, trained counselors for maximum objectivity etc. to determine if your results are truly due to the differences in format.
• It is a shame that topics were not controlled or recorded – they likely played an important role in which emotions were evoked. Perhaps video-based counselling provides more social/personal distance, which facilitates talking about difficult topics (you mentioned openness on p. 2); whereas face-to-face promotes stronger impression management (i.e., reinforcing positive expressions). You have made important points that reinforce the opposite, as well (pp. 7-8), so we do not really know which is the case, but I have a feeling your follow-up interviews may offer some insight here. For future studies, I would recommend balancing positive and negative topics.

Experiencing emotions = perceiving or feeling?

• At first, reading “experiencing emotion” I expected that the measures would reflect how much participants have perceived positive and negative emotions, instead of felt. So perhaps you want to elaborate this aspect some more as early as in your abstract. What was instructed when PANAS/AD-ACL were employed? Was it something like “how did you experience the counselling” or “how did it make you feel” or “how did you feel towards the client/counselor”?
• Conceptually, this goes a little further, so bear with me for adding some more thoughts: Given the ms deals with felt emotions, your approach may rather reflect the mood of the conversation, not what emotional information was actively exchanged/mirrored between client and counsellor. Think that even if a talk goes awry, partners may still be smiling politely and value each other. They may later say the talk was bad but the person was nice – are there, too, differences between formats? Does the video-format facilitate the exchange of negative content by more openness and less social presence, but does the face-to-face format facilitate interpersonal positivity and attraction (which you mention very loosely)? So there is this distinct difference between feeling and perceiving, that might be interesting for future studies, too.
• Also, please consider that although negative emotions were more intense in the video- compared to face-to-face format, positive emotions were more intense than negative emotions in both So video-based counselling is not experienced as negative per se.

 

Minor issues and suggestions

• I find the role-play methodology interesting. I would have loved to get more insight into the role descriptions – how detailed were they, was there variation? This could also be helpful for other authors who would like to utilize a similar approach.
• 3.1. l. 77: “care must be taken to ensure that emotions actually occur” – How did you make sure of that in your experiment?
• Was sample size calculated a priori? Also, this seems to be a repeated measures design. So, were there four PANAS/AD-ACL questionnaires per person (one for each counseling)?
• 3.2.: Could you add gender ratio, mean age, specifications of what professional counselling is (e.g., years of experience), i.e., interesting variables you mention under 4.3?
• Was any specific technology/platform used for the video-based counselling?
• 3.3, footnote 3: the adjective “energetic did not prove itself” – what does that mean? Was it no reliable item of a scale?
• Generally, don’t you think it would be more clear for the reader to simply list the adjectives that were used in the method section (instead of later when presenting the results)? You could use two lists (positive, negative) that correspond to your rationale presented under 4.1., and list Cronbach’s alpha accordingly.
• There is an inconsistency between n=27 and n=26 participants. Was one case excluded?
• On p. 6 l. 202 you mention “level of qualification of the counsellor” – which levels did you distinguish, what were they based on?
• 4.3. I find it confusing that you term it correlation, and then report cross-tab/frequencies. When and how did you measure the frequency of emotions, I thought you assessed intensities?
• 5. You mention follow-up interviews. Would it not be interesting to elaborate on these in the method section? Why were they conducted, what else did they reveal?
• References: I am not sure if it is common in the present citation format, but please consider providing translations of non-English publications (e.g. APA recommends adding translation in square brackets: … Einführung von Videosprechstunden … [Introduction to video-based counselling] …)
• Look out for typos, faulty grammar and untranslated words (Fig 1). I recommend getting an English proofreader.

 

Gladly, here is what I meant: Perceiving emotions may address the emotional content of the counselling (i.e., we talked about sad and upsetting topics), but it may also concern the perception of the emotion conveyed by the counterpart (i.e., the client/counsellor seemed sad and upset about the topic). The locus of perceiving emotion is outside the individual who reports it. Feeling emotions addresses one’s own experiences (i.e., the topics made me feel sad and upset, or: I feel our relationship was bad). Here, the locus of felt emotions is the individual herself.

Perhaps, interpersonal attraction is a matter of how perceived and felt overlap, since this may signal understanding between both counterparts (maybe this fits with the correlations you mentioned?), and that would ultimately address the relationship. However, this probably gets too complicated for what you want to show in your manuscript, so we don’t have to go into details if this is not going to be constructive for your writing process.

To sum it up, my criticism was rather that it could be more clear in your manuscript which you are referring to (perceived or felt) when you say emotion has been ‘experienced’. The easiest way to do this, I suspect, would be by simply including how exactly you asked about the emotions – i.e., the instructions participants received when they filled in the PANAS etc.

Author Response

Dear reviewer 1,

Please see the attachment.Kindest regards

 

Author Response File: Author Response.pdf

Reviewer 2 Report

Thank you for the opportunity to review your study. I find this research timely in the era of COVID, which prompted more use of telehealth services in the psychological and psychiatric fields. I was excited to read what you discovered in your study, as this very much relates to these professions right now.

I would like to know more details about the counseling interviews. Maybe either a table with descriptions of what topics were discussed or possibly even examples within your text. I could not get a sense from your article how much positive versus negative emotions would be expected from the content within these interviews. I also would like to then see follow-up in your discussion about how much consistency you saw qualitatively between what was discussed and then the emotions rated on your empirical scales.

I would also like if you provided details on what the positive emotions and negative emotions were that you were looking at in this study. I got a sense of what scales were used to derive these, but it would be more helpful if you listed out the specific emotions in each of these categories.

I am wondering if there should be a note in there about the relevance of this study in the context of the now COVID pandemic...I know that was not on your radar when going through the collection of this data, but I can't help but to think that people reading this now will have that forefront in their minds when they read your article. In fact, I think the COVID pandemic will make your article more relevant as well.

There were a few edits that I have listed below that you will also need to do before publication:

 

Line 20 - "appropriate medical care" -- I think instead of medical used here, maybe say psychiatric/psychological care

Line 24 - Remove the parentheses around "emotional"

Line 49 - Remove the open parentheses before "virtual reality"

Line 50 and 51 - Maintain consistent verb usage (For example:  "...this article presents the study design, justifies the selection of methodology, and then critically assesses the evidence..."

Line 60 - Change "the study" to "this study"

Line 120 - Remove the comma after "not appropriate"

Line 220 - Change the ? mark after "communication" to a period

Line 257 - Add a comma after "video format,"

Author Response

Dear reviewer 2,

Thank you for your precise and constructive comments. We found your approach to perceived vs. felt emotions very fruitful. Your substantial comments helped us to revise and improve our paper.

We specified the article - along the following main aspects:

• Developed a better description of the study design and its controlled conditions including an added description of the counselling topics (see section 3.1. „Experimental setup and procedure“) and a discussion of its influence on emotional experience (see section 5 „Critical appraisal of the evidence“),
• Added a listing of the emotion items to get a sense of what scales were used and to make it easier for the reader to follow the results reported later (see section 3.3. „Selection of the measuring instrument“),
• Added a note about the relevance of this study in the context of the COVID 19 pandemic (see chapter 1 „Introduction and relevance of video-based counselling“).

In addition, we have corrected all your mentioned edits (e.g. typos, incorrect grammar).

We hope that this addresses your critique and that you consider the revised changes to be adequate.

Reviewer 3 Report

The authors presented an interesting approach to the problem. However, I had a few comments.
1. The authors pay little attention to the analysis of previous studies. Especially, published over the past 5 years. This is sad because they themselves claim to have reviewed 28 publications, three of which were considered very relevant to the subject of the study.
2. The authors paid great attention to the description of the study procedure, but not its results.
2.1. The article does not specify the sex of study participants, which seems important for comparing study results with other data.
2.2. Table 1 shows the data in isolation from the methods of mathematical analysis used. Mean sample values are given under different conditions. And the proposed methods analyze the dynamics (change) of participants indicators in different conditions.
2.3. Table 1 should be redone because in the proposed version it is not clear what kind of differences is indicated by the sign "*".
2.4. The sample size should be clearly indicated. The annotations and introduction indicate 27 cases and tables 26. It is not clear where one case of observation went.
2.5. It is believed that variance analysis is incorrect to apply in a situation where the distribution of the test feature in the sample is different from the normal distribution of scales. Therefore, the results of the Wilcoxon Test deserve more attention. Authors should draw them up on a table.
2.6. The authors claim that "In video format, negative emotions, such as being depressed, tired, or stressed, are experienced more intensely than positive emotions". This conclusion cannot be drawn from the data presented in table 1.

Author Response

Dear reviewer 3,

Thank you for your precise and constructive comments that helped us to revise our paper.

We specified the article extensively - along the following main aspects:

• more detailed description of the sample including sex of study participants (2.1; see section 3.2. „Description of the sample),
• improved description of the study design including specification of repeated measures design under different conditions (2.2)
• more precise labelling of the table in which the results are reported, in addition to a legend inserted explaining the sign "*" (2.3, 2.4; see section 4.2. „Intensity of the emotional experience“),
• justification of the applied analysis methods including a discussion of required prerequisites (2.5; see section 4.2. „Intensity of the emotional experience“),
• added listing of the emotion items in order to be able to draw the conclusion you mentioned (2.6, see section 3.3. „Selection of the measuring instrument“),
• clarification of the focus regarding the term "emotional experience",
• added description of the counselling topics (see section 3.1. „Experimental setup and procedure“),
• additions to the last part „Conclusions“

We hope that this addresses your critique and that you consider the revised changes to be adequate.

Round 2

Reviewer 1 Report

I thank the authors for this revision and for addressing my comments in their responses! I find the manuscript has been noticeably improved and is much clearer in explaining the methodology.

I would like to raise a few points, some of which I already mentioned in my previous review and I hope the authors can take one further look at. On top of that, I have just a few minor additional issues to point out.

 

Old remarks:

• I enquired about an a-priori calculation of the sample size. If none was conducted, a post-hoc power analysis would be helpful. Considering your repeated-measures/within-design, it is hard for the reader to gauge how close or far your results are from an “optimal” sample that would let you draw appropriate conclusions. (I’m glad you recommend using a larger sample in a between-design for future studies.)
• If possible, I would still like to see a brief explanation concerning the follow-up interviews in the method section (e.g., under 3.1.).
• I like that you point out that “Rather, a certain adjustment period is necessary…” (p. 9 ll. 332-333), and thus include a constructive outlook on video-based counselling! (Consider that this and the preceding sentence may even be better suited at the end of that particular paragraph.)
• References: I actually looked up the ACS style that the journal requires (see https://www.mdpi.com/authors/references), and it recommends to “Cite journals published in a foreign language either by the actual non-English title or by a translated form” and “When citing an article printed in the English translation of a foreign-language journal, include reference to the original article, if possible, and use a semicolon to separate the two citations” (p. 299 in the ACS guidelines). I’m glad you followed my suggestion to translate titles, but (a) it might in fact not be necessary (in which case I apologize for my initial suggestion), or (b) all remaining non-English titles in your list should be translated as well. I guess it could be up to you, or perhaps the handling editor has suggestions that are more accurate on this issue.

 

Additional remarks:

• Abstract: remove brackets from “emotional” (l. 5), same on p. 7 l. 251
• Regarding the fact that your topics were job-related, I suggest you replace ‘health communication’ in the introduction with the broader term ‘psychological counselling’, but you can then keep specifying the relevance of emotional issues for health communication in the conclusion section, thus raising an interesting point for future studies.
• I am a little worried about the description of roles & topics, for “Roles and topics were described very specifically: …” (p. 3, l. 108) contradicts the line “At the same time, they were free to introduce their own topics” in the initial version (p. 3, ll. 94-95). How do you reconcile that? By the latter, do you perhaps mean that you let the interlocutors interact within the boundaries of the role and topic descriptions, thus giving the counselling space to develop freely?
• Could you include the scale (from 1 = low intensity …) on p. 5, l. 151 following the question “How do you feel after the consultation?”
• 6.: You stated that face-to-face format improved the quality of the relationship, which was also assessed in your study. Could you include this in your method section, and briefly report the results? You further mention that correlations between emotions and relationship-quality should be investigated in future studies, but it reads that you could have done so very well yourself because you assessed it, too. Is there no space left in this manuscript, or for what reason did you not include it?
• The following slipped my notice in the first version (my apologies!) on p. 7, ll. 256-257: “Selection bias … completely randomized.” Usually, you randomize to prevent selection bias, right? That means, selection bias might have arisen if the sample was not randomized, or even though it was randomized, but only within a specific pool of participants (i.e., communication students). Could you be clearer here?
• Finally, the ms still requires some careful proof-reading, here is some examples I have spotted:
  • p. 2, ll. 55-57: “For this purpose, this article presents the study design … and then critically assesses the evidence…”
  • p. 2, l. 60: “… - and in particular the emotional experience - …”, I think it would be better to remove this part of the sentence, because you lead up to the emotional experience being one of three central characteristics.
  • The sentence “In 41% of the role-plays…” occurs twice: once on p. 4 l. 130, and a second time on p. 5 ll. 153-154.
  • p. 6, ll. 211-212: The sentence “Finally, assess whether … as significant” is missing a noun.
  • p. 6, ll. 216-218: In the sentence “In video format, negative emotions … than positive emotions”, I assume you mean “than in the face-to-face setting” instead of “positive emotions”.

Author Response

Dear Reviewer 1,

thank you very much for reading and annotating our contribution again so thoroughly. Please find my bolded responses incorporated directly into your notes.

• I enquired about an a-priori calculation of the sample size. If none was conducted, a post-hoc power analysis would be helpful. Considering your repeated-measures/within-design, it is hard for the reader to gauge how close or far your results are from an “optimal” sample that would let you draw appropriate conclusions. (I’m glad you recommend using a larger sample in a between-design for future studies.)
• Thanks for the suggestion. I have included the power analysis so that the reader can get an idea regarding the sample size and frame the results (see chapter 5, p. 8).
• If possible, I would still like to see a brief explanation concerning the follow-up interviews in the method section (e.g., under 3.1.).
• I would like to kindly suggest not to explain the qualitative follow-up methods in further detail and beyond the mentioned indications. Since this would shift the focus of the paper, which in fact centers on the quantitative data. However, I would like to offer to remove the referrals to the mentioned qualitative methods.
• I like that you point out that “Rather, a certain adjustment period is necessary…” (p. 9 ll. 332-333), and thus include a constructive outlook on video-based counselling! (Consider that this and the preceding sentence may even be better suited at the end of that particular paragraph.)
• I have added a paragraph after the sentence you mentioned and marked it in bold to highlight that statement. I hope this meets your requirements.
• References: I actually looked up the ACS style that the journal requires (see https://www.mdpi.com/authors/references), and it recommends to “Cite journals published in a foreign language either by the actual non-English title or by a translated form” and “When citing an article printed in the English translation of a foreign-language journal, include reference to the original article, if possible, and use a semicolon to separate the two citations” (p. 299 in the ACS guidelines). I’m glad you followed my suggestion to translate titles, but (a) it might in fact not be necessary (in which case I apologize for my initial suggestion), or (b) all remaining non-English titles in your list should be translated as well. I guess it could be up to you, or perhaps the handling editor has suggestions that are more accurate on this issue.
• I contacted the editor and we came to an agreement regarding the bibliography. The journal provides formatting service after the paper is accepted.

• Abstract: remove brackets from “emotional” (l. 5), same on p. 7 l. 251
• I removed the brackets.
• Regarding the fact that your topics were job-related, I suggest you replace ‘health communication’ in the introduction with the broader term ‘psychological counselling’, but you can then keep specifying the relevance of emotional issues for health communication in the conclusion section, thus raising an interesting point for future studies.
• I replaced ‚health communication‘ as suggested.
• I am a little worried about the description of roles & topics, for “Roles and topics were described very specifically: …” (p. 3, l. 108) contradicts the line “At the same time, they were free to introduce their own topics” in the initial version (p. 3, ll. 94-95). How do you reconcile that? By the latter, do you perhaps mean that you let the interlocutors interact within the boundaries of the role and topic descriptions, thus giving the counselling space to develop freely?
• Please excuse my improper oversimplification. Since the free topics were exclusively related to the question of whether a doctoral thesis should be pursued, and thus had a clear connection to career topics, I inadmissibly deleted the passage. An amendment in section 3.1 (l. 109) and an explanation in footnote (chapter 5, p. 8) are now included, which hopefully illustrate the methodological procedure and the outcomes sufficiently.
• Could you include the scale (from 1 = low intensity …) on p. 5, l. 151 following the question “How do you feel after the consultation?”

• Do you mean in addition to the legend in Table 2? I hope that I have understood you correctly because I have inserted it additionally in the spot where you suggested.

• 6.: You stated that face-to-face format improved the quality of the relationship, which was also assessed in your study. Could you include this in your method section, and briefly report the results? You further mention that correlations between emotions and relationship-quality should be investigated in future studies, but it reads that you could have done so very well yourself because you assessed it, too. Is there no space left in this manuscript, or for what reason did you not include it?
• In Chapter 3, we briefly highlight which three variables were assessed in the study - among them the quality of the relationship. At the same time, we mention that only the emotional experience is focused on in this paper (ll. 75). The quality of the relationship is certainly an interesting variable. However, this would shift the focus of the paper. Since apart from a theoretical and methodological introduction of this variable, a report of the results and a discussion of the findings would be required. This would significantly increase the dimensions of the paper and raise further issues to be addressed.
• The following slipped my notice in the first version (my apologies!) on p. 7, ll. 256-257: “Selection bias … completely randomized.” Usually, you randomize to prevent selection bias, right? That means, selection bias might have arisen if the sample was not randomized, or even though it was randomized, but only within a specific pool of participants (i.e., communication students). Could you be clearer here?
• I must apologize. There is a "not" missing which may have got lost in the process of reworking, proofreading and editing. It is now added.
• Finally, the ms still requires some careful proof-reading, here is some examples I have spotted:
• p. 2, ll. 55-57: “For this purpose, this article presents the study design … and then critically assesses the evidence…”
• p. 2, l. 60: “… - and in particular the emotional experience - …”, I think it would be better to remove this part of the sentence, because you lead up to the emotional experience being one of three central characteristics.
• The sentence “In 41% of the role-plays…” occurs twice: once on p. 4 l. 130, and a second time on p. 5 ll. 153-154.
• p. 6, ll. 211-212: The sentence “Finally, assess whether … as significant” is missing a noun.
• p. 6, ll. 216-218: In the sentence “In video format, negative emotions … than positive emotions”, I assume you mean “than in the face-to-face setting” instead of “positive emotions”

• I corrected the requested remarks.
• We also arranged to have a native-speaking editor proofread the manuscript. In addition, the editor of the journal told me, that they provide formatting service and proofreading after the paper is accepted.

I hope you agree with the suggestions in this document and revisions made in the manuscript.

Kindest regards

Reviewer 2 Report

The sentence on lines 35-37 could be reworded to make more sense. For example, "This study does not specifically address the relevance on this topic in the context of the COVID-19 pandemic, primarily because this research was started before the pandemic."

Then the very next sentence in line 37-39 could read:  "Nevertheless, the COVID-19 pandemic did lead to a complete shift in applying the results of this study in everyday communicative.." (Is that what you are trying to say?)

 

You did a nice job editing the participants section. It is more clear to me who these individuals were and their experiences.

 

In line 204 (I believe), you need a comma:  "As the distribution of the test feature in the sample is different from the normal distribution of scales, it needs to be examined..."

 

 

Author Response

Dear reviewer 2,

 

Thank you for your review and approval. We considered your suggestions in the manuscript.

Kindest regards

Reviewer 3 Report

The article deserves the attention of the scientific community.

Author Response

Dear reviewer 3,

 

Thank you for your review and approval.

Kindest regards