Patient-Reported Outcomes Measurement in Radiation Oncology: Interpretation of Individual Scores and Change over Time in Clinical Practice

Round 1

Reviewer 1 Report

The presented topic is fascinating and innovative for oncology. PROMs are often underestimated in oncology care. However, the manuscript would benefit from some major revisions.
Background. The presentation of the phenomenon is well describe, but it lacks of a deeper perspective on patients, particularly regarding the introduced concepts of culture, and/or personality (and more), it is not exactly understandable what these concepts being referred. The authors should improve the presentation of these issues and the impact may these issues have on patients.
Method. The objective of the paper is clear only in part. It is unclear whom the sample is composed of and what methodology is used (Delphi panel?). Also it is unclear the analysis process: how did the results come to be presented? I find this section shallow and too brief. 
Results. The results are well presented, but the reference to the analysis is not understandable like the methods section. How were the data analyzed? Also, what was the process that led to obtaining these classifications? 
Discussion. I find the discussions very short and not very contextualized. I would find it interesting to discuss more implications for practice and proposed clinical implementations for care from an innovation manuscript like this. 

I ask the authors to pay attention to several typos in the text, i.e. number 8 in superscript, lines 92 - 94 text size smaller than the rest. 

Author Response

Reviewer 1

The presented topic is fascinating and innovative for oncology. PROMs are often underestimated in oncology care. However, the manuscript would benefit from some major revisions.
Background. The presentation of the phenomenon is well describe, but it lacks of a deeper perspective on patients, particularly regarding the introduced concepts of culture, and/or personality (and more), it is not exactly understandable what these concepts being referred. The authors should improve the presentation of these issues and the impact may these issues have on patients.

Thank you for your comments. We expanded on the cultural perspective in the introduction starting in line 66: “For example, the Western view may primarily view pain as an external disturbance that interferes with everyday life while other cultures may view pain as being part of one’s life journey [14], which may influence how patients respond to PROM items.”

Method. The objective of the paper is clear only in part. It is unclear whom the sample is composed of and what methodology is used (Delphi panel?). Also it is unclear the analysis process: how did the results come to be presented? I find this section shallow and too brief. 

 

Thank you for this feedback. As this was a knowledge translation project, we have added a section Approach and described the workshop participants (line 107-114) and the analysis process (line 127-133):

 

1.2 Approach

The results presented were drawn from discussions from the members of the Clinical Practice and Response Shift Special Interest Groups at the International Society of Quality of Life (ISOQOL) as part of the KT initiative. The members were invited through the ISOQOL listserv and include researchers and clinicians in various practices such as family medicine, neurology and cancer (N=12) who shared a common interest in improving the capture and interpretation of PROMs to support individual-level decisions in clinical practice settings.

 

Extrapolation of the vignettes were guided by a collaborative examination of the discussions in the online workshops focusing on the type of dialogue between Bill and a clinician as well as literature on DIF and RS. Through this review, we identified important aspects of DIF and RS to further explore and identify in the dialogue. For example, we explored differences in meaning that the PROM scores may have for Bill compared to others depending on life situations. In addition, the research team and the workshop participants reviewed the vignettes.

 

Results. The results are well presented, but the reference to the analysis is not understandable like the methods section. How were the data analyzed? Also, what was the process that led to obtaining these classifications? 

See above methods section changes in line 107-114 as this was a knowledge translation project (not a research study).

Discussion. I find the discussions very short and not very contextualized. I would find it interesting to discuss more implications for practice and proposed clinical implementations for care from an innovation manuscript like this. 

Thank you for this feedback. We have added the implication section and added a new paragraph (line 360-365):

3.1 Implications for practice

The implication in practice is that PROM scores need to be interpreted via dialogue with the patient to avoid misinterpretation due to DIF and RS. For example, when a choice needs to be made between difference treatment options, patient’s cultural, environmental, personal, or experiential factors need to be taken into account, in a process of shared decision-making [22]. Thus, a key recommendation…

In addition, since this is a knowledge translation paper, the implications in practice are discussed throughout the manuscript including Table 1, and after each of the vignettes.

 

I ask the authors to pay attention to several typos in the text, i.e. number 8 in superscript, lines 92 - 94 text size smaller than the rest.

 

Number 8 superscript has been fixed. Text sizes that are smaller has now been fixed.

Author Response File: Author Response.docx

Reviewer 2 Report

Dear authors

I would like to thank you for giving me the opportunity to review this well-written manuscript. As a person researching cancer care, I enjoyed reading this manuscript. However, I have some suggestions to improve the presentation of your work:

Abstract

The abstract should be more informative about the study and can be consisted of the content about methods, results, and possible conclusions of the study.

Introduction

In the introduction, I would like to see more information regarding radiation oncology and the patients' experience when undergoing RT as a key concept in your work.

Methods

I think you can add the methods section to the manuscript and add relevant contents under the heading of the methods section.

Discussion

Please add recommendations and clinical practice implications of the study and recommendations for future research in this field as well as summarize the key message of the study in one paragraph under the heading of conclusion.

Author Response

Reviewer 2

Abstract

The abstract should be more informative about the study and can be consisted of the content about methods, results, and possible conclusions of the study.

 

Thank you for this feedback. As this was a knowledge translation project, we have added more details of the approach and key recommendation (line 30-32, and line 34-36):

 

“…we aim to review clinical implications and solutions for addressing DIF and RS by providing vignettes from collaborative examinations with workshop participants as well as the literature.”

 

“PROM scores need to be interpreted via dialogue with the patient to avoid misinterpretation due to DIF and RS that could diminish patient-clinician communication and impede shared decision-making.”

 

Introduction

In the introduction, I would like to see more information regarding radiation oncology and the patients' experience when undergoing RT as a key concept in your work.

 

Thank you for this comment. We have added a sentence in the intro about patients undergoing radiation therapy as follows (line 55-57): “For example, patients undergoing radiation therapy have high levels of anxiety and depression that is often under detected and undertreated [6].” And replaced treatment for radiation therapy

Methods

I think you can add the methods section to the manuscript and add relevant contents under the heading of the methods section.

Thank you for this feedback. Thank you for this feedback. As this was a knowledge translation project, we have added a section Approach and described the workshop participants (line 107-114) and the analysis process (line 127-133):

1.2 Approach

The results presented were drawn from discussions from the members of the Clinical Practice and Response Shift Special Interest Groups at the International Society of Quality of Life (ISOQOL) as part of the KT initiative. The members were invited through the ISOQOL listserv and include researchers and clinicians in various practices such as family medicine, neurology and cancer (N=12) who shared a common interest in improving the capture and interpretation of PROMs to support individual-level decisions in clinical practice settings.

 

Extrapolation of the vignettes were guided by a collaborative examination of the discussions in the online workshops focusing on the type of dialogue between Bill and a clinician as well as literature on DIF and RS. Through this review, we identified important aspects of DIF and RS to further explore and identify in the dialogue. For example, we explored differences in meaning that the PROM scores may have for Bill compared to others depending on life situations. In addition, the research team and the workshop participants reviewed the vignettes.

Discussion

Please add recommendations and clinical practice implications of the study and recommendations for future research in this field as well as summarize the key message of the study in one paragraph under the heading of conclusion.

Thank you for this feedback. We have added the implication section and added a new paragraph (line 360-365):

3.1 Implications for practice

The implication in practice is that PROM scores need to be interpreted via dialogue with the patient to avoid misinterpretation due to DIF and RS. For example, when a choice needs to be made between difference treatment options, patient’s cultural, environmental, personal, or experiential factors need to be taken into account, in a process of shared decision-making [22]. Thus, a key recommendation…

In addition, since this is a knowledge translation paper, the implications in practice are discussed throughout the manuscript including Table 1, and after each of the vignettes.

We also included in the conclusion section recommendations for future research and summarized key message of the paper (line 383-390):

3.2 Conclusion

Given the novelty of considering DIF and RS in the context of healthcare decision-making, we need empirical studies to examine under what circumstances DIF and RS affects the types of decisions made in practice. Such studies may also teach clinicians how DIF and RS relate to other known biases due to, for example, social desirability bias. With more empirical data available, we expect to better understand how to account for DIF and RS for more accurate conclusions about the meaning of PROM scores to facilitate improved patient-clinician communication and shared decision-making.

Author Response File: Author Response.docx

Reviewer 3 Report

This is a paper with interesting content that I think is particularly relevant in radiation oncology as a lot of the research in recent years has shifted towards PROMs, and with the shift towards treating oligometastases with high dose resource-intensive radiotherapy that is still investigational (and ensuring pt quality of life is not compromised). Please see specific comments below.

 

Line 107

I feel that more detail should be given on the background of the 12 participants to demonstrate there was adequate radiation oncology representation which I think is important

 

Lines 124-125: After completing radiation to treat painful bone metastases with stage 3 lung cancer, he follows up with his radiation oncologist and completes PROMs during clinic visits

-I assume the authors meant stage 4 lung cancer since there are bone metastases

 

Line 148

-was there supposed to be a case vignette for this example? If so, I think it it should be listed as it would help the reader understand why Bill could have received more aggressive cancer treatment based on his response to the question

 

Line 170: Asking follow-up questions helped Bill clarify that he values quality of life as being more important compared to others who may prefer more years of life, even if those years potentially promise a poorer quality of life.

-"potentially promise" to me is more of a euphemism and I would avoid following it with a negative outcome. I would just truncate to "Asking follow-up questions helped Bill clarify an important distinction".

 

Line 177: Emotional well-being needs

-this might be better changed to just "emotional well-being" or "definition of health"

 

Emotional well-being vignette: Can you tell me more about why you’re dissatisfied or not happy with your health despite not having any symptoms?

-I would remove either "dissatisfied" or "not happy" as they are synonyms

 

Recalibration vignette:

-radiation oncologists typically assess pain on a pain scale out of 10 (and the major studies on efficacy report it this way) which is inverse to an overall health scale out of 10. I feel like this example would be more clearly communicated if a pain scale out of 10 was used as I had to constantly remind myself that the scale being used was inverse to how I normally think about pain

 

Reprioritization vignette:

-it cuts short in the manuscript. Also, assuming this is the same Bill as the RS case before it, radiation therapy for bone metastases is not intended to render pts cancer-free. Radiation for bone metastases also typically has very few side effects and is intended to improve a patients quality of life. This example needs to be changed to someone undergoing more intense curative intent treatment (e.g. radiation for stage 3 lung cancer which does not involve stage 4 bone metastases) or modified in some way

 

Reconceptualization vignette is also cut short in manuscript.

 

Author Response

Reviewer 3  

Line 107

I feel that more detail should be given on the background of the 12 participants to demonstrate there was adequate radiation oncology representation which I think is important

Thank you for confirming the importance of this work. We included more details of workshop participants (line 113-120):

The members were invited through the ISOQOL listserv and include researchers and clinicians in various practices such as family medicine, neurology and cancer (N=12) who shared a common interest in improving the capture and interpretation of PROMs to support individual-level decisions in clinical practice settings.

 

Lines 124-125: After completing radiation to treat painful bone metastases with stage 3 lung cancer, he follows up with his radiation oncologist and completes PROMs during clinic visits

-I assume the authors meant stage 4 lung cancer since there are bone metastases

 

Thank you for pointing this out, we have now changed to stage 4 lung cancer.

 

Line 148

-was there supposed to be a case vignette for this example? If so, I think it it should be listed as it would help the reader understand why Bill could have received more aggressive cancer treatment based on his response to the question

 

This vignette has now been added. It has been cut out during formatting to this journal (see updated lack of scalar invariance example).

 

Line 170: Asking follow-up questions helped Bill clarify that he values quality of life as being more important compared to others who may prefer more years of life, even if those years potentially promise a poorer quality of life. -"potentially promise" to me is more of a euphemism and I would avoid following it with a negative outcome. I would just truncate to "Asking follow-up questions helped Bill clarify an important distinction".

 

Thank you for this suggestion, we have now revised the following sentence to: Asking follow-up questions helped Bill clarify an important distinction that he values quality of life as being more important compared to others who may prefer more years of life.  

 

Line 177: Emotional well-being needs

-this might be better changed to just "emotional well-being" or "definition of health"

 

Thank you for this suggestion. The figure 3 is now “emotional well-being”.

 

Emotional well-being vignette: Can you tell me more about why you’re dissatisfied or not happy with your health despite not having any symptoms?

-I would remove either "dissatisfied" or "not happy" as they are synonyms

For the emotional well-being vignette, the term “dissatisfied” was removed.

Recalibration vignette:

-radiation oncologists typically assess pain on a pain scale out of 10 (and the major studies on efficacy report it this way) which is inverse to an overall health scale out of 10. I feel like this example would be more clearly communicated if a pain scale out of 10 was used as I had to constantly remind myself that the scale being used was inverse to how I normally think about pain

 

Thank you for pointing this out. We have now used a pain scale (instead of health scale) as per your suggestion (see updated recalibration figure 4 and corresponding vignette).

 

Reprioritization vignette:

-it cuts short in the manuscript. Also, assuming this is the same Bill as the RS case before it, radiation therapy for bone metastases is not intended to render pts cancer-free. Radiation for bone metastases also typically has very few side effects and is intended to improve a patients quality of life. This example needs to be changed to someone undergoing more intense curative intent treatment (e.g. radiation for stage 3 lung cancer which does not involve stage 4 bone metastases) or modified in some way

 

The reprioritization vignette has been updated with intense radiation treatment, and Bill’s example has been changed to stage 3 lung cancer (see reprioritization vignette and Bill example).

 

Reconceptualization vignette is also cut short in manuscript.

 

The reconceptualization vignette has been added.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The authors implemented the required revisions well. I have no other comments to add.

Author Response

Thank you for your feedback that there is no additional revisions required. 

Reviewer 2 Report

Dear authors 

Thank you for addressing my comments. 

Author Response

Thank you for your comment that no additional revisions are needed.

Reviewer 3 Report

Reprioritization vignette: this example would be considered very atypical. Patients with stage 3 lung cancer rarely present with pain. I would either move this example to the end of the section and change the scenario so that Bill developed bone metastases after treatment and was receiving palliative-intent radiotherapy to help with pain from those bone metastases; or I would keep it as stage 3 lung cancer and change the symptom to shortness of breath from his lung cancer that improved as the tumor shrank from radiation, but given his relief is not quite adequate, a re-referral to the respirologist (respirologists are often involved in diagnosis as they often do the bronchscopy and biopsy of the tumor) to optimize his breathing (e.g. they will often prescribe inhalers).

Author Response

Thank you for this feedback. The reprioritization vignette does not mention symptoms but rather how Bill has re-prioritized his health towards social functioning. Please let us know if any specific changes are needed for this example or if it is fine as is.