Experienced Effects on Well-Being following Smoking Cessation: Findings from the 2020 ITC Four Country Smoking and Vaping Survey

Round 1

Reviewer 1 Report

This paper addresses a compelling question: Do ex-smokers experience changes in subject assessments of well-being as a result of both the length of time since they quit and whether they continue to use nicotine via electronic cigarettes. To answer the question, the authors draw on a sample of ex-smokers in the 2020 ITC Four Country Smoking and Vaping Survey. They find that, in general, ex-smokers experienced positive changes in well-being after quitting, and that in some cases these changes were dependent on the length of time since quitting. Despite a promising research question, there are significant issues with the paper, some as fundamental as the appropriateness of the survey items the authors use to operationalize particular concepts. As such, I do not believe the manuscript warrants publication in its current form.

General comments:

-The paper does not appear to have undergone editing before submission. There are multiple grammar and wording errors as well as a number of random changes in font/wording size/spacing which made reviewing the substance of the paper substantially more difficult than it otherwise might have been.

-The authors do not provide justification for including “don’t know” responses with “stayed the same” responses. These are not obviously similar responses. Justification should be provided and, as a sensitivity analysis, the authors should include results when “don’t know” responses are not combined with “stayed the same.”

-The end of the paper’s introduction states that the paper aims “to attempt to replicate findings from previous research showing that reported experiences of positive effects of quitting smoking are more common over time since quitting than reported negative experiences of quitting and that reported positives peak after several months quit” (emphasis added). However, only one of the pertinent survey items asks respondents their views about the direct effects of quitting (“daily functioning”). The other three items query respondents about changes since quitting. This raises the possibility of substantial measurement error, as it cannot be assumed that respondents reporting on changes they have experienced since quitting attribute those changes (in part or in whole) to having quit.

-If “Time since quitting smoking” was measured exactly as reported in section 2.2.2, then the response categories survey participants were offered are not mutually exclusive, raising reliability concerns. Two respondents who know they quit “about a year ago,” for example, might report that they quit either “4 months – 1 year” ago or “1 -2 years” ago. The same issue would arise for respondents who believe they quit “about 2 years ago.”

-Given that there may be country-level differences that affect subjective levels of well-being over any given period of time, I wonder if it would be better to include country-level fixed effects rather than a single “country” variable in the models.

-The authors operationalize “perceived past health effects of smoking” with the item “To what extent, if at all, has smoking cigarettes damaged your health?” However, the question does not ask respondents about past health issues. Indeed, a respondent currently dealing with worsening lung cancer from smoking could have truthfully answered “a great deal.” That same respondent could reasonably be expected to report that her enjoyment of life and daily functioning have worsened since quitting, for example.

 

Specific comments:

-I believe Table 1 should be titled, “Duration of Time Quit by Sample Characteristics,” as this reflects the discussion of the table in the text as well as the fact that sample characteristics are treated as the independent variables in the cross-tabulations (i.e., it is the rows, not the columns that sum to 100).

-The discussion of Table 1 seems incomplete given the number of significant differences that could be discussed. The authors could identify, for example, the vaping status and age groups most likely to have quit for 0-3 months and most likely to have quit for 2-5 years, as these time periods represent the bookends of the measure of duration of time quit.

-It was not clear why the authors included/discussed the information in Table 2. If the goal is to show that many of the measures of experienced well-being are related, then should these items not be combined into a single measure? Relatedly, if the authors leave in Table 2, they should explain why they decided to use multiple, separate indicators rather than combining them into a single measure.

-In Table 3, the categories of “Stress coping” in the “Overall %” column sum to nearly 124. The authors should double check these numbers and where they describe them in the text.

-In describing Table 3, the authors write that they “also explored possible interactive effects of time quit for both vaping and NRT use, and as we found no significant effects, report the analyses without the interaction term included.” Table 3 only includes percentages and chi-square tests, however, so I am not sure what interaction terms would have/could have been included. Does this comment belong in the description of Table 4?

-Several P values in Table 3 appear as “p<.000” but are discussed in the text as “p<.001.”

-The text describing Table 3 refers to NRT use, but nothing about NRT use appears in the table.

-The paragraph that begins “Overall, among those quit for less than 2 years…” does not appear to refer to information contained in Table 3. Should this information have been included in the table?

-Table 4 appears to contain only multivariable models. I am not sure what the authors mean when they say, “As for the bivariate results…” in the second sentence in the section describing the results from Table 4.

-Unless I am misreading Table 4, individuals who had quit 1-2 years prior saw improvement in negative affect coping and daily functioning. Those who quit 2-5 years prior saw improvement in enjoyment of life. This does not seem to match the description of these results in the text.

-Authors should refrain from interpreting the covariates they’ve included in their models, as their goal was not to model the effect of any of these covariates on the study outcomes. If it were, one might imagine it would be necessary to include other potential confounders/covariates in those models before we could reasonably discuss their impact on the outcomes of interest.  

Author Response

General comments:

-The paper does not appear to have undergone editing before submission. There are multiple grammar and wording errors as well as a number of random changes in font/wording size/spacing which made reviewing the substance of the paper substantially more difficult than it otherwise might have been.

The authors do not provide justification for including “don’t know” responses with “stayed the same” responses. These are not obviously similar responses. Justification should be provided and, as a sensitivity analysis, the authors should include results when “don’t know” responses are not combined with “stayed the same.”

Response:-:  We now report additional analysis demonstrating that this does not affect the Results in any material way.  The two were combined as numbers of Don’t knows were modest and both responses demonstrate a lack of directional change

-The end of the paper’s introduction states that the paper aims “to attempt to replicate findings from previous research showing that reported experiences of positive effects of quitting smoking are more common over time since quitting than reported negative experiences of quitting and that reported positives peak after several months quit” (emphasis added). However, only one of the pertinent survey items asks respondents their views about the direct effects of quitting (“daily functioning”). The other three items query respondents about changes since quitting. This raises the possibility of substantial measurement error, as it cannot be assumed that respondents reporting on changes they have experienced since quitting attribute those changes (in part or in whole) to having quit.

Response:  The reviewers concern is misplaced.  It is true inference of causality is suspect, and at the individual level.  We were not asking about the respondents perceptions of causality, merely about  changes.  It does not matter if this is since quitting or in any given period.  What we get is the perception of change, some no doubt is due to factors other then quitting, but it is reasonable to suppose that this is equivalent across the two types of quitters, As should be clear we discuss the viability of this proposition in the discussion.  We make no claim as to absolute level os benefit in large part because we know the net benefit is affected by other causes as and this applied equally for perceived improvements and decrements. 

-If “Time since quitting smoking” was measured exactly as reported in section 2.2.2, then the response categories survey participants were offered are not mutually exclusive, raising reliability concerns. Two respondents who know they quit “about a year ago,” for example, might report that they quit either “4 months – 1 year” ago or “1 -2 years” ago. The same issue would arise for respondents who believe they quit “about 2 years ago.”

Response:  There is always some error in measurement of recalled events.  The actual questions asked provided greater detail, indeed for those reporting having quit on a previous wave and remaining quit, the interwave interval was added to their previous estimate.   In practice, distinguishing between up to 1 year and 1-2 years as compared with up to 1 year and over 1 year but less than 2 makes little difference, and will not affect the comparative analyses. In reality there was no overlap, and we added “>” to indicate where the range does not include the lower bound.

-Given that there may be country-level differences that affect subjective levels of well-being over any given period of time, I wonder if it would be better to include country-level fixed effects rather than a single “country” variable in the models.

Response: In past studies, this has not made an difference, but we have not tested it here and see no reason to do so.

-The authors operationalize “perceived past health effects of smoking” with the item “To what extent, if at all, has smoking cigarettes damaged your health?” However, the question does not ask respondents about past health issues. Indeed, a respondent currently dealing with worsening lung cancer from smoking could have truthfully answered “a great deal.” That same respondent could reasonably be expected to report that her enjoyment of life and daily functioning have worsened since quitting, for example.

 Response: The reviewer is correct some will report reduced health because of diseases thy may have already had as well as any new diseases, but this is not a problem. Most respondents  Again our focus is on differences between the two types of quitting.

Specific comments:

-I believe Table 1 should be titled, “Duration of Time Quit by Sample Characteristics,” as this reflects the discussion of the table in the text as well as the fact that sample characteristics are treated as the independent variables in the cross-tabulations (i.e., it is the rows, not the columns that sum to 100).

Response:  As we understand it this is not a fixed convention and prefer the focal uon thee “more focal” variable

-The discussion of Table 1 seems incomplete given the number of significant differences that could be discussed. The authors could identify, for example, the vaping status and age groups most likely to have quit for 0-3 months and most likely to have quit for 2-5 years, as these time periods represent the bookends of the measure of duration of time quit.

Response:  We agree we could have extended the Discussion, but not within reasonable word limits.  We thought about commenting on the Country effects, but as we have no clear theory as to why the observed differences, this would not have been very illuminating for readers, so in the interests of brevity, we chose not to.

-It was not clear why the authors included/discussed the information in Table 2. If the goal is to show that many of the measures of experienced well-being are related, then should these items not be combined into a single measure? Relatedly, if the authors leave in Table 2, they should explain why they decided to use multiple, separate indicators rather than combining them into a single measure.

Response: We agree a case could be made for combining the two coping measures, but as our original plan was to teat them separately and they were not included in the study as a scale, we chose to present them separately.

-In Table 3, the categories of “Stress coping” in the “Overall %” column sum to nearly 124. The authors should double check these numbers and where they describe them in the text.

Response: Our error, which we have corrected

-In describing Table 3, the authors write that they “also explored possible interactive effects of time quit for both vaping and NRT use, and as we found no significant effects, report the analyses without the interaction term included.” Table 3 only includes percentages and chi-square tests, however, so I am not sure what interaction terms would have/could have been included. Does this comment belong in the description of Table 4?

Response:  We now make it clearer that the NRT results were not in the Table, but were included in the multivariate analyses and showed no effects.

-Several P values in Table 3 appear as “p<.000” but are discussed in the text as “p<.001.”

Response: Corrected

-The text describing Table 3 refers to NRT use, but nothing about NRT use appears in the table.

RE: Sorry about the confusion. Neither the results on NRT use nor the reporting of positive/negative/no effects among those quit for various durations were part of Table 3. We now make this clearer at the beginning of the paragraph describing these results.

-The paragraph that begins “Overall, among those quit for less than 2 years…” does not appear to refer to information contained in Table 3. Should this information have been included in the table?

RE: See the response to the previous comment.

-Table 4 appears to contain only multivariable models. I am not sure what the authors mean when they say, “As for the bivariate results…” in the second sentence in the section describing the results from Table 4.

RE: Thanks for pointing this out. We have now deleted “As for the bivariate results in,” from the sentence.

-Unless I am misreading Table 4, individuals who had quit 1-2 years prior saw improvement in negative affect coping and daily functioning. Those who quit 2-5 years prior saw improvement in enjoyment of life. This does not seem to match the description of these results in the text.

RE: In the text we looked at the pattern.  A problem with reliance on p values in multivariate analyses is that  non-significant effects can be very close to significant ones in effect size, and thus where conceptually related, it is important consider the pattern, not each individual effect in isolation.

-Authors should refrain from interpreting the covariates they’ve included in their models, as their goal was not to model the effect of any of these covariates on the study outcomes. If it were, one might imagine it would be necessary to include other potential confounders/covariates in those models before we could reasonably discuss their impact on the outcomes of interest.  

RE: We disagree about refraining from discussing covariates, but agree caution is required, in considering whether these are properly controlled for, something we believe we did.

 

 

Reviewer 2 Report

This study tries to replicate at least two previous studies the authors already published. I don't see originality in research here. The upfront should better justify the reasons/contribution of the current study so readers of this journal would get something new from the findings. Authors include various measures including persistent worrying, but do not flesh out the psychological impact of worrying/anxiety on the main outcomes of the study.

I am also concerned with the data handling of "don't know" for the survey responses. Authors did a number of sensitivity tests, but none pertain to the treatment of "don't know." Authors essentially neutralized "don't know" - but not knowing is not same as being neutral ("stayed the same"). It is unclear the impact of re-assignment "don't know's" because authors did not provide distribution of the survey response. What happens if you exclude the "don't knows" from the analysis?

Author Response

This study tries to replicate at least two previous studies the authors already published. I don't see originality in research here. The upfront should better justify the reasons/contribution of the current study so readers of this journal would get something new from the findings. Authors include various measures including persistent worrying, but do not flesh out the psychological impact of worrying/anxiety on the main outcomes of the study.

I am also concerned with the data handling of "don't know" for the survey responses. Authors did a number of sensitivity tests, but none pertain to the treatment of "don't know." Authors essentially neutralized "don't know" - but not knowing is not same as being neutral ("stayed the same"). It is unclear the impact of re-assignment "don't know's" because authors did not provide distribution of the survey response. What happens if you exclude the "don't knows" from the analysis?

Response;  We disagree with this reviewer regarding replication.  Our study is a partial replication and extension.  Replication is critical to confirming relationships and gives increased confidence in their reality.  It also provides support that the extensions are likely to be common as they occur in studies which have been shown (through the replication parts) to be demonstrating robust relationships.

As noted above, the relationships found are robust to dropping the “Don’t know” respondents

Reviewer 3 Report

It is an interesting study exploring data received from an international web-based survey covering 4 countries. The Authors have elaborated quite intensively on the topic of the study in the Introduction. The study design does not raise reservations. The results are presented rather clearly. The Discussion is adequate and supported by appropriate literature.

Overall, the paper is well-conceived and transparent as for the main parts.

Table 4 is the only part that would require some amendments as below:

·        Table 4 – why is the value for ‘daily functioning” not provided for ‘2-5 years’; instead, there is ‘NA”?

·        The information below Table 4 – ‘the most extreme country making…’ – what does it mean concerning Canada

·        It seems that the Authors are inconsistent in providing p values, e.g., below table 4, ‘p<0.05’ and ‘p<.001’. In some places, the p was also provided as .0.000.

·        What is ‘Cis’? – also, in the text below, table 4.

 In general, the manuscript's language is difficult to follow for non-native speakers.

 

Author Response

It is an interesting study exploring data received from an international web-based survey covering 4 countries. The Authors have elaborated quite intensively on the topic of the study in the Introduction. The study design does not raise reservations. The results are presented rather clearly. The Discussion is adequate and supported by appropriate literature.

Overall, the paper is well-conceived and transparent as for the main parts.

Table 4 is the only part that would require some amendments as below:

• Table 4 – why is the value for ‘daily functioning” not provided for ‘2-5 years’; instead, there is ‘NA”?

RE: Thanks for pointing this out. We have now added the following to the notes of the table: NA: not available (Note: Daily functioning was only asked among those who quit within the last 2 years).

• The information below Table 4 – ‘the most extreme country making…’ – what does it mean concerning Canada

RE: It means Canada is the country with the highest adjusted ORs across all the measures of perceived effects of quitting. We have now made this clearer in the notes of the table.

• It seems that the Authors are inconsistent in providing p values, e.g., below table 4, ‘p<0.05’ and ‘p<.001’. In some places, the p was also provided as .0.000.

RE: We have now made this consistent throughout the manuscript.

• What is ‘Cis’? – also, in the text below, table 4.

RE: Sorry that was a typo. It stands for confidence intervals (CIs).We have now corrected it.

 In general, the manuscript's language is difficult to follow for non-native speakers.

RE:  The first author is a non-native speaker of English.  We have reviewed and made a fewminor changes to try to improve readability.

We have replaced p values of 0.000 with “p<.001” 

Reviewer 4 Report

This manuscript aimed to assess well-being related positive and negative experiences after quitting smoking over time in a sample of complete nicotine and tobacco quitters as well as vapers who vape nicotine after quitting smoking. In overall, this manuscript is well written, the topic is interesting and has practical implication in nicotine and tobacco cessation counselling.

The Introduction and Discussion is quite long and it is a bit hard to understand the most important messages of this paper. Therefore, I suggest that the Authors should rewrite the text of these sections to be more concise. Please also consider writing more clearly and precisely emphasizing the most important findings and their practical implications in the Discussion and Conclusions.

I think that the term “time quit” makes it difficult to clearly understand the text. Please consider rewording this term in the whole text, e.g., time since quitting.

Unfortunately the manuscript’s rows were not numbered, therefore it is a bit complicated to provide my detailed comments.

 

Specific comments

Abstract

“The largest improvement here (56.3%) was…” Please define “here”. What does it refers to?

 

Introduction

2^nd page, 5^th paragraph, last sentence: “Controlling for time quit, reported improvements since quit-ting were not associated with reduced relapse,…” a citation is missing at the end of this sentence.

3^rd page, 2^nd paragraph, sentence “Vaping provides an alternative source of nicotine albeit likely less effectively than from a cigarette, …” I argue with this as recent studies found that novel generation devices have as effective nicotine delivery as conventional cigarettes. Please revise the text according to the relevant literature.

3^rd page, 2nd paragraph, sentence „Nicotine is a stimulant and for some people appears to have cognitive enhancing benefits (15), so maintenance of nicotine in these persons may be important…” For how long do you mean maintaining nicotine after quitting smoking? Please revise the text as continuous and/or unlimited maintenance of nicotine intake is not a favorable outcome of smoking cessation as it still maintain addiction and could lead to nicotine-related health harms.

3^rd page, 5^th paragraph, 1^st sentence “We might expect those who find…” This sentence is a bit confusing. Please revise it and make it clearer and easier to read.

 

Materials & Methods

4^th page, 1^st paragraph: Wave 3 survey was conducted between “February-June 2022”, but this manuscript was submitted to IJERPH at the end of May. Please revise the data collection time interval or give an explanation for June.

4^th page, subsection 2.2.3. Other measures: at socio-demographics, please consider changing “gender” to “sex” in the text. In the tables, the term “sex” is used. Please consider the SAGER guideline of IJERPH’s Instructions for Authors.

 

Results

5^th page, 3^rd paragraph: “For all four well-being measures there were increases in positive relative to negative effects from 0-3 months…” It is confusing what “increases in positive to negative effects” means. Please revise the text.

6^th page, Table 1: In the cell “Overall”, include an “n” between the text and the sample size. In column “Significance”, please insert an Enter between the χ² value and the p-value in row „Country”.

8^th page, Table 3: Please edit the table. Currently, the distance between “improved” and “same” categories is smaller than between the values belonging to these rows.

9^th page, Table 4: Please edit the table, related aORs and 95%CIs are better to be in one row. In Table 4 notes under the table, please correct “(Cis)” to “(CIs)”.

11^th page, 1^st paragraph: please unify writing of p-value, that is, p<.001 or p<0.001 (I prefer using the latter according to the text).

11^th page, 3^rd paragraph: it is unclear whether there is any table for this paragraph. In Table 3, such results could not be identified. If there is no table for this paragraph, please indicate it in the text.

 

Discussion

13^th page, 1^st paragraph: “…so there is no evidence of any negative effect of regular vaping post quitting on well-being.” Please consider a slight change in the text, that is, insert “…no evidence of short and medium term negative effect…”. Detecting possible long term effects of vaping on well-being is impossible based on this study (considering time since quitting is maximum 5 years), but it would be important to emphasize that this study did not explored short and/or medium term negative effects of vaping on well-being.

In this same paragraph, regarding sentence “…it suggests that there are more likely to be net benefits on well-being of continuing to vape after smoking cessation.”, please consider being more cautious whit this statement. On the one hand, we do know nothing about the daily vaping habits (e.g., intensity, type of device, nicotine concentration of the e-liquid) of respondents, and whether they have used it as a consumer product or was offered by a health professional with medical supervision. Therefore, the Authors should be more careful to draw such strong conclusion based on results with serious limitations.

Also in the same paragraph, in the last sentence, please amend the text of “there are no differential experienced adverse effects of regular vaping” with “on mental well-being”, that is, “adverse effects of regular vaping on mental well-being”.

13^th page, 3^rd paragraph: sentence “were less common among “daily vaping quitters” suggests that vaping may be a means to reduce these negative effects for some at least.” Please consider wording more nuanced. We have no information about whether other cessation aids/supports were used or whether these were used adequately which might influence experiencing negative or positive effects after quitting smoking.

Author Response

This manuscript aimed to assess well-being related positive and negative experiences after quitting smoking over time in a sample of complete nicotine and tobacco quitters as well as vapers who vape nicotine after quitting smoking. In overall, this manuscript is well written, the topic is interesting and has practical implication in nicotine and tobacco cessation counselling.

The Introduction and Discussion is quite long and it is a bit hard to understand the most important messages of this paper. Therefore, I suggest that the Authors should rewrite the text of these sections to be more concise. Please also consider writing more clearly and precisely emphasizing the most important findings and their practical implications in the Discussion and Conclusions.

I think that the term “time quit” makes it difficult to clearly understand the text. Please consider rewording this term in the whole text, e.g., time since quitting.

RE:  This would add considerably to length as the term is used a lot, but we now define time-quit in these terms where we first use it

 

Unfortunately the manuscript’s rows were not numbered, therefore it is a bit complicated to provide my detailed comments.

 

Specific comments

Abstract

“The largest improvement here (56.3%) was…” Please define “here”. What does it refers to?

RE:  We have now made the sentence clear 

Introduction

2^nd page, 5^th paragraph, last sentence: “Controlling for time quit, reported improvements since quit-ting were not associated with reduced relapse,…” a citation is missing at the end of this sentence.

RE: No this is a continuation of the point previously referenced

3^rd page, 2^nd paragraph, sentence “Vaping provides an alternative source of nicotine albeit likely less effectively than from a cigarette, …” I argue with this as recent studies found that novel generation devices have as effective nicotine delivery as conventional cigarettes. Please revise the text according to the relevant literature.

RE: We have added a “perhaps”, while some studies do show comparable biological responses, this is not consistent and some smokers report inferior experiences.  Fully referencing this point would require several references, which we don’t think would add value.

3^rd page, 2nd paragraph, sentence „Nicotine is a stimulant and for some people appears to have cognitive enhancing benefits (15), so maintenance of nicotine in these persons may be important…” For how long do you mean maintaining nicotine after quitting smoking? Please revise the text as continuous and/or unlimited maintenance of nicotine intake is not a favorable outcome of smoking cessation as it still maintain addiction and could lead to nicotine-related health harms.

RE: The reviewer is correct, persistent use may have adverse effects, but likely far less than from continuing or relapsing back to smoking, so we think a legitimate, if not ideal, second line strategy for those who cannot stay quit smoking without the nicotine.  We clarify what we said along these lines

3^rd page, 5^th paragraph, 1^st sentence “We might expect those who find…” This sentence is a bit confusing. Please revise it and make it clearer and easier to read.

RE:  Revised, and we think clearer now.

 

Materials & Methods

4^th page, 1^st paragraph: Wave 3 survey was conducted between “February-June 2022”, but this manuscript was submitted to IJERPH at the end of May. Please revise the data collection time interval or give an explanation for June.

RE: Sorry that was a typo. It should be “February-June 2020 “. We have now corrected it. Thanks for your careful reading.

4^th page, subsection 2.2.3. Other measures: at socio-demographics, please consider changing “gender” to “sex” in the text. In the tables, the term “sex” is used. Please consider the SAGER guideline of IJERPH’s Instructions for Authors.

RE: Done.

 

Results

5^th page, 3^rd paragraph: “For all four well-being measures there were increases in positive relative to negative effects from 0-3 months…” It is confusing what “increases in positive to negative effects” means. Please revise the text.

RE: We have clarified and spelled out the pattern in more detail.

6^th page, Table 1: In the cell “Overall”, include an “n” between the text and the sample size. In column “Significance”, please insert an Enter between the χ² value and the p-value in row „Country”.

RE: Done.

8^th page, Table 3: Please edit the table. Currently, the distance between “improved” and “same” categories is smaller than between the values belonging to these rows.

9^th page, Table 4: Please edit the table, related aORs and 95%CIs are better to be in one row. In Table 4 notes under the table, please correct “(Cis)” to “(CIs)”.

11^th page, 1^st paragraph: please unify writing of p-value, that is, p<.001 or p<0.001 (I prefer using the latter according to the text).

RE: Done.

11^th page, 3^rd paragraph: it is unclear whether there is any table for this paragraph. In Table 3, such results could not be identified. If there is no table for this paragraph, please indicate it in the text.

RE: Neither the results on NRT use nor the reporting of positive/negative/no effects among those quit for various durations were part of Table 3. We have now made this clearer at the beginning of the paragraph describing these results.

 

Discussion

13^th page, 1^st paragraph: “…so there is no evidence of any negative effect of regular vaping post quitting on well-being.” Please consider a slight change in the text, that is, insert “…no evidence of short and medium term negative effect…”. Detecting possible long term effects of vaping on well-being is impossible based on this study (considering time since quitting is maximum 5 years), but it would be important to emphasize that this study did not explored short and/or medium term negative effects of vaping on well-being.

RE:  Change made

 

In this same paragraph, regarding sentence “…it suggests that there are more likely to be net benefits on well-being of continuing to vape after smoking cessation.”, please consider being more cautious whit this statement. On the one hand, we do know nothing about the daily vaping habits (e.g., intensity, type of device, nicotine concentration of the e-liquid) of respondents, and whether they have used it as a consumer product or was offered by a health professional with medical supervision. Therefore, the Authors should be more careful to draw such strong conclusion based on results with serious limitations.

RE:  The statement is a qualified suggestion, not a conclusion.  With the above qualification added, this might be clearer

Also in the same paragraph, in the last sentence, please amend the text of “there are no differential experienced adverse effects of regular vaping” with “on mental well-being”, that is, “adverse effects of regular vaping on mental well-being”.

RE:  We now include mental well-being explicitly in the revised sentence.

 

13^th page, 3^rd paragraph: sentence “were less common among “daily vaping quitters” suggests that vaping may be a means to reduce these negative effects for some at least.” Please consider wording more nuanced. We have no information about whether other cessation aids/supports were used or whether these were used adequately which might influence experiencing negative or positive effects after quitting smoking.

RE:  We used “may be” to indicate this was speculative.  We think this is enough. 

 

Overall response: We thank this reviewer in particular for their very thorough reading.  We believe the manuscript has been enhanced markedly by his/her suggestions and those of the other Reviewers.

 

 

Round 2

Reviewer 1 Report

-The authors respond that my concerns that some of the items they employ are not actually measuring the concepts they assert are misplaced because they “were not asking about the respondents’ perceptions of causality, merely about changes.” However, this is precisely the issue: throughout the paper, the authors assert that they are indeed measuring respondents’ perceptions of the causal impact of quitting smoking when their survey items do not permit them to do so. To cite two such examples:

·        The intro claims that the goal of the paper is to replicate previous findings “showing that reported experiences of positive effects of quitting smoking are more common over time since quitting than reported negative experiences of quitting…” (emphasis added)

·        In the discussion of the results concerning two measures of coping, the authors write “Indeed, about 60% thought quitting had no impact on either.” (emphasis added)

The authors further contend that it is reasonable to assume that any error introduced by the fact that some of their items ask about changes since quitting, rather than changes due to quitting, should be equivalent across both types of quitters. Why? Respondents were not randomly assigned to be one type of quitter or another. What mechanism would lead us to expect that the propensity to interpret these items differently than the authors do would be equal in both groups?

-On a similar note, the authors assert that it is not a problem that their measure of perceived past health effects of smoking does not actually ask respondents about the past. They contend that they are simply interested in differences between the two types of quitters. However, the authors’ interests do not erase issues of measurement validity. The authors say they are measuring a construct, perceived past health effects of smoking, but use an item that is likely capturing some mixture of both past and present health effects. While survey researchers are often constrained by the items that are available in a survey, they are not forced to describe them as though they are measuring one thing when they may be measuring another.

-Some of the authors’ responses to my comments about the analyses they employ are along the lines of “we don’t think we need to do this” (e.g., their treatment of Table 1, the combination of the coping measures). While this is certainly their choice, I included the comments I did because I believe that considering them will strengthen the paper. As the authors provided little to no substantive justification for their choices not to consider my feedback, this is still my belief.

Author Response

The authors respond that my concerns that some of the items they employ are not actually measuring the concepts they assert are misplaced because they “were not asking about the respondents’ perceptions of causality, merely about changes.” However, this is precisely the issue: throughout the paper, the authors assert that they are indeed measuring respondents’ perceptions of the causal impact of quitting smoking when their survey items do not permit them to do so. To cite two such examples:

• The intro claims that the goal of the paper is to replicate previous findings “showing that reported experiences of positive effects of quitting smoking are more common over time since quitting than reported negative experiences of quitting…” (emphasis added)
• In the discussion of the results concerning two measures of coping, the authors write “Indeed, about 60% thought quitting had no impact on either.” (emphasis added).

Response: Firstly, we did attempt to take the Reviewers comments seriously , but as can be seen below, accept we missed the point in a couple of cases.  

We now understand the Reviewers concern. We were not interested in smokers’ causal beliefs as such, as we tried to make clear. We now realize that your problem was with the “of” in the aims; ie, it implies we were interested in smokers perceived causality.  We now change this to “following”.  Sorry about our misunderstanding. NB  Only one aim was replication.

Causal beliefs are often wrong and provide at best low value evidence as to likely actual causality.  We also think we have made clear that possible effects differ within individuals and on some measures most perceive no changes. We have further qualified the final conclusion to make doubly sure our conclusions are not seen as definitive.

The authors further contend that it is reasonable to assume that any error introduced by the fact that some of their items ask about changes since quitting, rather than changes due to quitting, should be equivalent across both types of quitters. Why? Respondents were not randomly assigned to be one type of quitter or another. What mechanism would lead us to expect that the propensity to interpret these items differently than the authors do would be equal in both groups?

Response:  We spend a considerable portion of the Discussion questioning whether the two groups are comparable and indeed conclude we cannot be sure they are equivalent, but that some of the differential differences are difficult to explain by differences in characteristics of the samples other than due to their vaping status (ie differences in past quit experiences).  Related to this, the problems of attributing causality are the same however the questions are asked.

-On a similar note, the authors assert that it is not a problem that their measure of perceived past health effects of smoking does not actually ask respondents about the past. They contend that they are simply interested in differences between the two types of quitters. However, the authors’ interests do not erase issues of measurement validity. The authors say they are measuring a construct, perceived past health effects of smoking, but use an item that is likely capturing some mixture of both past and present health effects. While survey researchers are often constrained by the items that are available in a survey, they are not forced to describe them as though they are measuring one thing when they may be measuring another.

Response: The Reviewer misunderstands or we misunderstood them.  Of course, our question is about the past and any current problems attributed to smoking. We used past to differentiate it from future, but now make clear in the Methods that it includes current.  As to its validity, we find it makes differences to some of the answers in ways we would expect, thus we demonstrate some predictive validity.  For example, those reporting past (or current) negative effects were more likely to report improvements in functioning as would be expected., based on any believing no past or current at the time of quitting adverse health effects would be less likely to find improvements.

Some of the authors’ responses to my comments about the analyses they employ are along the lines of “we don’t think we need to do this” (e.g., their treatment of Table 1, the combination of the coping measures). While this is certainly their choice, I included the comments I did because I believe that considering them will strengthen the paper. As the authors provided little to no substantive justification for their choices not to consider my feedback, this is still my belief.

Response:  We explained the decision not to combine the two coping measures as being due to the original choice to have them was not explore any differential effects.  The two coping measures were certainly correlated enough to justify combining them.  We have now run the analyses and report them briefly as subsidiary analyses.  They don’t seem to tell us anything clearly new.