Exploring the Engaged Worker over Time—A Week-Level Study of How Positive and Negative Work Events Affect Work Engagement

Round 1

Reviewer 1 Report

Congratulations on a job well done. Both the reasons given and the modifications made have dispelled my initial doubts. 

Reviewer 2 Report

Thank you to the authors for their revisions to their original submission.

While I could offer criticisms about some aspects of the revised paper, I do not think that doing so at this time would be productive and, quite frankly, I think that the authors have done that best that can be expected with the manuscript. As such, I see no reason why the paper should not be accepted for publication. 

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

I am happy to have the opportunity to read and review the manuscript “Exploring the vital worker over time- A week-level study on how positive and negative work events contribute to affect and sustain work engagement”. The authors of this manuscript analyze the influence of affective events (positive and negative) on employee work engagement using a week-level design. This design allows them to analyze various patterns of interactive effects between affective events measured both at the same time and at different times.

I think the manuscript is conscientiously drafted and has the potential to make a non-trivial contribution to the literature. The paper is well written and demonstrates a great knowledge of the state of the art. The front part is concise and to-the-point. The research question is interesting, and the topic is relevant. Moreover, I do not have any suggestion about the hypotheses development or the discussion. My comments focus on the measures, and more specifically, in the demonstration of their validity and reliability.

With regard to the work events measure, I agree with the authors that the reliability of the formative constructs cannot be valued by means of Cronbach's alpha. However, it is necessary that they provide information that proves the validity and reliability of these constructs. While the specific indexes to be provided will depend on the approach employed, the review of Diamantopoulos' work on formative constructs may be useful for this purpose. For example, in the case of using SEM for this check they could follow what is indicated by Diamantopoulos (2011).

Diamantopoulos, A. (2011). Incorporating formative measures into covariance-based structural equation models. Mis Quarterly, 335-358.

With respect to the work engagement measure, the selection of items does not seem to me very successful since it is left out the vigor dimension, which together with the dedication are considered the main dimensions of work engagement. I would recommend that in future studies the 9 items be used or at least some vigor dimension items be collected.  Also, in the case of work engagement it is necessary that the authors go beyond the Cronbach alpha to prove their reliability and validity. As it is a reflective construct, I suggest that the authors follow what is indicated by Hair et al. (2010) p. 661-699.

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). Prentice Hall, Englewood Cliffs.

A minor issue that authors should review is the format of citations, since there are several mistakes throughout the manuscript (for example: in lines 195, 243, 254, 697 and 708).

Also, in line 183, they should change "Hypothesis 2" to "Hypothesis 1".

In conclusion, while I believe that the authors have done a good job, I consider that the evidence regarding the validity and reliability of their constructs will make this paper more robust. I wish the authors the best of luck in improving the paper.

Reviewer 2 Report

This manuscript reports on an interesting study that explores positive and negative work events and engagement. I applaud the aspirations represented in this paper. However, both formal and content aspects of the manuscript must be revised.

I hope the suggestions I give below will support you in advancing your research efforts on this topic. Following are my specific comments on this paper.

The title captures the reader’s attention and clearly informs the reader about the contents of the article. I have no suggestions for improving the title.

More information on the discussion could be included in the abstract.

The introduction section provides enough background to understand the rest of the paper and to justify the study hypothesis. However, the authors should detail more the contributions of their study.

Appropriate identification of participants is critical and this study has a questionable sampling method. The authors should detail the type of sampling, why this type of sampling is the best, what risks it has for its validity and how they have been solved so that they do not affect the result obtained.

The results are clearly laid out in a logical sequence and the discussion integrates the results with past published research and provides a clear explanation of the results, including their importance.

Reviewer 3 Report

This manuscript reports on the results of a longitudinal experience sampling study aimed at examining how past and current positive and negative work events impact current work engagement.

I found the study to be generally interesting and the manuscript to be fairly well written. I credit the authors for trying to incorporate temporal dynamics into an investigation on factors that contribute to work engagement. Nevertheless, I have a number of questions, issues, and concerns which lead me to recommend against publication of this manuscript.  Below are some of my more substantive points of feedback for the authors and editor to consider.

(1) Up front, I have some concerns about research ethics and the extent to which this manuscript has been prepared in accordance with journal policy.

On page 8, the following is stated in the participants subsection--"The procedure and materials of this study have not undergone examination by an ethics committee, as the measures and procedures of our study followed the protocols of standard self-report experience sampling research in applied psychology, and we did not touch sensitive topics (like e. g. sexual orientation). Our study fully complied with the standards of the Department of Psychology at the University of Hagen, which included strict guidelines to guarantee anonymity of the self-reported data."

From how this statement reads, it seems as though the authors decided on their own that their study did not present any risks to participants and that there was no consultation with any ethics body to support this claim. Furthermore, there is nothing stated about obtaining informed consent from participants prior to study participation.

Perhaps more importantly, however, is the fact that what the authors state does not appear to be consistent with IJERPH manuscript requirements.

The following is the verbatim text from the IJERPH website located at the following link: https://www.mdpi.com/journal/ijerph/instructions#ethics

"Research Involving Human Subjects

When reporting on research that involves human subjects, human material, human tissues, or human data, authors must declare that the investigations were carried out following the rules of the Declaration of Helsinki of 1975 (https://www.wma.net/what-we-do/medical-ethics/declaration-of-helsinki/), revised in 2013. According to point 23 of this declaration, an approval from an ethics committee should have been obtained before undertaking the research. At a minimum, a statement including the project identification code, date of approval, and name of the ethics committee or institutional review board should be cited in the Methods Section of the article. Data relating to individual participants must be described in detail, but private information identifying participants need not be included unless the identifiable materials are of relevance to the research (for example, photographs of participants’ faces that show a particular symptom). Editors reserve the right to reject any submission that does not meet these requirements.

Example of an ethical statement: "All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of XXX (Project identification code)."

A written informed consent for publication must be obtained from participating patients who can be identified (including by the patients themselves). Patients’ initials or other personal identifiers must not appear in any images. For manuscripts that include any case details, personal information, and/or images of patients, authors must obtain signed informed consent from patients (or their relatives/guardians) before submitting to an MDPI journal. Patient details must be anonymized as far as possible, e.g., do not mention specific age, ethnicity, or occupation where they are not relevant to the conclusions. A template permission form is available to download. A blank version of the form used to obtain permission (without the patient names or signature) must be uploaded with your submission.

You may refer to our sample form and provide an appropriate form after consulting with your affiliated institution. Alternatively, you may provide a detailed justification of why informed consent is not necessary. For the purposes of publishing in MDPI journals, a consent, permission, or release form should include unlimited permission for publication in all formats (including print, electronic, and online), in sublicensed and reprinted versions (including translations and derived works), and in other works and products under open access license. To respect patients’ and any other individual’s privacy, please do not send signed forms. The journal reserves the right to ask authors to provide signed forms if necessary."

I am of the opinion that this manuscript does not comply with this journal requirement and, more generally, does not conform with best ethical research practices.

On this basis alone, I do not think that this manuscript should be given consideration for publication.

Ostensibly, the authors need to address this issue and the editor needs to decide if the manuscript (whether in its current form or in a revised form) sufficiently satisfies journal requirements for research ethics.

(2) Data quality strikes me as poor for a variety of reasons. First, there is a tremendous amount of missing data.

Second, there is no clear statement as to how many participants completed at least one questionnaire per week for the entirety of the 15 weeks. Instead, the authors only make mention of the number of participants who completed at least one questionnaire for two consecutive weeks. This number appears to be 101 participants. In the results, the authors make mention of the number of participants who completed at least 8, 10, and 12 questionnaires, respectively. In all cases, the sample size is small (in fact n= 20 with participants who completed the questionnaires at least 12 times). However, there is no information given (a) on when, during the course of the 15 weeks, these participants completed the questionnaire for two consecutive weeks. Since the participants were active distance-learning students during the time of data acquisition, it seems likely that the timing of students completing the questionnaires may be associated with time periods with differing academic demands (e.g., participants may have been more likely to complete questionnaires when it became apparent to them that they needed the extra credit toward their course grade, such as after the completion of mid-term examinations). Alternatively, participants may have only responded when they perceived having experienced work events that conformed to the needs of the study. In both cases, potential bias is introduced into the data, (b) how many of the 101 participants completed the minimum number of 10 questionnaires to qualify for course credit, and (c) whether or not one or more of the authors had any direct or indirect involvement with any of the participants in an instructional or advisory capacity. With this latter point, potential ethical issues are raised (e.g., if one or more of the researchers had evaluative authority over any participants, then this could unduly influence their willingness to participate and complete questionnaires).

Third, the employment characteristics of the sample seem quite heterogeneous with the sample including participants who were employed full-time and part-time and who were employed in a variety of work settings with a wide of range of years of experience in each work setting. Though such heterogeneity may be seen as contributing to external validity of the study, it has a markedly deleterious effect on internal validity.

Based on what the authors report about their sampling and data collection and, more importantly , on what data they actually used in their analyses,  the study reported in the manuscript is a poor example of longitudinal research as it only considered consecutive two-week questionnaires in the main analyses without giving consideration to, or seemingly controlling for, differences associated with when the questionnaires were completed, type of employment setting, and whether or not employment was full vs. part-time. The amount of missing data is not only beyond excessive, but they are also (obviously) not missing at random.

(4) All measures/data were self-report. To the authors' credit, they do acknowledge this as a limitation.  

(5) While the analyses that were completed indicate that the authors are quite statistically savvy, the use of multimulti-level growth modeling and the completion of several supplementary analyses does not compensate for poor data. In essence, the authors were modeling data that are little more than noise in my opinion. 

(6) I was surprised and disappointed that the authors did not give more consideration to the potential influence of personality factors on the relation of work events to work engagement. Traits such as agreeableness and conscientiousness (in the five factor model of personality), and reward dependence and cooperativeness (in Cloninger's seven factor model of temperament and character) have relevance in accounting for how positive and negative events affect behavior.

(7) The authors grossly overinterpret their data and findings. Though the study certainly goes beyond the effect of work events on work engagement within a single day, the main analyses only provide some information for a two-week time period and not for a four month period.

Some housekeeping items--

(a) The numbering of hypotheses seems off. The first hypothesis provided on page 4 is given as "hypothesis 2" while a second "hypothesis 2" appears on page 6 along with hypothesis 3. The first hypothesis on page 4 should be stated as "hypothesis 1".

(b) The following sentence appears on page 6-- "In line with this perspective, [38] found that individuals reacted more sensitive to negative social events on a given day, when negative events had preceded the day before." The word "sensitive" should be "sensitively".

(c) At the top of page 8, the following sentence appears but is missing a period at the end. "Our analyses, therefore, also provide insights into the relative importance of positive 282 vs. negative events for work engagement in the long run"

(d) On page 9, the following sentence appears: "Given the focus and theoretical rationale of the present study, we confined analyses a set of eleven items, which were explicitly job-related." The word
"to" should appear between "analyses" and "a set".

Bottom line-- While the concerns surrounding research ethics are significant, the problems with data quality are so severe that confidence cannot be had in the results reported.  I recommend that this manuscript be rejected.