The Effects of Assumed AI vs. Human Authorship on the Perception of a GPT-Generated Text

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The abstract is clear and provides a concise overview of the study, its methodology, key findings, and implications. The introduction is well-structured, presenting a clear background and rationale for the study. It effectively highlights the relevance of the research question and situates it within existing literature. The literature review is thorough and cites relevant studies. It outlines the gap in the research regarding AI authorship on complex topics. The review could be enhanced by integrating a broader range of sources and more critically evaluating the cited studies. The methodology section is detailed and transparent, explaining the experimental design, sample, measures, and procedures clearly. The use of equivalence testing is a strong point, ensuring rigorous statistical analysis. The discussion effectively interprets the findings, linking them back to the research questions and hypotheses. It acknowledges limitations and suggests areas for future research. Including a more critical analysis of the implications for AI authorship in broader contexts would be beneficial. The conclusion summarizes the study well but could be more impactful by emphasizing the practical implications of the findings and suggesting specific future research directions.

Author Response

Dear Editor, dear Reviewers,

Thank you very much for allowing us to revise and resubmit our manuscript. We are convinced your helpful feedback improved the manuscript’s quality and now meets the publication standards. We carefully processed the criticized paragraphs and used track changes within the manuscript. In the following, we replied to each comment.

 

Comment 1: The abstract is clear and provides a concise overview of the study, its methodology, key findings, and implications. The introduction is well-structured, presenting a clear background and rationale for the study. It effectively highlights the relevance of the research question and situates it within existing literature. The literature review is thorough and cites relevant studies. It outlines the gap in the research regarding AI authorship on complex topics.

Response 1: Thank you very much for your supportive feedback.

 

Comment 2: The review could be enhanced by integrating a broader range of sources and more critically evaluating the cited studies.

Response 2: Thank you for your comment. In the introduction, we have further expanded and explained the sources cited.

 

Comment 3: The methodology section is detailed and transparent, explaining the experimental design, sample, measures, and procedures clearly. The use of equivalence testing is a strong point, ensuring rigorous statistical analysis. The discussion effectively interprets the findings, linking them back to the research questions and hypotheses. It acknowledges limitations and suggests areas for future research. Including a more critical analysis of the implications for AI authorship in broader contexts would be beneficial. The conclusion summarizes the study well but could be more impactful by emphasizing the practical implications of the findings and suggesting specific future research directions.

Response 3: Thank you again. We revised the discussion and conclusion sections to address this valuable recommendation and specifically addressed potential future research directions.

Reviewer 2 Report

Comments and Suggestions for Authors

 

The topic is extremely interesting and relevant for the future of journalism, congratulations on the topic choice. In order to maximise the readers’ understanding of the topic’s importance, I believe it’s crucial to remind them of what previous research has said as from the abstract. Thus I suggest to reference these previous investigations in lines 5-6: “Prior studies suggest no differences between human and AI authors regarding message credibility.” The same apply to similar assertions, such as “previous findings suggest significant differences in authorship perceptions” (lines 75-6): it would enhance the authors’ point to openly state which previous research contributed to the academic debates with such findings.

The method is explained in sufficient detail and the resulting findings are clearly linked to the hypotheses and research questions proposed. As a possible addition, I think the article would benefit from further explanations on how the respondents were selected (nationality, form of contact, preferences to represent certain segments -age, sex, etc.-). If, as I suppose, snowball/chain-referral sampling was used, this may be listed in the suggestions for further research (including stratified samples that can better represent the populations studied).

As a minor area with room for improvement, I noted that several epigraphs were numbered as “1” (see “Methodology” and “Discussion”).  

Finally, I believe that the authors should be more daring in the conclusion sections. More than a summary of what we have already read in the Discussion section, they should reflect on the implications (for journalism, readers, and ultimately society) of the increasing doubt on content authorship, as well as on mechanisms they believe should be considered to prevent undesired side effects such as undermined trust in the content industry or others.

 

Author Response

Dear Editor, dear Reviewers,

Thank you very much for allowing us to revise and resubmit our manuscript. We are convinced that your helpful feedback improved the manuscript’s quality and now meets the publication standards. We carefully processed the criticized paragraphs and used track changes within the manuscript. In the following, we replied to each comment.

 

Comment 1: The topic is extremely interesting and relevant for the future of journalism, congratulations on the topic choice. In order to maximise the readers’ understanding of the topic’s importance, I believe it’s crucial to remind them of what previous research has said as from the abstract. Thus I suggest to reference these previous investigations in lines 5-6: “Prior studies suggest no differences between human and AI authors regarding message credibility.” The same apply to similar assertions, such as “previous findings suggest significant differences in authorship perceptions” (lines 75-6): it would enhance the authors’ point to openly state which previous research contributed to the academic debates with such findings.

Response 1: Thank you very much for your recognition of the topic’s relevance. To address your suggestion, we have expanded on the points raised and precisely described the two most important studies in this field.

 

Comment 2: The method is explained in sufficient detail and the resulting findings are clearly linked to the hypotheses and research questions proposed. As a possible addition, I think the article would benefit from further explanations on how the respondents were selected (nationality, form of contact, preferences to represent certain segments -age, sex, etc.-). If, as I suppose, snowball/chain-referral sampling was used, this may be listed in the suggestions for further research (including stratified samples that can better represent the populations studied).

Response 2: Thank you for this comment. We added details on the sample recruitment in section “2.1. Sample”.

 

Comment 3: As a minor area with room for improvement, I noted that several epigraphs were numbered as “1” (see “Methodology” and “Discussion”).  

Response 3: Thank you for bringing this to our attention. We have corrected this.

 

Comment 4: Finally, I believe that the authors should be more daring in the conclusion sections. More than a summary of what we have already read in the Discussion section, they should reflect on the implications (for journalism, readers, and ultimately society) of the increasing doubt on content authorship, as well as on mechanisms they believe should be considered to prevent undesired side effects such as undermined trust in the content industry or others.

Response 4: We have comprehensively revised the discussion and conclusion sections and expanded them in line with the reviewer’s recommendations.

 

Thank you again very much for your overall positive feedback. We are convinced that your comments significantly improved our manuscript.