COVIDTAS COVID-19 Tracing App Scale—An Evaluation Framework

Round 1

Reviewer 1 Report

Overall a very nice and welcome piece of research which expands the survey of contact tracing apps from Europe/US to the rest of the world, highlighting strengths and weaknesses in the process. I look forward to seeing this in print.

Author Response

Dear Reviewer,

We sincerely appreciate your valuable time for review. 

Reviewer 2 Report

In this article, the authored reported their recent comparative investigations on COVID contact tracing apps, and gave some evaluations on them. The study's results may be valuable to app developers and government departments to provide better services to people, though its contribution in academic research is not so significant. Nevertheless, considering its merit in the current situation, I suggest the acceptance of the paper with the following revisions:

• A conclusion must be explicitly given. In the current version, I can see some summary, but it is not so clear. The paper, especially its major results, must be summarized in a concise yet strong way.
• Tables are given to show the concrete results, but they are not very visualized for a good readability. For readers to understand the work easily, please consider using some graphs/figures to illustrate the main results.
• Australia and New Zealand belong to the continent "Oceania". Please make the corresponding corrections.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

Thank you for the opportunity to review this manuscript. It deals with a very current and relevant topic. I have some suggestions for improvement the performed work:

- The introduction section is relatively weak. The motivation is well addressed but it is not fully explored how the authors address this theme in their manuscript. The adopted approach is not properly presented in the introduction section.

- The last paragraph of the introduction section should present the structured of the manuscript and not focus only on the next section.

- In Portugal I have been involved in the development of a similar application in this area. I can report my experience that is not fully covered in this study. We have had a very high number of app downloads. However, the number of users is much lower and, above all, the number of codes generated by doctors to enter the app is very low. In summary, we have had difficulties in involving clinicians in this process, since many of them have difficulties adopting the technology (e.g., technological literacy) and are physically and mentally exhausted in the face of strong pressure on health services. It would be important to address this issue because it may be a phenomenon in other countries.

- It is not totally clear how the apps were chosen to be included in this study. The authors state “Stage I involved screening and selection of countries that had high incidence of COVID”. What is a high incidence rate?

-Looking to the adherence to GDPR requirements is an interesting approach. However, this study involves different non-European countries. How was this situation assessed in non-European countries?

- It is not clear what is a partially voluntary adherence to the application.

- Line 193 has an extra space. Please correct it.

- There are minor scientific deficiencies in the adopted approach. Authors state that included both qualitative and quantitative parameters. It is a very interesting approach and I agree in its entirety. However, this approach implies the combination of this information which is complementary using a mixed-methods approach. This methodological process is not properly described. There are many alternatives to combine data that are not properly addressed.

- The description of the methodological between lines 232 and 277 is complicated to read and decreases the flow of manuscript. It is not a good idea to present the various elements separated by bullets in a paper. Instead, use a figure where you describe the workflow or a table. It would better fit your needs.

- The ethical issued raised by these kind of apps should be also better discussed in the discussion section.

- Strangely there isn’t a dedicated section for conclusions. I recommend its inclusion.

- The authors do not explicitly address the practical and theoretical implications of this study.

- Some future research directions should also be given.

- Several emerging references in this area were not included in this study. It is a very challenging field as new studies are emerging in the last weeks and months. Some recommendations include:

https://www.tandfonline.com/doi/full/10.1080/0960085X.2020.1803155

https://www.jmir.org/2020/12/e21572

https://www.tandfonline.com/doi/full/10.1080/25741292.2020.1850404

https://journals.sagepub.com/doi/full/10.1177/1747016120943622

https://academic.oup.com/jamia/article/28/1/193/5862635

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7510337/

https://www.nature.com/articles/s41562-020-01044-x

https://www.thelancet.com/journals/landig/article/PIIS2589-7500(20)30184-9/fulltext

These references can be very useful to improve the discussion section.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 3 Report

I appreciate the authors' answers and clarifications. In the technical and scientific dimension, the study is relevant and the improvements made have significantly improved the final quality of the manuscript. However, some formatting and writing problems have arisen that the authors should correct. I point out the main flaws: - Line 203 -> indentation issue - Line 212 -> idem - Line 257 -> it is not a good idea to present the header of a table in a different page - Line 370 -> extra blank line - Page 17 (line 29) -> indentation issue - Page 17 (line 46) -> idem - Page 17 and 18 -> the space between lines is not consistent - The format of the new introduced references is not consistent. Use always the same references format.

Author Response

Dear Reviewer,

You are a very meticulous person with sharp eyes. Kindly find attached the updated manuscript addressing all your suggested changes.

Sincerely appreciate your commitment to scientific rigor and precious time.

Regards

Authors