Overview of Federated Facility to Harmonize, Analyze and Management of Missing Data in Cohorts
Round 1
Reviewer 1 Report
Thank for your inviting me to review the paper on “Overview of Federated Facility to Harmonize, Analyze and Missing Data in Cohorts”. Handling miss data is an important topic for researchers. This paper provides an excellent summary of historical studies. I recommend publication in the Applied Science. I have several recommendations.
1. Line 63, the authors stated that “Meta-analysis is a common approach to pool results from multiple cohorts and is consistently used, for example, in GWAS [9]”. It is important to highlight other types of study. Please amend the statement as follows:
Meta-analysis is a common approach to pool results from multiple cohorts and is consistently used, for example, in GWAS [9], epidemiological studies [10], case-control studies [11] and randomized controlled trials [12].
New references
10. Lim GY, et al. Prevalence of Depression in the Community from 30 Countries between 1994 and 2014. Sci Rep. 2018 Feb 12;8(1):2861. PMID:29434331
11.Ng A et al. IL-1β, IL-6, TNF- α and CRP in Elderly Patients with Depression or Alzheimer's disease: Systematic Review and Meta-Analysis. Sci Rep. 2018 Aug 13;8(1):12050. PMID:30104698
12. Ng JH et al. (2014) Intratympanic steroids as a salvage treatment for sudden sensorineural hearing loss? A meta-analysis. Eur Arch Otorhinolaryngol. 272(10):2777-82. PMID:25217083
2. Line 65, the authors stated that “Frequently, due to the restrictions of participants confidentiality”, this may not be the main reason. From my experience, the main reason is refusal by authors to provide anonymized dataset. Please amend the statement as follows:
Frequently, due to the restrictions of participants confidentiality and refusal by authors to provide anonymized dataset, ….
3. Line 259, the authors stated “Since the focus of a meta-analysis is on the creation of summary statistics obtained from several studies, this method is most efficient when the original individual records used in prior analyses is not accessible or no longer occurs [29].” I recommend the authors to add a few disadvantages of meta-analysis to achieve a balanced view. Please add the following statements:
not accessible or no longer occurs [29]. Meta-analysis has several disadvantages including presence of high level of heterogeneity [30], unmeasured confounders [31], limitation by ecological fallacy [32] and most primary studies were conducted in developed or western countries [33]
References
30. Abraham N et al A Meta-Analysis of the Rates of Suicide Ideation, Attempts and Deaths in People with Epilepsy. Int J Environ Res Public Health. 2019 Apr 24;16(8). pii: E1451. doi: 10.3390/ijerph16081451. PMID:31022891
31. Low ZX et al Prevalence of Burnout in Medical and Surgical Residents: A Meta-Analysis. Int J Environ Res Public Health. 2019 Apr 26;16(9). pii: E1479. doi: 10.3390/ijerph16091479. Review. PMID:31027333
32. Foo SQ et al Prevalence of Depression among Migrants: A Systematic Review and Meta-Analysis. Int J Environ Res Public Health. 2018 Sep 12;15(9). pii: E1986. PMID:30213071
33. Ng TKS et al Decreased Serum Brain-Derived Neurotrophic Factor (BDNF) Levels in Patients with Alzheimer's Disease (AD): A Systematic Review and Meta-Analysis. Int J Mol Sci. 2019 Jan 10;20(2). pii: E257. doi: 10.3390/ijms20020257. Review. PMID:30634650
Author Response
Q1) Line 63, the authors stated that “Meta-analysis is a common approach to pool results from multiple cohorts and is consistently used, for example, in GWAS [9]”. It is important to highlight other types of study.
R1) We amended the statement as follows: Meta-analysis is a common approach to pool results from multiple cohorts and is consistently used, for example, in GWAS [9], epidemiological studies [10], case-control studies [11] and randomized controlled trials [12].
New references
Lim GY, et al. Prevalence of Depression in the Community from 30 Countries between 1994 and 2014. Sci Rep. 2018 Feb 12;8(1):2861. PMID:29434331
Ng A et al. IL-1β, IL-6, TNF- α and CRP in Elderly Patients with Depression or Alzheimer's disease: Systematic Review and Meta-Analysis. Sci Rep. 2018 Aug 13;8(1):12050. PMID:30104698
Ng JH et al. (2014) Intratympanic steroids as a salvage treatment for sudden sensorineural hearing loss? A meta-analysis. Eur Arch Otorhinolaryngol. 272(10):2777-82. PMID:25217083
Q2) Line 65, the authors stated that “Frequently, due to the restrictions of participants confidentiality”, this may not be the main reason. The main reason is refusal by authors to provide anonymized dataset.
R2) We amended the statement as follows: Frequently, due to the restrictions of participants confidentiality and refusal by authors to provide anonymized dataset.
Q3) Line 259, the authors stated “Since the focus of a meta-analysis is on the creation of summary statistics obtained from several studies, this method is most efficient when the original individual records used in prior analyses is not accessible or no longer occurs [29].”
R3) We added a few disadvantages of meta-analysis to achieve a balanced view. We included the following statements: “Meta-analysis has several disadvantages including presence of high level of heterogeneity [30], unmeasured confounders [31], limitation by ecological fallacy [32] and most primary studies were conducted in developed or western countries [33]. “
References
Abraham N et al A Meta-Analysis of the Rates of Suicide Ideation, Attempts and Deaths in People with Epilepsy. Int J Environ Res Public Health. 2019 Apr 24;16(8). pii: E1451. doi: 10.3390/ijerph16081451. PMID:31022891
Low ZX et al Prevalence of Burnout in Medical and Surgical Residents: A Meta-Analysis. Int J Environ Res Public Health. 2019 Apr 26;16(9). pii: E1479. doi: 10.3390/ijerph16091479. Review. PMID:31027333
Foo SQ et al Prevalence of Depression among Migrants: A Systematic Review and Meta-Analysis. Int J Environ Res Public Health. 2018 Sep 12;15(9). pii: E1986. PMID:30213071
Ng TKS et al Decreased Serum Brain-Derived Neurotrophic Factor (BDNF) Levels in Patients with Alzheimer's Disease (AD): A Systematic Review and Meta-Analysis. Int J Mol Sci. 2019 Jan 10;20(2). pii: E257. doi: 10.3390/ijms20020257. Review. PMID:30634650
Reviewer 2 Report
Congratulation! This is a great topic and the results of hard working. But I do suggest some changes to make it more clear and usable for the research community. In general, as a scientific manuscript, you need to come with more examples from your references. In several places, you just named a method or study and refer to references without explaining that. In this condition, your reader has to have several trips back and forth to the references to understand them and follow your discussion. Please consider my suggestions as follow:
The direct reference for the content of line 49 is missing. The ref/link that you provided is just for the website of CDC; same issued for line 47 Please reorganized the example in the introduction about genetic and other data. There is a jump from GWAS to Lifestyle and back again to genomics, etc. Line 37, 41, 49. Please make it organized to make it easier for the reader to follow your examples. Line 57 and following that; you have to explain and name the database/dataset. You cannot just use these and that and refer to the reference number of the study. Since this is a review paper, please explain and name the examples to make it more meaningful for your future reader to follow and understand. I suggest rewriting line 53 to 57 Add reference for line 72 Is there any reason that you put precision in the () in line 77? The example of cross-study by NIH in line 133 does not have any references In line 140-147 you described 3 studies for cohort selection, etc. But, you did not mention any federated system in these studies. If that was the case you need to explain what is the weakness of these studies that they have not used a federated system. In line 161 give an example for the Tools In general when we write a review paper and discuss the Tools/software/methodology, etc; we need to discuss the advantages and disadvantaged. For example line 161 to 172. Please consider that. Line 186: a few example will make it meaningful Line 193; I think this is not limited to a relational database. In like 267; please explain some of the limitations of meta-analysis more than just referring to reference #29; you explained a little bit later in the manuscript in the in a directed way. But, I think you need to explain directly in this paragraph as well. In section 7 you need to add examples for 3 steps. Use the real example from reference 35 and 43 and add to the manuscript. In line 308 and 309 add example. In line 347 add example. It is not clear from the text that how do you think we can decrease the missing value with increasing the quality of study design. This is generally speaking without detail or an example.17. The same issue as above when you say a new organization of database in line 375 to 379. We cannot say new, better quality, bad, better, etc. in the scientific literature. We need to be more precise.
Author Response
Q1) The direct reference for the content of line 49 is missing. The ref/link that you provided is just for the website of CDC; same issued for line 47 Please reorganized the example in the introduction about genetic and other data. There is a jump from GWAS to Lifestyle and back again to genomics, etc. Line 37, 41, 49. Please make it organized to make it easier for the reader to follow your examples. Line 57 and following that; you have to explain and name the database/dataset. You cannot just use these and that and refer to the reference number of the study. Since this is a review paper, please explain and name the examples to make it more meaningful for your future reader to follow and understand. I suggest rewriting line 53 to 57 Add reference for line 72 Is there any reason that you put precision in the () in line 77? The example of cross-study by NIH in line 133 does not have any references In line 140-147 you described 3 studies for cohort selection, etc. But, you did not mention any federated system in these studies. If that was the case you need to explain what is the weakness of these studies that they have not used a federated system. In line 161 give an example for the Tools In general when we write a review paper and discuss the Tools/software/methodology, etc; we need to discuss the advantages and disadvantaged. For example line 161 to 172. Please consider that. Line 186: a few example will make it meaningful Line 193; I think this is not limited to a relational database. In like 267; please explain some of the limitations of meta-analysis more than just referring to reference #29; you explained a little bit later in the manuscript in the in a directed way. But, I think you need to explain directly in this paragraph as well. In section 7 you need to add examples for 3 steps. Use the real example from reference 35 and 43 and add to the manuscript. In line 308 and 309 add example. In line 347 add example. It is not clear from the text that how do you think we can decrease the missing value with increasing the quality of study design. This is generally speaking without detail or an example. The same issue as above when you say a new organization of database in line 375 to 379. We cannot say new, better quality, bad, better, etc. in the scientific literature. We need to be more precise.
R1) Following the reviewer’s indication we stated more examples with explanation and references. Consequently, the manuscript has been extensively revised and streamlined. We considered all the points of the reviewer and revised the manuscript accordingly.
