Clustering of Heart Failure Phenotypes in Johannesburg Using Unsupervised Machine Learning

Round 1

Reviewer 1 Report

This work is a good example of the numerous potentialities that exist when new data science methodologies (in this case machine learning) are combined with the need to better understand the physiological mechanisms underlying the changes that are expressed through biomarkers. In particular, the authors apply unsupervised clustering techniques with the aim of summarizing the demographic and clinical characteristics of heart failure in a group of hospitalized patients.

In general, the paper is well structured, written in the correct technical language, easy to read, and, therefore, structurally and formally complies with the writing rules that papers of this nature should have. Additionally, and as is mandatory, the protocol was approved by a hospital ethics committee.

In terms of methodological novelty, it cannot be said that it is an innovative methodology because the application of clusters for biomedical classification (whether of heart failure or other) has already been in practice, so there is no novelty. Even so, the possibility of obtaining new heart failure classification scores that do not require the use of echocardiography is of interest (particularly in places where this imaging test is not accessible).

More objectively, I think the following observations/questions could be analyzed by the authors and included in the text:

1. Given the importance of the applied methodology, I believe that more emphasis should be given to formalizing the clustering techniques that were presented in the Methods section;

2. Although 7 clustering techniques are mentioned, in the results section, it is not clear if they were all used or only one. I suggest the presence of at least one table with the possible results of the 7 techniques (or a justification for presenting only some of them);

3. Regarding the 3 clusters obtained, it would be important to understand the true importance of the LVEF variable, for example, doing a new analysis without this feature;

4. It would be important to make a correspondence between the three levels of LVEF and the three clusters obtained. For example, understand the distribution of the 91 mLVFE in the three clusters (how many of these 91 are in C1, C2, and C3). The same for the 409 rLVFE and 136 pLVFE;

5. In Table 1, and table 2 the units of the mean variable, are not correct. As it is, mean(SD) the unit of mean is SD! 

6. In tables 3 and 4, the values in columns 3, 4, and 5 are the p-values? 

 

As I mentioned before, the clarification of these aspects will improve the quality and impact of this work, and for this reason, I suggest their introduction in a new version of the text.

 

 

Author Response

1. Given the importance of the applied methodology, I believe that more emphasis should be given to formalizing the clustering techniques that were presented in the Methods section;

Response 1: Thank you for the suggestion. Please see page 2, line 20-25. We have elaborated on the clustering techniques in the introduction as well as in the methods section, page 5, line 19-32, page 6, line 2-27. The aim of the manuscript is to introduce unsupervised machine learning techniques to clinicians, and as such, we are targeting clinicians and have simplified the manuscript.

2. Although 7 clustering techniques are mentioned, in the results section, it is not clear if they were all used or only one. I suggest the presence of at least one table with the possible results of the 7 techniques (or a justification for presenting only some of them)

Response 2: Page 7. We have added Table 1 with the silhouette score of each cluster generated by each of the algorithms.  The algorithm with the highest silhouette score of 0.72 was chosen and used to cluster patients.

3. Regarding the 3 clusters obtained, it would be important to understand the true importance of the LVEF variable, for example, doing a new analysis without this feature;

Response 3: Page 16, line 12-22. Table 6, page 17. We have added a section on the results obtained using a dataset without the LVEF.

4. It would be important to make a correspondence between the three levels of LVEF and the three clusters obtained. For example, understand the distribution of the 91 mLVFE in the three clusters (how many of these 91 are in C1, C2, and C3). The same for the 409 rLVFE and 136 pLVFE;

Response 4: Page 9, line 12-13, line 21-22. We have compared three levels of the LVEF in various clusters.

5. In Table 1, and table 2 the units of the mean variable, are not correct. As it is, mean(SD) the unit of mean is SD! 

Response 5: we have removed SD in Table 2 and 3.. We previously included SD to specify that the value placed after ± is the standard deviation.

6. In tables 3 and 4, the values in columns 3, 4, and 5 are the p-values? 

Response 6: Page 16. Yes, these are p-values.  Since there are more than two categories of data compared, the post hoc analysis generated p-values to identify the category with values that are statistically significant.

As I mentioned before, the clarification of these aspects will improve the quality and impact of this work, and for this reason, I suggest their introduction in a new version of the text.

Thank you for the valuable comments.

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors of this manuscript use seven different unsupervised machine learning algorithms to phenotype heart failure patients into various clusters with multiple clinical parameters instead of LVEF solely. It appears that the section on Methods and Results is clear and reasonable. However, there are some major points that need to be resolved:

1. In this article, the primary focus is the use of seven relatively mature clustering methods for data analysis, which lacks novelty in both method implementation and result analysis. It is recommended that the authors propose more original solutions to this problem.

2. Although the entire article is relatively straightforward, we recommend adding more introduction and implementation details, especially at the beginning part.

 

3. To improve the credibility and usefulness of an article, we suggest authors disclose the code they used, for example, Jupyter notebook.

Comments for author File: Comments.pdf

Author Response

The authors of this manuscript use seven different unsupervised machine learning algorithms to phenotype heart failure patients into various clusters with multiple clinical parameters instead of LVEF solely. It appears that the section on Methods and Results is clear and reasonable. However, there are some major points that need to be resolved:

Point 1: In this article, the primary focus is the use of seven relatively mature clustering methods for data analysis, which lacks novelty in both method implementation and result analysis. It is recommended that the authors propose more original solutions to this problem.

Response 1: Page 20, line 73-78. We have added a section on recommendations for future studies. “Newer unsupervised machine learning techniques should take data analysis to the next level by creating a code that allows algorithms to interrogate the data and predict the risk of complications. Future studies should use larger, prospective cohorts of patients and, subsequently, train and test the performance of clustering algorithms using new data that was not used to create original clusters of patients.  Most importantly, the prognostic value of the unsupervised machine learning algorithms should be verified by measuring clinical outcomes and comparing them with outcomes obtained from patients classified solely with the LVEF.”

2. Although the entire article is relatively straightforward, we recommend adding more introduction and implementation details, especially at the beginning part.

Response 2: Page 2, line 20-25. The introduction and methods section (page 5, line 9-32, Page 6, line 2-27) has been amended to introduce implementation details. Since we are targeting predominantly clinicians, and the primary author is also a clinician, we have decided to omit the mathematical calculations and derivation of algorithms.

3. To improve the credibility and usefulness of an article, we suggest authors disclose the code they used, for example, Jupyter notebook.

Response 3:Page 5, line 16. The script from the Jupyter notebook has been attached as a supplementary file.

Author Response File: Author Response.pdf

Reviewer 3 Report

Mpanya et al. presents an unsupervised machine learnig to learn about heart failure in their particular database. The study is interesting and of merit, however, it cannot help much to learn since the database is local and their result can not be extrapolated to other populations (this was acknowledge in the manuscript, indeed), nevertheless, for their particular case, this kind of studies will help them to improve the treatment results but the evaluation presented in this paper is very superficial (no much discussion in this line).

The following comments support my decision:

• Regarding data processing: Authors did test their algorithm without the usage of the "filled" data? Why authors did not use the median value to fill the data as it is know to be a robust estimator where possible outlayers could affect the mean? Authors are 100% sure that the discharge state is not an informative variable? perhaps some algorithms could say a little bit about the risk of death in some certain HF conditions.

• Authors state that they evaluate different unsupervised algorithms for this task, however, no information could be found by this reviewer (in general, not only to justify the choosen algorithm). Authors may report their results with the rest of algorithms as well as a study justifying the reason of their choosen (at least in a supplement). Learning from this algorithms is a good thing, however, since the authors did not report the results from the other algorithms, this reviewer tend to think that they choose the clustering with more relevant clinical interpretation. This is fine when all the information is reported so the reader can figure out what happened using other algorithims. Moreover, a discussion on this will be much appreciated.

• Regarding the agglomerative cluster algorithm used, Authors should consider a figure here to illustrate the concept behind the silhouette score so the reader (that could be not very used to this kind of abstract concepts) better understand the authors rationale.

• Figure 2: Please add labels to the axis or indicate something in the caption, also, try to improve the quality of the image.

• Pages 12 and 13 have some duplications.

• Why the authors did not perform a survival analysis of the resulting clusters? (not only a mean hospitalization value)

Author Response

Mpanya et al. presents an unsupervised machine learnig to learn about heart failure in their particular database. The study is interesting and of merit, however, it cannot help much to learn since the database is local and their result can not be extrapolated to other populations (this was acknowledge in the manuscript, indeed), nevertheless, for their particular case, this kind of studies will help them to improve the treatment results but the evaluation presented in this paper is very superficial (no much discussion in this line).

The following comments support my decision:

Point 1:Regarding data processing: Authors did test their algorithm without the usage of the "filled" data? Why authors did not use the median value to fill the data as it is know to be a robust estimator where possible outlayers could affect the mean? Authors are 100% sure that the discharge state is not an informative variable? perhaps some algorithms could say a little bit about the risk of death in some certain HF conditions.

Response 1: Page 5, line 13-14. We used both mean and median, depending on the Gaussian distribution of the data. The variable is informative and we have reported the death rate in each cluster. However, we wanted to characterize patients without considering the effect of death.

• Point 2:Authors state that they evaluate different unsupervised algorithms for this task, however, no information could be found by this reviewer (in general, not only to justify the choosen algorithm). Authors may report their results with the rest of algorithms as well as a study justifying the reason of their choosen (at least in a supplement). Learning from this algorithms is a good thing, however, since the authors did not report the results from the other algorithms, this reviewer tend to think that they choose the clustering with more relevant clinical interpretation. This is fine when all the information is reported so the reader can figure out what happened using other algorithims. Moreover, a discussion on this will be much appreciated.

Response 2: Page 7. We have added Table 1 that justifies why the agglomerative clustering algorithm with 3 clusters was chosen.

 

• Point 3: Regarding the agglomerative cluster algorithm used, Authors should consider a figure here to illustrate the concept behind the silhouette score so the reader (that could be not very used to this kind of abstract concepts) better understand the authors rationale.

Response 3: Page 7. We have added table 1 that justifies why the agglomerative clustering algorithm with 3 clusters was chosen.

 

• Point 4: Figure 2. Please add labels to the axis or indicate something in the caption, also, try to improve the quality of the image.

Response 4: Page 11. Caption amended below the figure to describe labels on the axis.

 

• Point 5: Pages 12 and 13 have some duplications.

Response 5: Page 9 and 10. The two sections describe clinical characteristics and outcomes such as mortality and rehospitalisation in the various clusters

• Point 6: Why the authors did not perform a survival analysis of the resulting clusters? (not only a mean hospitalization value)

Response 6: Page 15. Survival analysis added for the respective clusters as figure 4

 

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The author has answered our questions. We think the manuscript can be accepted as its current form.