Optimized Stacking Ensemble Learning Model for Breast Cancer Detection and Classification Using Machine Learning

Round 1

Reviewer 1 Report

The article is well written and easy to understand. However, some concerns need to be addressed before acceptance. The issues are as follows: 

1. The introduction needs to be concise. The authors should highlight the problem statement. 

2. All the abbreviations should be abbreviated when used first time. 

3. Please provide a table for simulation parameters and explain the experimental setup.

4. Please add some latest work in the related works. 

5. The conclusion lacks a summary of their contributions. Please revise the conclusion.

6. How is your proposed solution better than existing methods like AdaBoostM1, Gradient Boosting, Stochastic Gradient Boosting, CatBoost, and XGBoost?

7. Please improve the quality of figure number 11.

8. Please define each KPI mentioned in this paper. Please improve the discussion of the result section. 

 

 

 

Author Response

Reviewer 1#

The article is well written and easy to understand. However, some concerns need to be addressed before acceptance. The issues are as follows: 

1. The introduction needs to be concise. The authors should highlight the problem statement. 

Ans: The necessary changes have been incorporated in the revised manuscript.

2. All the abbreviations should be abbreviated when used first time. 

 Ans: Thanks for pointing out this mistake and according to that changes have been done in the revised manuscript.

3. Please provide a table for simulation parameters and explain the experimental setup.

Ans: Authors are thankful for the reviewer for pointing out this. The detail regarding the tool, performance metrics and classification algorithms mentioned in the material and method section.

4. Please add some latest work in the related works. 

Ans: The necessary changes have been incorporated in the revised manuscript.

5. The conclusion lacks a summary of their contributions. Please revise the conclusion.

Ans: According to the reviewer suggestion conclusion section has been updated in the revised manuscript.

6. How is your proposed solution better than existing methods like AdaBoostM1, Gradient Boosting, Stochastic Gradient Boosting, CatBoost, and XGBoost?

Ans: Authors are thankful for the reviewer for pointing out this. Our proposed solution uses heterogenous approach as comparison to other existing methods used homogenous approach. All the above listed classifiers be in the categories of homogeneous ensemble methods and in table-3 the detail regarding the same is given.

7. Please improve the quality of figure number 11.

Ans: The necessary changes have been incorporated in the revised manuscript.

8. Please define each KPI mentioned in this paper. Please improve the discussion of the result section. 

Ans: The necessary changes have been incorporated in the revised manuscript.

 

Reviewer 2 Report

Authors present the optimized stacking ensemble learning model for breast cancer detection and classification using machine learning. A new approach for breast cancer detection and classification is proposed which is the novelty of this research work. Results are analysed with the existing ensemble learning model. Apart from this I have some concerns which need to be rectified before acceptance of the research paper in this journal: 

1. The novelty of the item should be clearly stated in the abstract section.

2. Add some empirical results in the abstract section and compare them with the existing ones.

3. I suggest adding the conclusion part in the abstract section.

4. Please write the motivation of your study in the introduction before your contributions.

(a short paragraph)

5. Please summarize some relevant and latest survey articles and compare your contributions with the existing surveys.

6. Describe clearly the simulation part.

7. To maintain the reader’s attention please highlight the main contributions with a diagram/chart.

8. Please highlight your contributions in the conclusion section.

9. The Limitations of the proposed study need to be discussed in the conclusion.

10. In figure 2, 3, the captions used for boxes can be improved to make it more understandable.

11. Authors are advised to explain the algorithm 1 so as the reader understand it easily.

12. The paper is well structured; however, the authors must check it for typographical errors and grammatical mistakes.

13. How the authors have done the pre-processing of the data used in the study.

14. The authors should clearly state the significance of the proposed methodology in the introduction section.

15. Authors should declare the acronyms first whenever they appear in the literature first.

Author Response

Authors present the optimized stacking ensemble learning model for breast cancer detection and classification using machine learning. A new approach for breast cancer detection and classification is proposed which is the novelty of this research work. Results are analysed with the existing ensemble learning model. Apart from this I have some concerns which need to be rectified before acceptance of the research paper in this journal: 

1. The novelty of the item should be clearly stated in the abstract section.

Ans: The necessary changes have been incorporated in the revised manuscript.

2. Add some empirical results in the abstract section and compare them with the existing ones.

Ans: Required changes have been incorporated in the revised manuscript.

3. I suggest adding the conclusion part in the abstract section.

Ans: Thanks for your valuable suggestion to improve the readability of the abstract, we do all the recommended changes in the revised manuscript.

4. Please write the motivation of your study in the introduction before your contributions.

Ans: Separate motivation section has been added in the revised manuscript.

5. Please summarize some relevant and latest survey articles and compare your contributions with the existing surveys.

Ans: Thanks for your valuable suggestion to improve the quality of this research paper, The necessary changes have been incorporated in the revised manuscript. Table 1 and Table 4 are related to literature and comparison part of this work.

6. Describe clearly the simulation part.

Ans: Thanks for this suggestion and we do the change in the manuscript in material and method section.

7. To maintain the reader’s attention please highlight the main contributions with a diagram/chart.

Ans: Authors are very thankful for your comments and diagram/chart are added to make reader understand the things in better way.

8. Please highlight your contributions in the conclusion section.

Ans: The necessary changes have been incorporated in the revised manuscript.

9. The Limitations of the proposed study need to be discussed in the conclusion.

Ans: The necessary changes have been incorporated in the revised manuscript.

10. In figure 2, 3, the captions used for boxes can be improved to make it more understandable.

Ans: All figures are revised in the manuscript.

11. Authors are advised to explain the algorithm 1 so as the reader understand it easily.

Ans: Required changes have been incorporated in the revised manuscript.

12. The paper is well structured; however, the authors must check it for typographical errors and grammatical mistakes.

Ans: The manuscript is checked thoroughly for typographical errors and grammatical mistakes.

13. How the authors have done the pre-processing of the data used in the study.

Ans: It is mentioned in the materials and methods section.

14. The authors should clearly state the significance of the proposed methodology in the introduction section.

Ans: The necessary changes have been incorporated in the revised manuscript.

15. Authors should declare the acronyms first whenever they appear in the literature first.

Ans: Required changes have been incorporated in the revised manuscript.

 

 

 

Reviewer 3 Report

Please see the attached PDF for comments.

Comments for author File: Comments.pdf

Author Response

The breast cancer mortality rates can be efficiently lowered due to the early diagnosis. However, the early detection of the disease is challenging since machine learning remains a difficult field which need the application of specialized knowledge and skills. In this study, OSEL model is developed to solve the classification problems in early breast cancer prediction by incorporating optimization models to determine the most effective combination of them. The experimental results demonstrate that the proposed method achieves the highest accuracy in the breast cancer prediction compared to other existing models.

The manuscript is well-organized. The objective is well-articulated and reached. The results and analysis presented in the manuscript are interesting for this field and Sustainability is the appropriate place to submit it. But the manuscript is not well-written. There are obvious mistakes or improper expression, including text, citations, abbreviations, equations, algorithms, definitions, tables, and figures. There are still some points that the authors should consider, as described in the following. Also, some suggestions are provided, in case the authors consider them interesting to carry out.

In 1. Introduction line 41, does 6,85,000 mean 685,000? Please use the correct way to write a number.

Ans: Thanks for the suggestion. It is correct in the revised manuscript.

In 1.1 Stacking Ensemble Learning Architecture line 113, is the base-level classifier the same as base-classifier? Please keep the term expression consistent, like base-level classifier/base-classifier/base classifier, meta classifier/meta classifier.

Ans: Thanks for pointing out this mistake and according to that changes have been done in the revised manuscript.

In 1.1 Stacking Ensemble Learning Architecture line 120-124, “Stacking is a well-known heterogeneous ensemble method .... It is analogous to the concept of boosting. Another method is called “stacking” and in this method, ....”. These sentences make people confused. The authors first mentioned “stacking is a well-known method”, but later also said “another method is called stacking”. In Figure 1, since meta-classifier is labelled in this figure, base-classifiers should be labelled, too.

Ans: Thanks for pointing out this mistake and according to that changes have been done in the revised manuscript. Figure 1 is also revised in the manuscript.

In line 158-159, “The Naive Bayes method performed the best, ... First and third place went to the RBF Network and J48 algorithms, respectively”. Since the Naive Bayes method is the best, how can the RBF Network get the first place?

Ans: The necessary changes have been incorporated in the revised manuscript.

In section 2 Related work, the authors’ names are mentioned in different ways. Please keep the name format consistent in the entire manuscript.

For example, Kharya, S. et al. (a comma after the last name)

Chaurasia V. et al. (no comma after the last name)

1. Sahu et al. (last name is placed after the first name’s initial)

Ans: Thanks for your valuable suggestion to improve the readability of the literature review, we do all the recommended changes into the revised manuscript.

The section 2 Related work is not well-written. There are some major problems in this section, including the verb tense, abbreviations, and citations. First, it is important to use the proper verb tenses (past, present, and future) in the scientific paper. On one hand, we usually use the past tense to report what you observed in your experiment, what you did in your study, what was published by someone else, etc. On the other hand, we use the present tense to describe the general truths or facts, e.g., the background information and the conclusions. The same verb tense should be kept whenever necessary within the same sentence or paragraph in most time. The authors should carefully examine this section or even the entire manuscript about the verb tenses and correct the mistakes. For example, in line 164-165, “but Ojha U. et al., emphasize the importance of parameter selection”. “emphasize” should be “emphasized” here.

Ans: Thanks for your valuable suggestion to improve the readability of the literature review, we do all the recommended changes into the revised manuscript

Second, there are too many abbreviations without the corresponding full expression. When using abbreviations in academic writing, the first time you mention a phrase that can be abbreviated, spell it out in full and provide the abbreviation in parentheses. Use only the abbreviation thereafter. The authors should check the entire manuscript and see if abbreviations are used properly.

Ans: All the abbreviations are abbreviated during its first use in the entire manuscript.

Third, the authors should think about how often to cite and where to cite in text. There are too many citation problems to list one by one. Some examples are shown here, and the authors need to carefully read the manuscript and learn how to use in-text citation. For one thing, it is recommended to cite a source the first time it is used in each paragraph. It is a good idea to name the source(s) in your sentence and immediately cite after the name(s) of author(s) (or the name of method/model/algorithm). If you do not explicitly name your source, a single citation can be added at the end of the sentence or clause it covers. For example, the weighted idea of NBC for the diagnosis of breast cancer was changed by Kharya, S. et al. [8] ...

1. Sahu et al. [13] used a neural network to classify breast cancer data ...

A Breast Classifier developed by Kumar V. et al [12] ...

The proposed JSDA [17] technique not only improves ...

For another, one citation at the end of a string of sentences or a paragraph cannot “cover” the entire section.

Ans: Thanks for your valuable suggestion to improve the readability of the literature review, we do all the recommended changes into the revised manuscript.

Every sentence thereafter in the paragraph that uses information from the same source must contain either a signal phrase or a citation clearly indicating where the information came from. If there are multiple sentences about the same source, it is very important to cite at the proper place(s) and use some signal phrases (e.g., according to XXX’s study, XXX proposed, XXX used, the same study indicated that) to connect these sentences.

Ans: The necessary changes have been incorporated in the revised manuscript.

The in-text citation placement is a big problem in this manuscript since it is very hard to track the source(s) of some sentences between two citations. For example, from line 173 to 177, it is unclear that these sentences point to reference [13] or [14]. From line 242 to 247, it is unclear that these sentences point to reference [23] or [24].

Ans: Required changes have been incorporated in the revised manuscript.

Moreover, the authors should try to avoid using “we”, “our” when introducing other people’s work, and avoid using “this study”, “this paper”, “the proposed method”, “the authors” without any citations next to them. For example, line 180-185, “The authors proposed ... It was also important for us to compare our model’s accuracy to ...”. Who are those authors? Does “us” mean the authors of this manuscript about OSEL approach? Does “our model” mean the OSEL model? The citation [15] appears after several sentences and the readers have no clue about “the authors”, “us” and “our model”.

Ans: The necessary changes have been incorporated in the revised manuscript.

In line 186, “A Modified Bat Technique (MBT) was proposed in this paper”. Which paper?

Ans: The necessary changes have been incorporated in the revised manuscript.

In line 187, “We rewrote the Bat algorithm ...”. Who rewrote the Bat algorithm?

Ans: Required changes have been incorporated in the revised manuscript.

In line 206, “Mammograms from the miniMIAS database are used in this study’s experiment.” Which study? There is no related citation next to it.

Ans: The necessary changes have been incorporated in the revised manuscript.

In line 234-235, “This study suggests a solution for dealing with categorization problems: the GONN strategy, which is described in the paper”. Which study? Which paper? Reference [23] should be cited in this sentence.

Ans: The necessary changes have been incorporated in the revised manuscript.

In line 237, “We propose new crossover and mutation operators ...”. Do “we” point to the author of this manuscript about OSEL approach?

Ans: Thanks for the suggestion. We have corrected it in the revised manuscript.

In Figure 2, the subplots located along the diagonal seem like line graphs, not scatter plots. Do these four subplots represent histograms? If yes, please clearly state it in the figure legend.

Ans: Yes, these are histograms and necessary change have been done in the revised figure.

In line 297, “heat map” should be “heatmap” here.

Ans: Correction has been done in the revised manuscript.

In line 299, “whereas values that are less than zero imply that there is no association at all”. However, in the figure legend of Figure 3, we can see that -1 is used for negative linear correlation between two features of the dataset. Which one is true?

Ans: This is true and -1 is used for negative linear correlation between two features of the dataset.

In line 313, the phrase “information importance” refers to a collection of methods for assessing the relative value of various input data in a predictive model. Does “information importance” mean “feature importance” because the subtitle of this section is “Feature Importance”?

Ans: Changes have been made in the revised manuscript.

In line 342, there are out of 569 instances for the selected dataset. Why not use the full dataset with 683 instances in total as stated in line 265?

Ans: Thanks for pointing out this question here, but we do it to decrease the class level difference. So, we have taken 569 instances out of total 683 instances.

In Figure 5, it would be a plus if the countplot (left) can use the corresponding colors (read and purple) to represent two classes as the pie chart (right) does.

Ans: Thanks for this suggestion but we need to take two color to represent it as two classes in the pie chart.

In Figure 6, please label the x-axis and y-axis. What do x-axis and y-axis represent, respectively?

Ans: Figure 6 is improvised in the revised manuscript.

In Equation 1, there are obvious mistakes in this equation. We know that p and q are used to represent two points. However, the left part of this equation uses ?(?, ?) = ?(?, ?), not including p. Moreover, in the middle of this equation, (?! − ?! ) " appears twice under the square root symbol, and there is an extra “)” after (?# − ?# ) " under the square root symbol.

Ans: The necessary changes have been incorporated in the revised manuscript.

In Equation 4, those parameters in this equation are not fully introduced. What are a and b, respectively? According to Equation 4, ?$% should be ?! !" in line 431, ?&$% should be ?& !" in line 432, ?!% should be ?! " and ?'% should be ?' " in line 433.

Ans: The necessary changes have been incorporated in the revised manuscript.

In line 398-408, we can see the mixture of Random Forest/random forest. Please keep the phrase (capitalized or not) consistent. The same as Support Vector Machine/support vector machine/SVM in line 446-458.

Ans: The necessary changes have been incorporated in the revised manuscript.

In Equation 5, the correct expression should be ?( = ∑ ?[?()! (?$ ) + ?( ℎ(?$ )]$ Equation 5 is written poorly. Please try to use the subscripts in the equation in a right way. Microsoft Word has its function to input equations. And the parameter t is not introduced here.

Ans: Thanks for the suggestion. The necessary changes have been incorporated in the revised manuscript.

In line 478, there an extra quotation mark (“) before Regressors.

Ans: Extra quotation mark has been removed in the revised manuscript.

In line 482, a citation is required next to “Friedman”.

Ans: Citation have been mentioned according to the reviewer suggestion.

In line 493, a citation is required next to “Yandex”.

Ans: Citation have been mentioned according to the reviewer suggestion.

In 3.4.1 Accuracy line 526-528, “An official definition is the number of true positives minus the number of false positives minus the number of false negatives divided by the total number of both true positives and false negatives.” This is not the definition of accuracy.

Ans: Thanks for pointing out this mistake and according to that changes have been done.

In 3.4.2 Precision line 536-537, “precession is defined as the ratio of correctly identified events to total events”. “precession” is a typo here. And this definition is not correct. Precision is defined as the ratio of correctly classified positive samples to a total number of classified positive samples (either correctly or incorrectly).

Ans: Thanks for this suggestion and necessary change have been made.

In 3.4.2 Recall line 541-542, “The recall is calculated by dividing the total number of positive samples by the number of positive samples that were correctly identified as positive.” The definition should be the number of positive samples that were correctly identified as positive divided by the total number of positive samples. “a ÷ b” is read “a divided by b”.

Ans: Authors are thankful for the reviewer for pointing out this mistake and according to that we have made necessary changes.

In line 577, “An architectural framework for stacking ensemble learning”. This is not a complete sentence.

Ans: Thanks for pointing out this mistake and according to that changes have been done.

In Algorithm for selecting Optimized Base Classifier for Stacking line 593-608, this section is not well-written. Several things need to be clarified.

What does k-train or m-train mean? Like the kth or mth train base classifiers?

Ans: Thanks for this suggestion and necessary change have been made. k-train means total numbers of base classifiers to be trained on the input features vector space. m-train means total number of base classifiers which are selected for the stacking purpose.

In Equation 10 line 598, it is not a good idea to use ?$ to express in the right part. In Equation 10, i represents the i th train base classifier. However, ?$ can be misunderstood with the ?! , ?" , ... , ?# in line 594 which indicate 1st ,2nd , ... n th feature in the input feature vector. The authors should find a better way to describe.

Ans: Thanks for this suggestion and necessary change have been made in the revised manuscript.

In line 599, why is b stated here again? It is already shown in line 595. Do the authors want to say the meta classifier ? * is chosen from b?

Ans: Thanks for your valuable comment. Yes, m_c is chosen from b.

In Equation 11 line 601, the left part is represented by z, but the final output is defined as p in line 600.

Ans: Thanks for pointing out this mistake and according to that changes have been done.

In line 602, “the actual value of the dataset sample level ? = [?! , ?" , ... , ?# ].” This is not a complete sentence.

Ans: Thanks for this suggestion and necessary change have been made in the revised manuscript.

In Equation 12, we know the predicted value ? = [?! , ?" , ... , ?# ] and the actual value ? = [?! , ?" , ... , ?# ]. Thus, the index i in Equation 12 should be 1 ≤ ? ≤ ?, but line 608 shows 1 ≤ ? ≤ ?. Moreover, the index is i at ?$ and ?$ , however, the index letter becomes n next to the summation sign Σ. In addition, ! ' should be ! # before the summation sign Σ if the authors agree that 1 ≤ ? ≤ ?.

Ans: Thanks for this suggestion and necessary change have been made in the revised manuscript.

In line 620-621, “Finding the harmonic mean of the accuracy score and the recall score is the first step in calculating the F1 score”. F1 score is calculated using precision and recall, not accuracy and recall.

Ans: Authors are thankful for the reviewer for pointing out this mistake and according to that we have made necessary change.

In line 626-628, the algorithms are listed, but not including XGBoost.

Ans: Authors are thankful for the reviewer for pointing out this mistake and according to that we have added XGBoost.

In Table 2, the numbers are very small in the columns of Precision (e.g., 0.97%), Recall (e.g., 0.91%) and F-Measure (e.g., 0.94%). For example, the k-NN precision is shown as 0.97%, which is equal to 0.0097. Are they correct? Do the authors want to say 97%? The percentage numbers shown in the column of Accuracy look reasonable. The same problem occurs in Table 3.

Ans: Thanks for this suggestion and necessary change have been made in the revised manuscript.

Figure 9 look redundant. It is recommended to plot 9(a) 9(b) 9(c) 9(d) in one graph. The values in the y-axis of 9(b) 9(c) 9(d) are still questionable since they are from Table 2 and they are too small (e.g., 0.97%=0.0097). The same problem occurs in Figure 10. Figure 9 and Figure 10 are not necessary because their information is clearly shown in Table 2 and Table 3, respectively.

Ans: Thanks for this suggestion and necessary change have been made in the revised manuscript. Figure 9 and Figure 10 is improvised in the revised manuscript.

In line 721-722, “The two models were put through the same set of tests.” What are “two models”?

Ans: Thanks for pointing out this mistake in the manuscript. By mistake we have added this line. As this line does not have any relevance in the manuscript, so I have removed that line.

In line 726-728, “in this investigation, we compare various classification models by using a wide range of classification algorithm metrics, including accuracy, recall, precision, and the F- measure.” This section indicates the comparison of OSEL model with existing models. However, we can only see accuracy in Table 4. Where are the results of recall, precision, and the F- measure?

Ans: Thanks for this wonderful suggestion and we will work on this in future. Table 4 includes the accuracy parameters because most of the existing state of art comparison includes only Accuracy parameter.

In Table 4, the accuracy of RS-SVM is much higher than 89.2% according to Chen’s paper [29].

The accuracy of WNN-GA is 98.61% as shown in Yi’s paper [32], not 97.38%. And the dataset (mammographic masses breast cancer dataset) used in Kaushik’s paper seems not the Wisconsin Breast Cancer dataset. The authors should double check these details.

Ans: Thanks for pointing out this mistake and according to that changes have been done.

In line 745, a citation is required next to RS-BPNN approach.

Ans: Citation is added in the revised manuscript.

In line 752-753, “RS SVM may be able to accurately categorize things up to 89.20% based on the results of the tests.” “RS SVM” should be “RS-SVM”. And the accuracy of RS-SVM is much higher than 89.2% according to Chen’s paper [29].

Ans: The necessary changes have been incorporated in the revised manuscript.

In Figure 11, the title above the figure is “COMPARISON OF THE PROPOSED OHML MODEL WITH EXISTING MODELS”. What is OHML MODEL?

Ans: Thanks for pointing out this mistake and according to that changes have been done. It is not OHML but it is OSEL.

The 7th bar is labelled as “Liu et al. [32]”, however, it is “Yi X. et al. [32]” in Table 4. Figure 11 is not necessary since all information is from Table 4.

Ans: Authors are thankful for the reviewer for pointing out this mistake and according to that we have made necessary changes.

The authors should provide the computational time comparison among different classifiers and among different models.

Ans: Thanks for your valuable suggestion and this point will be considered while working on the next phase of this research work. In this present work only accuracy, recall, precision, and F-measure are considered and for the comparison purpose accuracy of the model considered. As most of the existing research work considered accuracy as their parameter for model building.

The author should provide the results from more than one dataset to demonstrate the outperformance of the proposed OSEL model.

Ans: Authors are welcome for your kind suggestion and consider this suggestion in future scope of this manuscript. As authors already compared the proposed model with the existing research work on the same dataset, but surely your suggestion be kept in mind while working on the next phase of this research work.

What is the training-test partition? Like the ratio for training and testing dataset?

Ans: Thanks for your valuable comment. Its 70%-30%.

 

Round 2

Reviewer 1 Report

The authors have addressed my all comments. 

Author Response

The authors have addressed my all comments. 

Ans: Thanks for the acknowledgement.

Author Response File: Author Response.docx

Reviewer 2 Report

The revised version covered all the comments. The paper is acceptable for publication in its revised version. 

Author Response

The revised version covered all the comments. The paper is acceptable for publication in its revised version.

Ans: Thanks for the acknowledgement.

Author Response File: Author Response.docx

Reviewer 3 Report

Please see the attached file about the minor comments.

Comments for author File: Comments.pdf

Author Response

In the revised manuscript, the authors have resolved the problems and answered the questions I pointed out in my previous review. I don’t have further questions on this revised manuscript except some minor comments listed below.

In the revised manuscript line 254, the authors may place a period (.) after “strategy” to break the long sentence into two sentences.

Ans: Thanks for the suggestion. It is correct in the revised manuscript.

In the revised manuscript line 506, please keep the format of in-text citation consistent. The format of in-text citation on “Yandex” is different from others.

Ans: Thanks for pointing out this mistake and according to that changes have been done in the revised manuscript.

In the revised manuscript line 615, it may be proper to say the meta classifier m_c is chosen from the b vector.

Ans: Thanks for pointing out this mistake and according to that changes have been done in the revised manuscript.

In the revised manuscript line 623 (Equation 12), according to ? = [?1, ?2,…, ?n] and ? = [?1, ?2,…, ?n], should we define 1 ≤ ? ≤ ? and 1/n Σ(‖?i − ?i‖)² in Equation 12? If the authors want to define 1 ≤ ? ≤ ?, it should be 1/mΣ(‖?i − ?i‖)².

Ans: The necessary changes have been incorporated in the revised manuscript.

Author Response File: Author Response.docx