Hybrid-Recursive Feature Elimination for Efficient Feature Selection

Round 1

Reviewer 1 Report

The paper suggested the use of hybrid-recursive feature elimination method for feature selections. Results are tested against 4 classifiers.

I think this paper needs some major revision becuase of it's experimental designs and originality

1) Combining multiple methods for feature selection is not a new idea, what makes your work stands out from others? if you are benchmarking your work against other methods, why don't you benchmark against other hybrid methods?

2) The literature review is lacking, please take a look at this (http://featureselection.asu.edu/algorithms.php) and update your review list. This is something I quickly search, there are probably more you can talk about

3) How come you don't talk about feature correlation and feature dependence? 

4) Line 200: If you decide to compare the GDS results, why did you choose the other datasets at the beginning? 

5) I don't think Figure 3 is necessary

Unless these questions can be clearly justified, I would not consider this manuscript is suitable for publication. 

 

Author Response

1) Combining multiple methods for feature selection is not a new idea, what makes your work stands out from others? if you are benchmarking your work against other methods, why don't you benchmark against other hybrid methods?

Response: There exist various kinds of feature selection approaches. The Proposed paper focused to improve ‘feature importance-based recursive feature elimination (RFE)’ because it shows good performance for high dimensional data. Especially SVM-RFE is widely used for ‘gene selection’ in the bioinformatics area. And, we have not found the research about other hybrid feature importance-based RFE. Therefore, we were benchmarking the proposed method against other feature importance-based RFE.

 

2) The literature review is lacking, please take a look at this (http://featureselection.asu.edu/ algorithms.php) and update your review list. This is something I quickly search, there are probably more you can talk about

Response: Feature selection is a wide research area. It is difficult to mention all of them. Instead, we added helpful references (review papers) in the introduction section.

 

3) How come you don't talk about feature correlation and feature dependence?

Response: We agree that we should consider feature correlation and feature dependency for feature selection task. In this study, we have not found how to measure those factors and combine them with our feature importance. We mentioned them as a further research topic.

 

4) Line 200: If you decide to compare the GDS results, why did you choose the other datasets at the beginning?

Response: We choose 4 datasets from UCI repository and 4 datasets from NCBI. UCI datasets have relatively small number of features and large number of instances than NCBI datasets. We want to show proposed method is effective for various kinds of datasets.

 

5) I don't think Figure 3 is necessary

Response: To increase understanding, we deleted Figure 3 and added Table 4 instead

Reviewer 2 Report

This article proposes a new method to perform feature selection for training machine learning models. The method is based on the integration of three standard methods, whose results are averaged (with or without accuracy weighting). The method is validated on 8 publicly available databases and appears to consistently outperform the standard approaches.

Overall, the article is clear, novel, timely, well-written and interesting.

I recommend publication.

Author Response

There is no comment for revision.

Reviewer 3 Report

The present paper discusses a very important issue in the ambit of machine learning and proposes an interesting solution to the the growing of datasets.

It is clearly structured and it is formally fine.

I think that it worthy to published to stimulate further discussion in the field of machine learning

Author Response

There is no comment for revision.

Round 2

Reviewer 1 Report

1) Combining multiple methods for feature selection is not a new idea, what makes your work stands out from others? if you are benchmarking your work against other methods, why don't you benchmark against other hybrid methods?

Response: There exist various kinds of feature selection approaches. The Proposed paper focused to improve ‘feature importance-based recursive feature elimination (RFE)’ because it shows good performance for high dimensional data. Especially SVM-RFE is widely used for ‘gene selection’ in the bioinformatics area. And, we have not found the research about other hybrid feature importance-based RFE. Therefore, we were benchmarking the proposed method against other feature importance-based RFE.

[second comment]: Thanks

2) The literature review is lacking, please take a look at this (http://featureselection.asu.edu/ algorithms.php) and update your review list. This is something I quickly search, there are probably more you can talk about

Response: Feature selection is a wide research area. It is difficult to mention all of them. Instead, we added helpful references (review papers) in the introduction section.

[second comment]:Yes, I agree, but you can categorize them and mention a few that are not rfe-based. I do appreciate you put some review papers in the revision. Thanks.

3) How come you don't talk about feature correlation and feature dependence?

Response: We agree that we should consider feature correlation and feature dependency for feature selection task. In this study, we have not found how to measure those factors and combine them with our feature importance. We mentioned them as a further research topic.

[second comment]: as simple as constructing a correlation matrix you can quickly check feature which two features are highly correlated. I am not familiar with your datasets but you should check if they had dealt with this issue. In an extreme case, imagine you have all your features perfectly correlated, how would you benefit from feature selection? 

4) Line 200: If you decide to compare the GDS results, why did you choose the other datasets at the beginning?

Response: We choose 4 datasets from UCI repository and 4 datasets from NCBI. UCI datasets have relatively small number of features and large number of instances than NCBI datasets. We want to show proposed method is effective for various kinds of datasets.

[second comment]: Thanks

5) I don't think Figure 3 is necessary

Response: To increase understanding, we deleted Figure 3 and added Table 4 instead

[second comment]: Thanks