ASPDC: Accelerated SPDC Regularized Empirical Risk Minimization for Ill-Conditioned Problems in Large-Scale Machine Learning

Round 1

Reviewer 1 Report

The manuscript (electronics-1820175), ASPDC: Accelerated SPDC Regularized Empirical Risk Minimization for Ill-Conditioned Problem in Large-Scale Machine Learning, shows interesting results of the proposed methods for regularized empirical risk minimization problem. Authors presented quite comprehensive analysis in the manuscript. Some minor comments would like to provide here for authors' reference, shown as following - 

1. For all the Figure sets (Figure 1 to Figure 3), please provide the unit for X and Y to make sure this referee and potential readers can catch the results. 

2. Can authors provide a simple/summarized benchmark Table comparison between the current proposal works/methods and previously work/methods before conclusion? This would be quite helpful for this referee and potential readers to get further insight of this research work impact and contributions as compared to previous work. 

Due to the above comments, this referee would like to put the manuscript status as "Major Revision" in the current phase.

Author Response

1. For all the Figure sets (Figure 1 to Figure 3), please provide the unit for X and Y to make sure this referee and potential readers can catch the results.

Response：

Thanks for your helpful comments. We have added units to axis X and Y, they are “# of epochs” and “Error rate”, respectively.

2. Can authors provide a simple/summarized benchmark Table comparison between the current proposal works/methods and previously work/methods before conclusion? This would be quite helpful for this referee and potential readers to get further insight of this research work impact and contributions as compared to previous work.

Response：

Thanks. We have made the modification according to your suggestion, we compared ASPDC with the state-of-art dual methods: stochastic dual coordinate ascent method (SDCA) and stochastic primal-dual coordinate method (SPDC). The comparison results are shown in Table 2 to Table 4 and Figure 1 to Figure 2.

Reviewer 2 Report

1) In introduction section, the "paper organization" is missing. please add, the organization is as follows. In section 2 we had discussed xx, xx, xx.

2) The "Related work" section is missing in your paper. Therefore, please add this section along-with most recent studies.  

3) in future work section, please add future work of your study.

4) Please add a table and write complete abbreviations that you have used in this study, for example; ASPDC.

5) more explanation is required for "Algorithm1".

Author Response

1) In introduction section, the "paper organization" is missing. please add, the organization is as follows. In section 2 we had discussed xx, xx, xx.

Response：

Thanks for your advice. We have added the paragraph of paper organization in the introduction section. It is also listed as follows:

This paper is organized as follows. In section 2, we describe the related work. In section 3, we list the related assumptions and preliminary. In section 4, we discuss the accelerated stochastic primal-dual coordinate method. We present ASPDC in Algorithm 1 and its convergence analysis for the saddle point. In section 5 we present the extension of ASPDC to the ill-conditioned problem. The extension method is called ASPDC-$i$, where $i$ means "for ill-conditioned problems". In section 6 we had evaluated the performance of our ASPDC algorithms with several state-of-art algorithms for solving machine learning problems. We also discussed the experimental results in this section. In section 7, we conclude our work in this paper and discuss the future work.

2) The "Related work" section is missing in your paper. Therefore, please add this section along-with most recent studies.  

Response：

Thanks. We have added the section of related work as section 2.

3) In future work section, please add future work of your study.

Response：

Thanks. In section 7 we had concluded our work in this paper and discussed the future work. We believe that it is possible to extend the proof for more general regularized function and we leave it as a future work.

4) Please add a table and write complete abbreviations that you have used in this study, for example; ASPDC.

Response：

Thanks a lot. We have added a table and listed complete abbreviations in this study in the section of Assumptions and Preliminary.

5) more explanation is required for "Algorithm1".

Response：

Thanks. We have added relevant description in revised section 4 (Accelerated stochastic primal-dual coordinate method). In this section, we present ASPDC in Algorithm 1 and its convergence analysis for the saddle point problem in Eq.(3). Each iteration in ASPDC can be divided into two steps.