Review Reports - Toward an Ideal Particle Swarm Optimizer for Multidimensional Functions

Round 1

Reviewer 1 Report

The authors introduce three modifications to the method that aim to reduce the required number of function calls while maintaining the accuracy of the method in locating the global minimum. These modifications affect important components of the method, such as how fast the particles change or even how the method is terminated. The modifications are tested on a number of known universal optimization problems from the relevant literature and the results were compared with similar techniques.

The English level of the paper should be improved.

The deviations should be given with boxplots for the results.

Please include the following references:

BoussaïD, I., Lepagnot, J., & Siarry, P. (2013). A survey on optimization metaheuristics. Information sciences, 237, 82-117.

Dokeroglu, T., Sevinc, E., Kucukyilmaz, T., & Cosar, A. (2019). A survey on new generation metaheuristic algorithms. Computers & Industrial Engineering, 137, 106040.

Hussain, K., Salleh, M. N. M., Cheng, S., & Shi, Y. (2019). Metaheuristic research: a comprehensive survey. Artificial Intelligence Review, 52(4), 2191-2233.

Author Response

1. COMMENT

The deviations should be given with boxplots for the results.

RESPONSE

We have added two new boxplots in the revised version of the manuscript

1. One plot for the standard deviation of function calls without the gradient check mechanism.

2. One plot for the standard deviation of function calls with the gradient check mechanism.

The added text in subsection 3.3 now reads: “Also, the boxplots for the proposed stopping rules without and with the gradient check of equation eq:gradientCheck are illustrated in Figures and respectively.”

2. COMMENT

Please include the following references:

BoussaïD, I., Lepagnot, J., & Siarry, P. (2013). A survey on optimization metaheuristics. Information sciences, 237, 82-117.

Dokeroglu, T., Sevinc, E., Kucukyilmaz, T., & Cosar, A. (2019). A survey on new generation metaheuristic algorithms. Computers & Industrial Engineering, 137, 106040.

Hussain, K., Salleh, M. N. M., Cheng, S., & Shi, Y. (2019). Metaheuristic research: a comprehensive survey.Artificial Intelligence Review,52(4), 2191-2233.

RESPONSE

Done. The added text in the introduction section reads: “The reader can find some complete surveys about metaheuristic algorithms in some recent works [meta1, meta2, meta3].”

Reviewer 2 Report

The contributions and the hypothesis should be stated more clearly. The structure of the proposed algorithm should be explained.

Author Response

1. COMMENT

The contributions and the hypothesis should be stated more clearly.

RESPONSE

The following text has been added/modified in the revised version of the manuscript:

“The PSO method is an iterative process, during which a series of particles evolve through a process that involves updating the position of the particles and fitness computation, i.e. evaluation of the objective function. Variations of PSO that aim at the global minimum in a shorter time may include the use of a local optimization method in each iteration of the algorithm. Of course, the above process can be extremely time consuming and depending on the termination method used and the number of local searches performed may require a long execution time.

This text introduces three distinct modifications to the original method, which drastically improve the time required to find the total minimum by reducing the required number of function evaluations. These modifications cover a large part of the method: the speed calculation, a new method of avoiding running local search methods and a new adaptive termination rule.”

2. COMMENT

The structure of the proposed algorithm should be explained.

RESPONSE

The following text has been added at the end of Subsection 2.1 to clarify how the proposed modifications have been added in the base PSO algorithm.

“The current work modifies the above algorithm in three key points:

In step 2, where a new termination rule based on asymptotic considerations is introduced.
In step 3b, where the the algorithm calculates the new position of the particles. The proposed methodology modifies the position of the particles based on the average speed of the algorithm to discover new minimums.
In step 3c, where a method based on gradient calculations will be used to prevent the PSO method from executing unnecessary local searches. ”

Reviewer 3 Report

Review of the article „Towards an ideal Particle Swarm Optimizer for multidimensional functions by authors: Vasileios Charilogis, Ioannis G. Tsoulos

Shortcomings of the article:

In article show old literature sources. About 97 percent of cited articles more 5 years old. In the journal should be overviewed the only new information. The literature review needs to be extended to include new references. Introduction is expected to have an extensive literature review followed by an in-depth and critical analysis of the state of the art.

The review conclusions and aim of the research needs to be clearly stated and presented at the end of the introduction.

Experiment methodology not provided.

Picture without title.

Need to list the sizes units in the graph axes.

Not possible to end a chapter with a table (Subsection 3.2).

Needs add discussions section.

The table in the conclusions need to be moved to the results or annex section. The data in the tables must be presented graphically.

In the conclusions must clearly show what problems the researchers have solved and how much to get results are better than the results of other researches. The conclusions should be clear and concise with the numerical values provided to support and justify the results obtained.

The formalization of the article is not in accordance with the requirements of the journal.

The article is superficial and lacks completeness. The article submitted for review needs to be authors fully reordered.

Author Response

1. COMMENT

RESPONSE

A) We have added the following references in the introduction:

1. BoussaïD, I., Lepagnot, J., & Siarry, P. (2013). A survey on optimization metaheuristics. Information sciences, 237, 82-117.

2. Dokeroglu, T., Sevinc, E., Kucukyilmaz, T., & Cosar, A. (2019). A survey on new generation metaheuristic algorithms. Computers & Industrial Engineering, 137, 106040.

3. Hussain, K., Salleh, M. N. M., Cheng, S., & Shi, Y. (2019). Metaheuristic research: a comprehensive survey.Artificial Intelligence Review,52(4), 2191-2233.

and the added text reads: ” The reader can find some complete surveys about metaheuristic algorithms in some recent works [meta1, meta2, meta3].“.

B) A discussion on recent hybrid method used with PSO has heen added in the introduction along with the relevant references. The added text reads:

”The method of PSO has been integrated into other optimization techniques like the work of Bogdanova et al [ge_pso1] who combined Grammatical Evolution with swarm techniques like PSO [ge_mainpaper], the work of Pan et al [siman_pso] to create a hybrid PSO method with simulated annealing. Also, Mughal et al [simman_pso_energy] used a hybrid technique of PSO and Simulated Annealing for photovoltaic cell parameter estimation. Similarly, the work of Lin et al [pso_de] utilized a hybrid method of PSO and Differential Evolution for numerical optimization problems.” The added references are:

A. Bogdanova, J.P. Junior, C. Aranha, Franken-Swarm: Grammatical Evolution for the Automatic Generation of Swarm-like Meta-Heuristics, In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 411-412, 2019.
X. Pan, L. Xue, Y. Lu et al, Hybrid particle swarm optimization with simulated annealing, Multimed Tools Appl 78, pp. 29921–29936, 2019.
M.A. Mughal, Q. Ma, C. Xiao, Photovoltaic Cell Parameter Estimation Using Hybrid Particle Swarm Optimization and Simulated Annealing, Energies 10, 2017
G.H. Lin, J. Zhang, Z.H. Liu, Hybrid particle swarm optimization with differential evolution for numerical and engineering optimization. Int. J. Autom. Comput. 15, pp. 103–114, 2018.

2. COMMENT

The review conclusions and aim of the research needs to be clearly stated and presented at the end of the introduction.

RESPONSE

The following text has been added at the end of Introduction:

“From the experiments performed, it was shown that the proposed modifications significantly reduce the required execution time of the method by drastically reducing the number of function calls required to find the total minimum. Also, these modifications can be applied either alone or in combination with the same positive results. This means that they are quite general and can be included in other techniques based on PSO. ”

3. COMMENT

Picture without title. Need to list the sizes units in the graph axes.

RESPONSE

A new figure has replaced the old Figure 1 in the revised version of the manuscript.

4. COMMENT

The table in the conclusions need to be moved to the results or annex section. The data in the tables must be presented graphically.

RESPONSE

We have added three new plots in subsection 3.3 in the revised version of the manuscript:

1. One plot for the standard deviation of function calls without the gradient check mechanism.

2. One plot for the standard deviation of function calls with the gradient check mechanism. The added text in subsection 3.3 now reads: “Also, the boxplots for the proposed stopping rules without and with the gradient check of equation eq:gradientCheck are illustrated in Figures and respectively.”

3. One plot for the average function calls of the proposed termination rule on a series of test functions with and without the gradient check criterion. The added text reads: “Also, the effect of the application of the rejection mechanism based on gradients of the equation eq:gradientCheck is illustrated graphically in Figure fig:ExperimentsStopping, where the proposed termination rule is applied on a series of test functions with the gradient check and without the gradient check.”

5. COMMENT

RESPONSE

The following text has been added as discussion / conclusion in the revised version of the manuscript in Conclusions section

“The proposed modifications were applied to the pso method either one by one or all together in combination. The purpose of the method is to find the total minimum of continuous functions using the smallest possible number of function calls. The experimental results showed that the modifications significantly reduce the number of function calls even when not used in combination. This means that they can be used individually and in other variations of pso. The reduction in the number of function calls reaches up to 80%. In addition, the amendments did not reduce the ability of the PSO to find the total minimum of the objective function. In addition, the first modification reduces the number of required calls, but only when the criterion for avoiding local minimization is not present. ”

6. COMMENT

The formalization of the article is not in accordance with the requirements of the journal.

RESPONSE

Done.

Round 2

Reviewer 3 Report

Based on the corrections made, it can be concluded that the authors does not make enough efforts to improve the quality of the article. The article needs to be corrected carefully and responsibly in the light of the comments made earlier.

Shortcomings of the article:

The literature review is based on old literature sources. The literature review needs to be extended to include new references. Introduction is expected to have an extensive literature review followed by an in-depth and critical analysis of the new results of researches. Insufficient addition of new articles.

The literature analysis is very formal, for example: “This problem finds application in a variety of objective problems in the real world, such as problems of physics [1–3], chemistry [4–6], economics [7,8], medicine [9,10] etc.” (Lines 14-15). A more detailed analysis of the articles is needed.

The review conclusions and aim of the research needs to be clearly stated and presented at the end of the introduction.

Unclear why the authors write that performed the experiment, maybe word calculations would be more appropriate, because there is no experiment here. If there was an experiment what equipment was used, the methodology and so on. Neither equipment nor methodology is provided.

Need to list the sizes units in the graphs axes.

Not possible to end a chapter with a picture (Experimental results 3.2).

Needs add discussions section. In this section need to discuss about research results.

The conclusions are written: “...pirmoji modifikacija sumažina reikalingų skaičių skambučių…”. Question how much?

The article is superficial and lacks completeness.

Author Response

1) comment ". If there was an experiment what equipment was used, the
methodology and so on. Neither equipment nor methodology is provided." is completely irrelevant to our paper or any other
paper in computer science!! What does he mean with the word equipment? I am not a chemistry by the way
2) comment " Need to list the sizes units in the graphs axes." is the same as the previous review!!! We have added axes and labels in our figures, but he does not read the revised paper or he did not realize where our plot are located!
3) comment " Not possible to end a chapter with a picture (Experimental results 3.2)." is completely out of question to perform!! The position of the figures and
tables in LaTeX is arranged by the Latex system and the journal. I do not think that he can write a paper in Latex format!
4) comment "The conclusions are written: “...pirmoji modifikacija sumažina
reikalingų skaičių skambučių…”. Question how much?" is written in some other language than English. I strongly believe that he can not understand the paper and he uses Google Translate to read it !!!