**5. Discussion**

In this section, the results from the experimental assessment introduced previously are discussed (The results, plots and source code are available at https://github.com/Tomas-Morph/MenuPlanning\_ MOEAD\_Mathematics). As mentioned earlier, the results attained by ILS-MOEA/D were compared to the results achieved by MA. The reader should recall that the working hypothesis behind this comparison is that by applying ILS-MOEA/D to a multi-objective formulation of the MPP, which considers not only the cost of the plan but also the level of repetition, it is possible to find solutions that are similar in terms of the cost to those provided by MA, but significantly better with respect to the level of repetition of specific courses and food groups contained in the menu plan.

First of all, MA seeks to obtain the cheapest menu plan that satisfies all the constraints defined in Section 2. Consequently, the menu plans generated by MA had a higher level of repetition in comparison to those provided by ILS-MOEA/D, as shown in Figure 1. The statistical procedure introduced previously shows that there were significant statistical differences in the results of ILS-MOEA/D and MA. Although an explicit diversity managemen<sup>t</sup> strategy was implemented in MA, not considering the level of repetition in the single-objective formulation of the MPP led to more affordable but less varied menu plans. In fact, in terms of the cost, the results obtained by MA were statistically better than those achieved by ILS-MOEA/D for the three instances considered.

Since ILS-MOEA/D considers the level of repetition as one of the objective functions to be optimised, the menu plans generated by this algorithm had a better ratio between the cost and the level of repetition. Considering the level of repetition, ILS-MOEA/D provided statistically significant better results than MA for the three instances. Furthermore, as Figure 1 shows, although the results of ILS-MOEA/D in terms of the cost were noticeably more scattered than those attained by MA, in a few executions, ILS-MOEA/D yielded the best menu plan cost obtained by MA with a significantly lower level of repetition, in the case of *n* = 20 and *n* = 40 days. As the results in Table 3 show, for *n* = 20 days, the mean cost of the solutions provided by ILS-MOEA/D was only 0.17% higher in comparison to the mean cost of the solutions attained by MA. However, the mean level of repetition of the solutions obtained by MA was 28.3% higher when compared to the mean level of repetition of the solutions given by ILS-MOEA/D.

**Figure 1.** Boxplot representation of the values obtained by memetic algorithm (MA) and iterated local search (ILS)-multi-objective evolutionary algorithm based on decomposition (MOEA/D) for both objective functions, i.e., cost and level of repetition of specific courses and food groups, considering menu plans for *n* = 20 (**upper-left**), *n* = 40 (**upper-right**) and *n* = 60 (**bottom**) days.

Something similar happened for the menu plans for *n* = 40 days. In this case, the mean cost of the solutions provided by ILS-MOEA/D was only 0.21% higher in comparison to the mean cost of the solutions attained by MA. The mean level of repetition of the solutions obtained by MA, however, was 38.5% higher when compared to the mean level of repetition of the solutions given by ILS-MOEA/D. Finally, for *n* = 60 days, the mean cost provided by ILS-MOEA/D was only 1.05% higher in comparison to the mean cost attained by MA. Still, the mean level of repetition of MA was considerably larger than that obtained by ILS-MOEA/D, specifically, 25.4%. Bearing the above discussion in mind, it is clear that the slight difference obtained by ILS-MOEA/D, in terms of the cost, is much lower when compared to the difference between the two approaches in terms of the level of repetition. As a result, ILS-MOEA/D clearly outperforms MA in this regard, which confirms our hypothesis.

In order to graphically confirm the above conclusions, the best solution found by MA, i.e., that with the lowest cost, was compared to the first of the thirty Pareto Fronts obtained by ILS-MOEA/D for each instance considered. To do this, the level of repetition of the solutions obtained by MA had to be calculated independently by considering the same source code implemented in the case of the multi-objective constrained formulation of the MPP. Figure 2 undoubtedly shows how MA obtains the best results in terms of the menu plan cost, for every instance. Nevertheless, the level of repetition of said solutions is significantly higher when compared to the level of repetition of the solutions belonging to the Pareto Fronts provided by ILS-MOEA/D. Note also how ILS-MOEA/D yielded

different Pareto Front shapes, depending on the instance size. Bearing the above in mind, it is difficult to apply a priori methods to focus the search on a certain region of the Pareto Front. Finally, we note that one of the main advantages of applying a multi-objective optimiser, such as ILS-MOEA/D, is that a diverse set of solutions, regarding the objective function space, is provided. All of these solutions are trade-offs between cost and level of repetition, and the above allows the decision maker to select one of those solutions depending on their particular requirements, something that is not possible when solving a single-objective formulation of the MPP.

**Table 3.** Statistics for the cost and level of repetition achieved by ILS-MOEA/D and MA, considering menu plans for *n* = 20, *n* = 40 and *n* = 60 days.


In the previous experiment, ILS-MOEA/D was unable to achieve the lowest cost provided by MA for a menu plan of *n* = 60 days in any execution. In order to determine if ILS-MOEA/D could yield the results attained by MA in large instances, such as *n* = 60, executions using a stopping criterion equal to ten hours were also performed. As in the previous experiment, 30 repetitions were also run by applying the same parameterisation of ILS-MOEA/D. The results obtained are shown in Table 4. We note that even though ILS-MOEA/D could not replicate the best menu plan cost obtained by MA after a five-hour execution (see Table 3), its results were improved in terms of both cost and level of repetition. As a result, devoting more effort to the development of ILS-MOEA/D could improve its performance when dealing with large instances.

Single-objective techniques that do not include any explicit diversity managemen<sup>t</sup> mechanism may fall into premature convergence. This means that solutions only improve for a short period of time before stagnating at local optima. At the same time, multi-objective optimisers may also converge to sub-optimal regions of the decision variable space of some MOPs, yielding effects that are similar to premature convergence [42]. Although our proposal ILS-MOEA/D does not include any explicit technique for properly managing diversity in the decision variable space, it would be interesting to analyse if the above could be possible implicitly as a consequence of promoting diversity in the objective function space.

**Table 4.** Statistics for the cost and level of repetition achieved by ILS-MOEA/D, considering menu plans for *n* = 60 days, after ten hours of execution.


**Figure 2.** Comparison of the Pareto Fronts achieved by ILS-MOEA/D versus a two-dimensional representation of the best solution found by MA, i.e., the one with the lowest cost and lowest level of repetition, considering menu plans for *n* = 20 (**upper-left**), *n* = 40 (**upper-right**) and *n* = 60 (**bottom**) days. The reader should recall that the level of repetition of the solutions attained by MA had to be computed separately. To this end, the same source code implemented in the case of the multi-objective constrained formulation of the MPP was considered.

Figure 3 shows, in the case of MA and ILS-MOEA/D, how the mean diversity in the population, considering the decision variable space, evolves during the runs for each instance considered. Diversity was computed by applying a distance-like function specifically created for the MPP formulations considered herein. This function determines distances among the courses assigned to the different days of the plan. Further details about the distance-like function can be found in the reference work [3]. Now how the mean diversity gradually decreases throughout the executions for both MA and ILS-MOEA/D, starting from a more diverse population and finishing with one that is less diverse. Figure 4 shows the evolution of the mean *HV* over the course of the ILS-MOEA/D executions. Not only does the mean diversity gradually decrease throughout the executions for every instance in the case of ILS-MOEA/D, but the mean *HV* also increases, which means that the effectiveness of ILS-MOEA/D is suitable. Since ILS-MOEA/D implicitly promotes diversity in the objective function space, diversity is also properly managed in the decision variable space. At this point, we should note, however, that MA preserves, in a smarter and more explicit way, a larger diversity in the decision variable space during the whole execution in comparison to ILS-MOEA/D, which results in better cost solutions, something that we had already stated in previous experiments.

**Figure 3.** Evolution of the mean diversity of the population in the decision variable space, considering menu plans for *n* = 20 (**upper-left**), *n* = 40 (**upper-right**) and *n* = 60 (**bottom**) days.

Consequently, we note that not considering explicit diversity managemen<sup>t</sup> schemes in the decision variable space in the case of a multi-objective algorithm, like ILS-MOEA/D, seems to impact the performance less than not considering them in the case of a single-objective optimiser. Even so, it would be interesting to check in the future whether explicitly managing diversity in the decision variable space helps ILS-MOEA/D to improve its performance in terms of the cost of the resulting menu plans.

Lastly, regarding the time complexity of the proposed algorithm, the average elapsed time and generations performed by ILS-MOEA/D are presented in Table 5. As it can be observed, ILS is the most computationally expensive step of ILS-MOEA/D, since it involves the majority of the total computational work. In particular, more than 99% of the total elapsed time of the algorithm is performed by ILS.



**Figure 4.** Evolution of the mean *HV* over the course of the executions, considering menu plans for *n* = 20 (**upper-left**), *n* = 40 (**upper-right**) and *n* = 60 (**bottom**) days.
