**4. Conclusions**

This work aimed to reinforce the methodology for sensory analysis of VOOs through adoption of supporting tools for training and monitoring of sensory panels. The results obtained from the sensory evaluation carried out by the six panels involved in the OLEUM project on a set of 334 samples confirmed the effectiveness of the application of the panel test. However, at the same time, it also confirmed that there are some critical issues related to questionable results in the case of: (i) borderline oils (between two product categories); or (ii) misalignments on the main perceived defect by panels when more than one negative attribute was present in the oil. The adoption of a decision tree based on the agreement of a category, main perceived defect, and application of formative reassessment in case of misalignments using the same reference materials (samples already classified by the six panels with an high agreement) allowed for reliable classification of oils that, at first evaluation, were borderline. Only 45 of 334 oils were reassessed (formative reassessment) and 41 of 45 samples were definitively classified, confirming the importance of alignment between panels, which can be achieved by sharing the same sensory reference materials. In fact, sensory information on both quality grades of samples and main perceived defect/s is fundamental for testing possible correlations with physical–chemical data and/or for building classification models; in this way, instrumental screening approaches can allow for a reduction in the number of samples that have to be assessed by panels,

excluding, for example, oils defintely classified by chemometric models as extra virgin or lampante, focusing the sensory analysis on samples that are not classified or classified with a low probability. This thus reduces the number of samples to be assessed by the sensory panel. The data provided by the panels were also used to verify performance in terms of discriminating capacity, agreement between panels, and accuracy of results by applying some of the procedures reported in the IOC guide for internal quality control of sensory laboratories. In general, the panels showed very good (sometimes excellent) performance even if, in some cases, problems were noted that were related to the use of the scales, lack of recognition of some sensory defects, or intensity values that were too distant between panels for the same sample, especially in the case of oils in which more than one defect was perceived. The large set of samples evaluated over 2 years allowed estimation of the performance of the panel test: the utility and peculiarity of this official method is undisputed, also considering that it has definitively improved the quality of VOOs over the last 28 years, opening the possibility to have a wide range of excellent oils with a deserved added value on the market. On the other hand, to improve its effectiveness, it is necessary that the sensory panels perform organoleptic evaluation by applying specific guidelines [6] and quality control of panel performance [25] in a rigorous manner. To enhance panel skills in recognizing, identifying, and quantifying sensory attributes, the use of new reliable reference materials is of absolute necessity. They could be both "synthetic", resembling a single negative attribute (e.g., rancid or viney-winegary ) or biotechnologically formulated, in the latter case being closer to actual virgin olive oils. The first type could be used to overcome some of the limitations of the natural matrix and offer advantages such as feasible preparation in each laboratory (open access composition), reproducibility over time, possibility of purchase, and therefore diffusion and availability for the global market. Even the cultural aspects related with knowledge of the sensory aspects of VOOs, i.e., the global recognition of its positive/negative attributes, could also be facilitated by the availability of these "simplified" materials; the formulation and validation of two of these "synthetic" sensory reference materials (rancid and winey-vinegary ) are still in progress within the framework of the OLEUM project. On the other hand, the use of the OLEUM decision tree could be an adequate instrument to classify natural sensory reference materials, for example, those obtained by biotechnological processes (programed fermentations for fermentative defects) or oxidation (for the nonfermentative rancid defect), the availability of which is also fundamental to achieve alignment between panels, thus reducing cases of discordant classifications, which is of vital importance for global trade and product reputation.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/2304-8158/9/3/355/s1, Table S1 (a–d): Sensory results of samples from the first year, Table S2 (a–d): Sensory results of samples from the second year.

**Author Contributions:** Conceptualization, S.B., A.B. and T.G.T.; formal analysis, K.B.B., M.B.-M., F.L., U.T. and O.W.; data curation, S.B. and A.B.; writing—original draft preparation, S.B.; writing—review and editing, S.B., A.B., K.B.B., M.B.-M. and T.G.T.; supervision, T.G.T. and D.L.G.-G.; project administration, T.G.T.; funding acquisition, T.G.T. All authors have read and agree to the published version of the manuscript.

**Funding:** This work is supported by the Horizon 2020 European Research project OLEUM "Advanced solutions for assuring the authenticity and quality of olive oil at a global scale", which received funding from the European Commission within the Horizon 2020 Programme (2014–2020), grant agreement No. 635690.

**Acknowledgments:** The information expressed in this article reflects the authors' views; the European Commission is not liable for the information contained herein. We are grateful to all producers who provide us with virgin olive oils for this study as well as all panel members who performed sensory analysis of virgin olive oils from each institution involved: Eurofins Analytik GmbH, Hamburg, Germany; Institute of Agriculture and Tourism, Poreˇc, Croatia; Institut des Corps Gras, Pessac, France; Alma Mater Studiorum—Università di Bologna; Science and Research Centre Koper, Slovenia and Ulusal Zeytin ve Zeytinya ˘gı Konseyi, Izmir, Turkey.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
