Image Analysis Methods in Classifying Selected Malting Barley Varieties by Neural Modelling

Abstract

Quality evaluation of products is a critical stage in the process of production. It also applies to the production of beer and its main ingredients, i.e., hops, yeast, malting barley and other components. The research described in this paper deals with the multifaceted quality evaluation of malting barley needed for the production of malt. The project aims to elaborate on the original methodology used for identifying grain varieties, grain contamination degree and other visual characteristics of malting barley employing new computer technologies, including artificial intelligence (AI) and neural image analysis. The neural modelling and digital image analysis assist in identifying the quality of barley varieties. According to the study, information concerning the colour of barley varieties presented in digital images is sufficient for this purpose. The multi-layer perceptron (MLP)-type neural network generated using a data set describing the colour of kernels presented in digital images was the best model for recognising the analysed malting barley varieties. The proposed procedure may bring specific benefits to malthouses, influencing the beer production quality in the future.

1. Introduction

New information technologies are entering different sectors of the food industry. Wherever possible, computerisation and automation of production processes replace human labour [1,2,3,4]. This aims to improve production processes by introducing better efficiency and, at the same time, maintaining the good quality of generated products and reducing expenditure [5,6,7]. These modern solutions are also being introduced to the food sector, and beer production is one of the rapidly developing branches of this industry [8,9,10]. The efficiency of the processes carried out at the brewhouse depends mainly on the raw materials applied and the selection of adequate technological parameters. It is of great importance to develop new technologies and equipment and use new and selected raw materials to achieve the desired quality of the final product [11]. Moreover, the final product should fulfil certain parameters to meet consumer requirements [12,13]. The high quality of products determines the improvements in ingredient production technologies (Figure 1).

The primary ingredients necessary for beer production include malting barley and malt extracted from selected barley varieties [14]. The malted barley grains are used for producing enzymes in such a quantity that will enable far-reaching hydrolysis of the polymers contained in barley caryopses and their proper loosening to facilitate the extraction process [15]. Malt, i.e., germinated and dried grains of brewing barley, is ground and heated up in water (to extract the nutrients) until nourishing sugar and a protein-rich solution known as wort (pronounced as wert) are obtained. It is the perfect medium for the growth and fermentation of yeast [16]. The process of fermentation by inoculated yeasts is of fundamental importance for the aromatic profile of the produced beer. The polyphenols from malt and hops (added during the production) fundamentally influence the physical quality of beer [17]. Malting barley is used only in the malt industry. Malt must quickly achieve complete physiological maturity and must germinate quickly and evenly. A good level of grain uniformity is also an important factor. Brewing barley is associated with two-rowed barley. It is due to the structure of the grain, which should be well-filled, bulbous, barrel-shaped and symmetrical. It must retain the characteristics of the highest quality grain, with a delicate and thin husk [18]. It cannot be infested with pathogenic fungi and have a musty smell.

Barley grains used in the malting plant must be of high quality, with a strictly determined composition and low protein content because grains with a high protein content decelerate the relaxation (during grist production) and limit the malt extract efficiency. Varietal purity determines the type of the produced malt, and the malthouses mix preferred varieties to make the specific type of grist expected by the breweries [19]. Therefore, grains must always be checked for varietal purity and technological quality of the process [20].

Several grain testing methods are used, including immunological analysis, protein electrophoresis, DNA analysis, high-performance liquid chromatography, isoenzyme analysis, a combination of attenuated total reflectance mid-infrared spectroscopy and chemometrics, as well as quantitative trait locus (QTL) analysis [21,22,23,24,25]. However, most of these methods are labour-intensive and expensive, and the tests can be only performed at specialised laboratories.

Visual evaluation of grains with regard to the variety, in line with the International Rules for Seed Testing developed by the International Seed Testing Association (ISTA), constitutes an alternative approach [26]. The ISTA method is the primary tool to promote uniformity in the seed testing industry. Since the technique is much easier to conduct in malting plants, it is more common than chemical methods. Its reliability, however, to a large extent, depends on the skills and experience of the person conducting the evaluation. The process begins with choosing a representative sample of the purchased grain. It is followed by a visual assessment of grain quality, which involves manual segregation. Even though it is carried out by highly trained malthouse workers, such classification is subjective.

Another technique—computer image analysis—is fast, inexpensive and relatively effective. It is an alternative and prospective method of the assessment of selected physical attributes of kernels [20]. Computer classification of grain varieties has already been studied for many years [14,27,28,29,30]. This method involves the creation of numerous algorithms for feature computation, focusing on various aspects of an image, such as texture, brightness and colour, shape and topology of the region [31]. These are the characteristics that are taken into account in the grain analysis. Digital image analysis is a developmental technique that is based on the creation of advanced algorithms. Older classification models were based on analyses of whole kernel regions only [27,28], while for some time now, manufacturers have been conducting segmentation of kernel images (grain images) into specific areas that are individually checked using various algorithms. The suggested image analysis procedure is based on the assumption that certain traits are determined genetically [14,32,33,34].

In this project, the focus is directed to the visual assessment of contamination and the hue of the grains as well as barley varieties. Each characteristic poses certain difficulties in terms of its recognition. Polish standards (PN-R-74109 and PN-R-74110) are applicable with respect to barley contamination. These standards help identify certain grain groups, such as broken or ungerminated grains. However, there are some reservations concerning the qualification of grains based on the subjective opinions of laboratory technicians and impacts on their input (human-related factors, e.g., fatigue). Furthermore, there are no codified standards to assist in distinguishing malting barley varieties. Therefore, malthouses and specialised laboratories have to develop their own criteria for identifying grains. Now, experts use chemical tests to establish malt parameters that enable variety identification.

First and foremost, this aim of this project was to identify algorithms required to carry out a digital image analysis and select optimal neural models, establishing the topology of artificial neural networks (ANNs). ANNs were used for the following reasons: (i) Neural models are excellent classifiers commonly used in engineering practice, for example in the process of solving the so-called autonomous problems; (ii) the problem of over-fitting the classifier to the learning data, the solution to which is provided by the ANN simulator used during the research, which divides the learning set into training, validation and test sets. The error analysis of the generated ANN for individual sets can be used to identify the phenomenon of fitting, over-training along with other aspects important to this study and (iii) the generated neural classifier algorithms can be easily implemented in the subsequently developed information systems supporting the beer production process.

The presented technology may help to solve the problems of variety identification and calculation of contamination level to a standard that is close or even superior to human abilities. Adequate results may lead to the automation of the visual evaluation process of grains. The developed ANN could be described as a dedicated information tool to support decision-making processes in broadly understood beer production.

2. Materials and Methods

As part of the project, a variety identification of barley grains was performed. The investigation was carried out following the neural image analysis methodologies concerning corn kernels and rapeseed. The study was performed in Poland at the Poznan University of Life Sciences. In this paper, three selected spring varieties were analysed: Beatrix, Sebastian and Xanadu. Selected barley varieties represent standard specifications (see

Table 1. Details of the three selected varieties of barley.

Specification	Beatrix	Sebastian	Xanadu
Brewing quality	5.10	6.85	6.60
Extract potential	3	7	7
Wort viscosity	7	7	7
Kolbach index	7	6	6

) applied to malt production. The parameters of the above-defined varieties of barley were compared to four exemplary parameters (brewing quality, extract potential, wort viscosity and Kolbach index) indicated in Table 1, which are commonly used in a wide range of image analysis.

The first stage involved selecting representative samples of barley grains to capture images. For each variety, up to 700 individual grains were selected. They complied with general qualitative criteria, i.e., showing no signs of mechanical damage or disease. Grain images were taken using a special test stand [14,20] (see Figure 2). The stand was equipped with an independent source of light: eight LED bulbs (with colour temperature resembling daylight) and luminance of 5.6 klx (for all enabled bulbs; 5.6 klx: standard LED lighting parameters, klx—calibration factor applies to both lighting intensity and luminance measurements; luminance is a photometric quantity which measures the intensity of light emitted in a given direction, (cd/m²).

The images were captured with a Nikon D90 camera, AF-S Nikkor 18–70 mm 1:3.5–4.5G ED lens and 8× magnification (2 rings, 67 mm, +2, +4). The camera was set up on a tripod with a special mount which ensured that the camera lens was directed towards the substrate, i.e., directly at the grain placed on the background. The background for the grain was a specially prepared plate spray-painted in matt blue. This prevented reflections caused by the lighting and provided contrast for the barley grain, which further facilitated segmentation of the grain in the computer system (barley grain does not contain blue colour).

Next, the images were processed and analysed using the in-house developed software “Hordeum 2.0” and “Hordeum 3.1” (Figure 3 and Figure 4), using MATLAB 2011b. This software acquires information on the features (Figure 3) of the grains from the pictures and then processes them using neural network models employing an additional toolbox of MATLAB (Neural Network Toolbox). A total of 1000 digital images of randomly selected samples (112 grains per image) were taken. The 800 best-quality images of grains of three varieties (Beatrix, Sebastian and Xanadu) were selected for further analysis using developed IT system. Each picture provided 46 input variables (geometrical and non-geometrical):

• − Geometrical parameters, such as the grain area, circumference, height, width, inertia moments, Feret’s diameter, (maximum and minimum) radius from the centre of gravity, aspect ratio and dimensionless quantities (14 variables);
• − Shape-related factors, such as Feret’s, Malinowska, circularity, Blair–Bliss, Haralick and ellipticity (12 variables);
• − Colour values, such as the maximum and minimum values, mean and median and standard deviation (15 variables);
• − Texture-related coefficients, such as a breakup and coefficients of the co-matrix (five variables).

Figure 3. The structure of “Hordeum 2.0” software with four graphic descriptor groups.

Figure 3. The structure of “Hordeum 2.0” software with four graphic descriptor groups.

Figure 4. The main window of the new version of “Hordeum 3.1” software.

Figure 4. The main window of the new version of “Hordeum 3.1” software.

With the “Hordeum 2.0” software, it was possible to obtain a training set to generate artificial neural networks [14]. In the project, 46 input variables were divided into four separate data sets (see Figure 3) [14,21].

During the work, significant modifications were made to the code of the “Horodeum 2.0” application. The “Horodeum 3.1” software significantly supports the process of developing training sets necessary for creating neural models. The working window of the created application is presented below (Figure 4). Design and operation of in-house developed software “Hordeum 2.0” and “Hordeum 3.1” has been described in detail in previously analysed materials [2,14].

A schematic procedure for neural identification of selected malting barley varieties is shown in Figure 5.

The first stage of the procedure (image acquisition) consisted of the acquisition of adequate digital images. The second stage involved the identification of relevant parameters of grain images and their conversion into training sets. The third stage was the process of the generation of classification neural network models. As a consequence of the mentioned stages, a qualitative classification of barley was carried out.

The Root Mean Square (RMS) error is a usual measure of classification correctness of the generated ANN. This measure can be described as the total error of the network concerning the data set (training, testing and validation). It is worth noting that the determination of the RMS error for previously defined subsets of training data is a standard during the operation of many neural simulators implemented in various systems, such as MATLAB or STATISTICA. For the calculation, the following formula is used [2,4]:

$$RMS=\sqrt{\frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}{\left(y_i-z_i\right)}^2}{n}}$$

(1)

• N—number of cases;
• y_i—real values;
• z_i—values determined using the network.

RMS error is the root of the mean square error and is the total error made by the ANN for a previously defined data set (training, validation and test). It is determined by summing the squares of individual errors, dividing the obtained sum by the number of included values and determining the square root of the quotient obtained. The RMS error is usually the most interpretable single value describing the total error of the generated neural network.

The design of four training sets suitable for four generated sets (Figure 3) used in Neural Network Toolbox (MATLAB) is shown in Figure 6.

A block diagram showing the neural model generated using the Neural Network Toolbox (MATLAB) is shown in Figure 7.

Figure 7 shows the structure of the optimal MLP-type neural network (multilayer perceptron) which was presented in the standard notation used in the ANN simulator implemented in the MATLAB system.

3. Results and Discussion

We attempted to establish neural network models that could help classify varieties of malting barley using the Neural Network Toolbox. Processing of data sets using the MATLAB Toolbox provided the outcomes illustrated in

Table 2. The preliminary outcomes of neural network processing using the Neural Network Toolbox.

Model Specification	Geometrical Parameters	Shape Factors	Colour Values	Texture Coefficients
Best ANN	MLP 14:14-18-3:3	MLP 12:12-11-3:3	MLP 15:15-15-3:3	MLP 5:5-13-3:3
Quality of training	0.670	0.573	0.967	0.647
Quality of validation	0.660	0.593	0.952	0.633
Quality of testing	0.567	0.587	0.949	0.680
Training error	0.393	0.420	0.120	0.392
Validation error	0.400	0.422	0.122	0.410
Testing error	0.434	0.428	0.135	0.377

. Therefore, the numerical values in Table 2 represent the computer simulation results generated by the NNT simulator implemented in the computer system used (MATLAB).

Table 2 presents the generated network topologies (for four groups of image parameters). The optimal ANN, being a multi-layer perceptron, was obtained for colour values as the representative parameter, MLP 15:15-15-3:3. It is worth clarifying that MLP 15:15-15-3:3 represents a three-layer multilayer perceptron containing 15 neurons in the input layer (representing 15 colour characterising parameters), 15 neurons in the hidden layer and three neurons in the output layer of the generated ANN (see Figure 7).

The RMS error is represented by a numerical value facilitating interpretation that illustrates the total error of the ANN. In the case of MLP models, 15:15-15-3:3 (the neural network type generated by Neural Network Toolbox (MATLAB)), RMS errors were acceptable in terms of learning, validation and test sets. They are shown in Table 2 (a blue frame).

The comparison of the four neural network models showed that the best one for identifying the malting barley varieties was the multi-layer perceptron (MLP) neural network type generated with the third data set concerning colour information (see Table 2).

The MLP topology employing the 15:15-15-3:3 structure was considered an optimal artificial neural network (ANN). The input layer included 15 neurons demonstrating a linear postsynaptic function. The sole hidden layer had 15 sigmoid neurons. On the other hand, the network output had one sigmoid neuron that represented a three-state nominal variable. The neural model was trained using the back-propagation (BP) method in 10 cycles, including 1000 epochs each, and its training was further optimised with the conjugate gradient (CG) algorithm for 2000 epochs.

In conclusion, colour turned out to be an identifying trait, represented by 15 characteristic parameters in classifying selected malting barley varieties. To indicate the hierarchy of these parameters in terms of significance level, sensitivity analysis of the input variables to the performance of the neural model was performed (MLP 15:15-15-3:3). However, the primary objective of this study was only to identify the optimal neural classifier. The obtained results are sufficient for conducting further research in the discussed area. The optimal selected ANN model could be used in the new software to swiftly identify the varieties of barley grains.

The classification usually involves calculation of image attributes. An image attribute is a numeric quantity that is typical to the image or its fragment. As already mentioned in the introduction, numerous algorithms for feature computation focus on various aspects of the image. For example, Neuman et al. (1989) analysed colour characteristics of various types of wheat and demonstrated their significant differences [35]. Paliwal et al. (1999) focused on attributes such as colour and shape to distinguish grains of wheat, barley, oats and rye [36]. Szczypinski and Zapotyczny (2012) discovered main areas of a single barley grain, which enabled estimation of their rotation and quality [32]. For individual grain images, Koziołek et al. (2017) proposed the analysis of a single grain tool based on several descriptors, including colour [32].

However, it is difficult to classify grains if more than one can be seen in a single picture. The application of computer vision to visible-light pictures, along with soft computing and learning methods [37], expert systems [38] or a discriminant analysis [20], could be a solution to the problem. Moreover, the application of neural networks or the k-nearest neighbour algorithm to classify varieties of wheat and barley (using GLCM features) could also be helpful [39,40,41]. As literature shows, imaging systems employing different light spectra have also been applied [23]. Support vector machines (SVMs) are another form of classifier (in addition to the already mentioned k-NN and neural networks) that has proved its usefulness during studies concerning the quality of rice [42]. It should be noted that this classifier has considerable capabilities in terms of both grain and plant identification [43]. Interesting results were also published by Ramirez-Paredes and Hernandez-Belmonte in 2020 [31]. In their article, the authors presented a concept of machine learning (ML) algorithms designed to evaluate the quality of malting barley grain varieties. Several trait vectors combined with a non-linear classifier were compared; it was found that a local phase quantisation (LPQ) descriptor combined with features of colour and shape could provide even better results than improved local descriptors. Despite the above-mentioned significant advances, it should be noted that neural classifiers have the advantage of generating results in “real time” during operation. This gives them a significant advantage over other classifiers and has led to their use as modules in complex IT systems that support the broadly understood production.

The neural modelling and digital image analysis techniques used in this study enable effective identification of the quality of barley varieties. It has been found that the multilayer perceptron (MLP), successfully used in other disciplines, is the best model for recognising the analysed malting barley varieties [44,45]. The analysis results proved that in the process of grain classification, MLP sufficed to gather information concerning the colour of barley types presented in digital images. The presented technique has practical importance, as it can be applied to an automated qualitative assessment of selected malting barley varieties. It also shows potential as an effective tool supporting the decision-making process in beer production.

Compared to other methods, MLP is uncomplicated in terms of its implementation and very effective. The standard activation function for MLP networks is the logistic function (also called the sigmoid function). In the ANN simulator implemented in the MATLAB system, it was used by default in all layers of the generated ANN. Currently, MLPs are among the most popular network architectures, particularly as effective neural classifiers. MLP networks can be trained using many algorithms, such as the conjugate gradient method, quasi-Newton algorithm, Levenberg–Marquardt algorithm, backpropagation error algorithm, fast propagation algorithm and delta-bar-delta algorithm.

To sum up, the proposed procedure may facilitate the method of barley assessment compared to the current method used by manufacturers. In addition to providing specific utility benefits for the malthouse, it may also positively affect the quality of beer production.

4. Conclusions

The study results proved artificial neural networks to be helpful in identifying selected varieties of malting barley. The MLP-type neural network with the 15:15-15-3:3 structure generated using a data set describing the colour of kernels presented in digital images was the best model for recognising the analysed malting barley varieties. The information concerning the colour of barley grains encoded in a set of 15 selected coefficients (graphic colour descriptors) proved to be the most significant in the identification process. The sensitivity analysis of all four generated neural models, especially in terms of empirical data reduction (e.g., using the Karhunen–Loeve transform or KLT method) is the subject of ongoing work.