Approach of AI-Based Automatic Climate Control in White Button Mushroom Growing Hall

Abstract

Automatic climate management enables us to reduce repetitive work and share knowledge of different experts. An artificial intelligence-based layer to manage climate in white button mushroom growing hall was presented in this article. It combines visual data, climate data collected by sensors, and technologists’ actions taken to manage climate in the mushroom growing hall. The layer employs visual data analysis methods (morphological analysis, Fourier analysis, convolutional neural networks) to extract indicators, such as the percentage of mycelium coverage and number of pins of different size per area unit. These indicators are used to generate time series that represent the dynamics of the mushroom growing process. The incorporation of time synchronized indicators obtained from visual data with monitored climate indicators and technologists’ actions allows for the application of a supervised learning decision making model to automatically define necessary climate changes. Whereas managed climate parameters and visual indicators depend on the mushroom production stage, three different models were created to correspond the incubation, shock, and fruiting stage of the mushroom production process (using decision trees, K-nearest neighbors’ method). An analysis of the results showed that trends of the selected visual indicators remain similar during different cultivations. Thus, the created decision-making models allow for the definition of the majority of the cases in which the climate change or transition between the growing stages is needed.

1. Introduction

White button mushrooms (Agaricus bisporus) are among the top cultivated mushrooms, and they are known for their culinary, medicinal and cosmetic values [1]. In the beginning of the industrial production, they were grown in caves [2] and later the production process was moved to climate-controlled growing halls. The growing process can be divided into stages of mycelium development (homogenization) and body growth (fruiting) [2,3]. In other sources, the growth process is subdivided to mycelium growth, colonization, pinning, and flushes [4,5]. Each stage requires different environmental parameters of a specific range controlled in a timely and appropriate manner [3]. The process is governed by a technologist who predefines the setpoints of the environmental parameters and modifies them later in the process. Thus, the decisions to modify the environmental parameters are subjective and depend on the expertise level and experience to evaluate mushroom quality.

Industrialized mushroom production in closed halls is similar to greenhouse production. The overview of the internet of things (IoT), artificial intelligence (AI), and big data technologies in smart farming is provided in [6]. The authors emphasize the benefit of intelligent systems as they enable the automatic management of the climate, the reduction of the level of human interaction, and therefore the reduction of the expenses for human resources. The environment control problem is to create a favorable environment for mushrooms to get the predetermined results of high yield, high quality, and low production costs. Appropriate climate control can also help to detect diseases [7] and minimize their effects [8]. The problem is complex, as the environmental parameters have strong nonlinear relations and display dynamic behavior. Moreover, there are some practical constraints caused by a micro-climate in the growing hall, stages of the growing cycle, outside weather, and others [3]. It was demonstrated that transforming a conventional greenhouse into a mushroom house with the IoT system to control the environment contributes to creating optimal growing conditions for the mushrooms despite the outside weather conditions [9].

The design of climate control systems is based on either conventional or optimal climate theory [10]. For the climate control systems based on the conventional theory, the goal is to adjust the variable of interest as close to the set point as possible [11]. Systems based on the optimal climate theory consider various parameters, including experts’ knowledge, product response, and greenhouse behavior to create the best climate for the crop. Various methods, such as fuzzy logic [3,8,12,13,14], evolutionary algorithms [15], artificial neural networks (ANN) [16,17,18], linear programming [19], and other algorithms can be applied to design both conventional and optimal greenhouse climate control systems [10]. Smart agricultural systems deal with security issues and computational resources in the process of handling a massive amount of data for the artificial intelligence model and storage from the acquisition to decision-making [20]. The technical challenges of the smart greenhouses include connectivity and recharge issues and the need to constantly use up-to-date technologies [21]. The authors also emphasize the need to incorporate experts’ knowledge about the cultivation process into the intelligent greenhouse systems.

One of the most popular approaches is the application of fuzzy logic. It enables the system to convert linguistic control strategy to automatic control strategy [8]. The fuzzy logic was employed to control the greenhouse climate in the automatic climate control system [12]. When taking sustainable development into account, the system considers the energy consumption, efficiency, and correlation of variables. The multivariate fuzzy controllers were designed to monitor and control environmental parameters (ambient temperature, ambient humidity, CO₂ concentration, temperature, and moisture of the compost) in the mushroom house for the industrialized production of Agaricus bisporus [3]. In the application, the control system is combined with the growth of the mushroom because different growth stages require different environmental parameters which directly affect the quantity and quality of the mushroom body. Two systems based on fuzzy logic and On/Off were developed to keep the preset temperature, relative humidity (RH), and CO₂ concentration in a mushroom growing hall [13]. The results showed that the system based on fuzzy logic has higher accuracy and stability; therefore, results in lower energy consumption. Fuzzy logic was employed to design a water control system in the open oyster mushroom farm to adjust the amount of water with respect to the temperature, humidity, and ventilation [22]. The ANN and fuzzy logic were applied to develop an irrigation management system to prevent frost in the greenhouse [23]. The mentioned systems employ the knowledge base of experts and do not improve their suggestions in time.

The artificial neural network was applied to predict temperature variation in the mushroom growing hall with input parameters such as ambient temperature, water temperature, fresh air, and circulation dampers [16]. ANN-based control schemes for the predictive time series modeling of the greenhouse climate control system were presented to control internal temperature and humidity [18].

Visual information can also be useful in greenhouse production. It enables the detection of diseases [7,24] and the state of the plants [25], the determination of the picking time and control strategy for the harvesting robot [26,27,28], the optimization of climate control, the estimation of the growth rate, and the generation of a harvest reminder [29]. The number of mushrooms and size of the mushroom caps can be used as a parameter in climate control systems [30]. A convolutional neural network (CNN) based algorithm to measure the size of white button mushroom caps through the entire fruiting period was introduced in [30]. The algorithm to get a more accurate size by detecting circles of round caps was proposed in [29]. The algorithm includes CNN, K-means, contour detection, and regional analysis. During the growing process, mushrooms overlap with each other, and the segmentation of mushrooms faces additional challenges. Image edge gradient characteristics, filtering, morphological analysis, and least square ellipse fitting algorithm were employed to segment overlapping Agaricus bisporus to use it as input for the harvesting robot [31]. All algorithms can be applied if the images are taken from above with the camera installed parallel to the ground.

In this paper, the approach to automatic climate control according to visual and climate information in white button mushroom cultivation hall is presented. It combines computer-vision and decision-making algorithms with human knowledge and experience. The paper is organized as follows. In Section 2, the approach design, data collection procedure, methods used in image preprocessing, and climate management algorithm are explained in detail. Section 3 consists of an approach analysis including the overview of collected data; results of morphological, Fourier, and object detection analysis through the cultivations; and development of the decision-making model. The paper ends with a discussion of the application of the approach in the real environment and conclusion sections.

2. Materials and Methods

2.1. Layer Design

The mushroom production process is based on a well-established procedure with climate parameters selected from a specific range. A popular approach is to manage climate automatically to adjust the parameters predefined by the technologist while during the growing process, technologist performs climate correction to the process deviations. However, this approach highly depends on the knowledge and experience of the single technologist because there are no strict rules which describe optimal climate management for each situation. In addition, manual monitoring of the growing process to take necessary actions on time is a time-consuming process and has a high-risk level for human error, especially if there are many halls at different growing stages under the technologist’s care. In the presented approach, a new layer with an AI-based component is included, which enables knowledge sharing and improves the decision-making process.

The general scheme of the process is presented in Figure 1. The currently used climate management system consists of a user interface, climate controllers, and sensors. It enables us to remotely set the values for climate parameters and monitor and control them in the mushroom growing halls. However, it adjusts the values to fit the predefined set points and does not make decisions or suggestions on how the climate parameters should be changed regarding the actual state of the mushrooms. The standard climate management system (Figure 1, black components) is extended with the AI-based component (Figure 1, blue components). The AI component combines a collection of structured data and decision-making parts. The climate data about the cultivation is supplemented with visual information: a series of images in different locations of the mushroom growing hall. Visual data (images and results of the analysis), climate parameter values (determined by the technologist and specific for each hall), and administrative data are stored in the storage element. The features extracted from the images are used to identify the actual state of the mushrooms and the dynamics of the cultivation. These features together with the values of climate parameters from the standard climate management system are employed to suggest the changes for the set points. In case changes are needed, they are transmitted to the controllers through the standard climate management system.

Implemented schema with a collection of historical data enables not only its direct use in the decision making model construction, but also the implementation of knowledge sharing and the ability to evaluate the quality and management of the data during the growth process. The changes made by the technologists are used as the ground truth when developing the decision-making model. Such an option has been selected instead of experimentally trying to define optimal scenarios for climate management due to the risk that an experimental decision may damage the full growth cycle and waste the current growth production. However, decisions made by humans are often subjective or can even be qualified as human error. For example, the technologist’s decision to start shock too late would result in an incorrect decision made by the model after training. Thus, to ensure the quality of the data used for training, all changes made by the technologists, the visual data, and the climate data are reviewed at the end of cultivation by removing the actions that were not suitable for training.

2.2. Climate Data Acquisition

Climate data was acquired by analyzing the time series of climate parameters during historical cultivations in which the changes were made by the experts. The time series are generated from the values of the climate parameters observed in 10-min intervals. Such a solution has been caused by the limitations of the main climate management system to collect data about the transition between states and changes. In the analysis of the historical time series of the climate parameters, the experts define rules to identify transition moments between the stages and baselines of important climate parameters in each stage. The summarized information is provided in

Table 1. Climate parameters observed in different stages of the production cycle and their termination conditions.

Stage	Observed Parameters	Criteria for Stage Termination in Historical Analysis
Inoculation (firstStage)	Set point of compost temperature	Moment of the first observed CO2 set point value
	Set point of CO2
	Out air flow
Incubation (secondStage)	Set point of CO2	Moment of the sharp decrease of CO2 value
	Set point of room humidity
	Set point of room temperature
	Out air flow
Shock, initiation of fruiting (shockStage)	Set point of CO2	Moment the CO2 value changes from gradually decreasing to constant
	Set point of room humidity
	Set point of room temperature
	Out air flow
Fruiting (pinning) (growingStage)	Set point of CO2	The end of flush formation (does not exceed 18 days of cultivation process)
	Set point of room humidity
	Set point of room temperature
(finalStage)	-	10 days after the end of flush formation

. Thus, the termination time of each stage was identified automatically with respect to the termination criteria defined in Table 1. Each stage starts with the end of the previous stage.

The production cycle starts with inoculation (firstStage) during which the spawn is introduced to the substrate and prepared for the further process by layering them on shelves and watering. The incubation (secondStage) starts when the mycelium leaps-off the spawn and starts growing on the substrate. After the mycelium colonizes the substrate, the shock (shockStage) can be initiated. A seasonal change is simulated in this stage by lowering the temperature and CO₂ concentration, and it results in the formation of fruits. The mushrooms are formed in the fruiting stage (growingStage) by adjusting the appropriate values of room temperature, humidity, and CO₂ concentration. The final stage (finalStage) consists of mushrooms growing until harvesting and the fruiting of secondary flushes. The full process lasts approximately 30 days.

The automatic management process was focused on incubation, shock, and fruiting stages. Aside from state transition actions (provided in Table 1), actions of climate corrections may be performed in each stage. The set of possible actions and criteria to identify them is provided in

Table 2. Criteria to identify actions in different stages of production cycle.

Stage	Action	Criteria
Incubation	Correction of parameters	Any change of observed parameters
Shock	Correction of parameters	Any change of observed parameters
Fruiting	Stimulate pinning	Decrease CO2 level and (or) temperature
	Slow down pinning	Increase CO2 level and (or) temperature
	Other correction	Change which does not meet criteria for actions of stimulate and slow down pinning

.

The example of managed parameters including stage transition and actions taken during the single cultivation process is provided in Figure 2.

2.3. Image Acquisition

The images for the system were taken in the active production hall. During the cultivation process, white button mushrooms grow in the substrate which is placed on the shelves. The height of the substrate is approximately 20 cm with a 5 cm casing layer. There are 4 shelves stacked vertically. The distance between two parallel shelves is ~25 cm. The distance is too short to place a camera between the shelves to take images vertically. Thus, it was decided to take pictures at an angle from the side of the stack of the shelves.

The progress of the mycelium spread, and mushroom growth speed depends on the microclimate, which is created because of location in the mushroom growing hall and level of the shelf. For example, the distance to ventilation sources, the gap between the ground, and the lowest shelf compared to the gap between the two lowest shelves can cause a difference in the actual values of the climate parameters and, therefore, impact the growth progress. Thus, it is not enough to make climate change decisions based on the visual information extracted in one location. Using multiple stationary cameras limits the number of observed locations by the number of camera devices, requires ensuring an internet connection, and requires additional steps in preparing data for the decision-making model. Thus, it was decided to construct a system that enables the movement of one camera device vertically (between the shelves) and horizontally (along the shelf) and the observation of a predefined number of locations (Figure 3a). The images are taken at 4 positions on one shelf. The top shelf is not monitored due to a different microclimate emerging because of the significantly longer distance from the shelf to the ceiling of the growing hall. To sum up, 12 images are generated during one cycle which is repeated every 4 h during cultivation process. The light needed for mushrooms to grow is too dim to get images of a good enough quality for the analysis. Thus, the bright light was used while images were taken. Due to the vibrations during camera motion, the camera stops at a slightly different position and takes a photo of a slightly different area in every cycle of taking photos. It is important to monitor the dynamics of the mycelium growth, so the markers from the dictionary of CHARUCO library are applied to define the position identifier. Another dictionary from the same library is used to define the identifier of the mushroom growing hall. The monitored area is detected during the analysis of every image based on the positions of the yellow part of the yellow-blue sticks. Only the upper part of the sticks is used for calibration because the lower blue part is partly covered with mushrooms in late growth stages. The sticks are in the corners of a square, and each part of the stick (blue or yellow) is 5 cm long. The distance between the sticks can be defined dynamically; 30 cm was used in the examples. The known distance between the markers is also applied to estimate the size of the mushroom caps. The results of the calibration for the images taken on the 6th, 10th, and 15th days of the cultivation process for the same position are shown in Figure 3b–d.

There are images that cannot be used in the decision-making due to various irregularities that occurred in the image acquisition process. A few examples of such images are provided in Figure 4:

• Images were calibrated incorrectly due to bad camera positioning (Figure 4a,b), and such calibration leads to distorted images that cannot be used in the numerical analysis.
• The external light source caused a significant change in the environment, and it was not possible to define room and position identifiers (Figure 4c).

2.4. Image Preprocessing

Various parameters, such as the mycelium coverage percentage, number of mycelium components, number of detected mushrooms of a different size, and others are used to determine the dynamics of mycelium and mushroom growth in different stages of the mushroom production process. The features extracted using the image analysis methods are discussed in the further sections. For the morphological and Fourier analysis, the monitored area is transformed to a square gray image (Figure 5a). To detect mushrooms in the monitored area, the image of it is divided in two parts (Figure 5b). This prevents the system from using computational resources to detect objects in irrelevant areas of image and enables the maintenance of high resolution in the images under analysis.

2.5. Methods for Image Analysis

2.5.1. Fourier Analysis

The concept of the Fourier analysis is to decompose signals into harmonic functions. An image is a 2D signal from the mathematical point of view. In the Fourier analysis, the image is transformed into frequency data, and the intensity represents the amplitude of the function. Low frequencies correspond to areas of low variation in intensity. High frequencies represent fine details and edges. The images were reconstructed using only those values from the magnitude spectrum that pass the respective filter. The examples of Fourier analysis are displayed in Figure 6 for the images taken with a 96-h gap between the adjacent images. The three types of filters (low-pass, medium-pass, and high pass) are used to define the numerical parameters of an image. The results of the Fourier analysis used as derived input features in AI-based system are as follows:

• the mean intensity value of reconstructed image based on the filtered frequencies;
• the standard deviation of the reconstructed image based on the filtered frequencies.

The mean intensity values represent the mean brightness of the reconstructed image. As the mushrooms grow, the images constructed of low frequencies become brighter and, therefore, the mean intensity value increases. Similarly, the levels of intensity become more diverse as the different mushroom sizes and forms appear. This results in an increase in the standard deviation of the intensity values. These features enable the identification of the change in the dynamics of the mushroom growing process.

2.5.2. Morphological Analysis

During the early stages of mushroom production, it is important to evaluate the dynamic of the mycelium growth, especially in the early stages of cultivation (before the pinning). The number of components represents the number of separate areas where the mycelium appears on the surface of the compost. A morphological analysis is used to process pixels in their neighborhood to find the connected components of a significant size. A threshold value is selected to filter out noise components. The examples of the morphological analysis results are provided in Figure 7d–f) for the respective grayscale images in Figure 7a–c). There is a 48-h time gap between the moments two adjacent images were taken. In the earliest image (Figure 7d), the number of components is high, and the average size is small compared to the later examples. As the time passes, separate components connect in between and form the components of large areas (Figure 7e,f).

The results of the morphological analysis are used as features in the AI-based system:

• number of all components;
• number of the components filtered by size;
• coverage coefficient (the ratio of summed pixels of filtered components and the number of image pixels);
• average size of the filtered components;
• standard deviation of the size of filtered components.

Obviously, the absolute numbers of the parameters depend on the monitored area and change in parameters is applied to evaluate the quality of the cultivation process and to make suggestions.

2.5.3. Object Detection

In later stages of mushroom production, it is important to monitor the dynamics of the mushroom growth. Faster R-CNN is employed to detect the mushrooms. The architecture of the Faster R-CNN combines a convolutional neural network, regional proposal network, and fully connected neural network (20). After preparing two images to represent different sides of the monitored area, Faster R-CNN is applied to detect bounding boxes of the mushrooms (Figure 8). The results are aggregated to show the detected objects in the initial image. The diameter of each object is estimated proportionally to the known distance between the markers of the monitored area. Object detection results used as features in AI-based system are as follows:

• Number of objects with diameter <5 mm;
• Number of objects with diameter of 5–10 mm;
• Number of objects with diameter of 11–20 mm;
• Number of objects with diameter of 21–30 mm;
• Number of objects with diameter of 31–40 mm;
• Number of objects with diameter >40 mm;
• Total number of objects.

2.6. Automatic Climate Management Approach

It is well known that climate parameters in the mushroom growing hall depend on the production stage. The appropriateness of the visual data cannot be guaranteed in the first and final stages as the hardware used to take images is not compatible with the watering process at the inoculation stage and production harvesting at the final stage. Moreover, during the inoculation stage, the mycelium development appears inside the compost, and visual analysis of the compost surface is not informative. Thus, in the proposed approach, the modifications of the climate parameters are suggested during the incubation, shock, and fruiting stages. By taking into account that different actions are performed based on the production stage, the different models are used to make suggestions in each stage. Finally, the implementation of the automatic climate management consists of the following steps:

1.

decide if the modification of climate values is needed at the current moment;

2.

choose values for climate parameters if so.

Decision making is applied based on a dataset constructed of historical data with changes made by technologists, known visual data, and climate parameter values. To evaluate the dynamics of the parameters used in the decision tree model, the difference between the values in 4-, 12-, and 24-h intervals are considered as input features. Finally, the decision tree is used to define whether the climate change is needed at the current time in combination with K-nearest neighbors employed to obtain what parameter values should be used after change. The values of ongoing cultivation were compared with historical cultivations and averaged values of the 3 most similar ones with time offset considered were suggested as recommendations.

Although in the Results section the data collection, trends of visual indicators, and performance of decision model is analyzed, the automatic climate management mushroom growing hall must be implemented with the following:

1.

A notification system about data collection disturbance so that technologist would take over the climate management in case the appropriate data for making automatic decision cannot be guaranteed.

2.

A model update system to review the data collected during the cultivation after it is finished and refitting the decision tree model with available new data.

The generalized scheme of the automatic climate management process is provided in Figure 9.

3. Results

3.1. Technologist’s Interface and Overview of Collected Data

During the data collection stage, the growing process from the technologist perspective continues as usual while a created layer collects visual data, climate data, and information about the actions performed by the technologist during the growing process. After the growing process, the technologist reviews the growing process and evaluates the applied decisions based on the historical data using a created WEB-based interface. Several user interface screens are provided in Figure 10 to demonstrate how the technologist can follow the growth process and perform action evaluation by analyzing the full growth process information. Here, the technologist can access all the information before deciding about action assessment for the selected cultivation for each stage (Figure 10a, part 1), see all changes (Figure 10a, part 2), and (Figure 10a, part 3) provide an assessment of the change in the evaluation form. To help the technologist make a decision, photos at the different positions closest to the time of the change are provided in the evaluation form. They can also access a detailed analysis of each photo (Figure 10a, part 4) or check the dynamics of the climate or visual parameter during the growth process as a time series graph. An example of the dynamics of mycelium coverage percent in a range of 3–12 days is shown in Figure 10b. The change of the total number of pins normalized per m² for 12–16 days is provided in Figure 10c. It also demonstrates the disturbance in data collection on the 15th day as there were no photos collected. Aside from the screens provided in Figure 10, the technologist can use the interface to access other information related to the growth process, such as administrative data about the substrate; watering process information; graphs of the visual indicators’ changes; and the setting and monitoring of values of different climate indicators such as compost/room temperatures, CO₂, and other. The values of the observed parameters were collected in 10-min intervals without the disturbance for all cultivations. It makes the review of the actions more convenient for the technologist and lets them evaluate the action to change the climate as accurately as possible.

Finally, in order to create the initial decision-making model, data of 16 historic cultivations were reviewed by the technologist and used as the training dataset. The production process was carried out in two cultivation halls in the north of Lithuania. The amounts of collected valid photos based on the cultivation number and growing day are provided in Figure 11. All selected cultivations had more than 1500 photos each to ensure the diversity of locations during the training process. The number of images taken per growth period (Figure 11a) can vary due to the different time the stages lasted in different cultivations, the interference of the image-taking process because of technical issues, and other reasons. The variance of stage duration results in a small number of images to represent the 3rd, 4th, and 17th days of cultivation (Figure 11b), but there are more than 1500 photos to represent the 5th–16th days (that is, for the period where changes are suggested).

In the mushroom growing process, there is a time range when certain actions are assumed to be suitable to perform. During the process of evaluating the changes, the technologist defines the time range. Based on this data, the final dataset was created by iterating all growing processes per hour period. Only the climate changes confirmed by the experts and having the corresponding visual information were used to construct decision tree models. It is assumed that visual data is valid if the gap between two sequential images was not longer than 2 h in a 24-h period up to the moment in consideration. Such an approach allows the simple elimination of growth periods in which collection disturbances occurred (i.e., Figure 10c, gap of visual information on 15th day) or actions marked as not correct by the technologist. A summary of the collected data is provided in

Table 3. Distribution of the prepared dataset instances by growth stage and stage actions.

Stage	Action	Valid Data Instances
Incubation	No changes	381
	Correction of parameters	77
	Start shock stage	480
Shock	No changes	1410
	Correction of parameters	75
	Start fruiting stage	359
Fruiting	No changes	1274
	Stimulate pinning	108
	Slow down pinning	48
	Other change	17

(here “do nothing” is also an action).

The detailed analysis of the managed climate parameters for the decision to stimulate pinning in the fruiting stage is provided in

Table 4. Changes of climate parameters for the action to stimulate pinning.

Parameter	Climate Parameter Change/Total Instances
CO2 set point	0 (no change): 19 instances; −200 ppm: 4 instances; −300 ppm: 7 instances; −500 ppm: 45 instances; −700 ppm: 5 instances; −1000 ppm: 13 instances; −1500 ppm: 15 instances;
room RH set point	0 (no change): 73 instances; −1%: 30 instances; −2%: 5 instances;
room temp set point	0 (no change): 38 instances; −0.5 °C: 47 instances; −1 °C: 18 instances; −1.5 °C: 5 instances;

. This action was analyzed because it had the highest number of instances (108). The analysis shows that in a majority of the cases, the action of pinning stimulation resulted in reducing the CO₂ set point by 500 and room temperature set point by 0.5. In all cases, the managed parameters (CO₂ set point, room relative humidity (RH) set point, and room temperature set point) remained the same or were reduced.

3.2. Visual Information

The features extracted from the visual data using different technologies are important indicators to define the dynamics of the growing process. The results of extracting features from the visual data using morphological analysis, Fourier analysis, and convolutional neural networks for object detection are provided in this section.

3.2.1. Morphological Analysis

The results of the morphological analysis are provided in Figure 12 for each monitored area. The percentage of mycelium coverage increases significantly in 4–6 days. During the same period, a decrease in the number of filtered components is observed; thus, larger segments are formed. On days 8–12, these values change slightly. Subsequently, a large part of the image is covered in mushrooms, which form large, combined segments (increasing the overall coverage percentage and decreasing the number of components).

Other results of the morphological analysis are provided in the Appendix A.

3.2.2. Fourier Analysis

The results of the Fourier analysis are provided in Figure 13 for each monitored area. The largest change is observed in the medium frequency filter images on days 4–6 of the growth (Figure 13b).

Other results of Fourier analysis are provided in Appendix A.

3.2.3. Object Detection

The results of the Fourier analysis are provided in Figure 14 for each monitored area. The presented figures show that the number of objects with a diameter of 5–10 mm changes steadily), reaches its highest value on days 14–15, and begins to decrease. During that period, the number of objects with a diameter of 11–20 mm starts to increase.

Other results of object detection analysis are provided in Appendix A.

3.3. Model Development

On the one hand, the number of valid photos and number of data instances provided in Figure 11 and Table 3 are quite solid (more than 20K valid photos can be used for the analysis). On the other hand, there were only 16 distinct cultivations that can be used in the analysis. This research focused on demonstrating that the AI model based on visual criteria can be used to make reasonable decisions, as it can analyze the performance of the model as the amount of training data increases (see bottom part of generalized proposed algorithm provided in Figure 9) and the importance of the visual indicators for making decisions in each stage. Due to the limited data, the tuning of the hyperparameters was out of the research scope. To create the decision-making model, only features extracted from the visual data were used. Obviously, sensor-based parameters such as the compost temperature or time elapsed since the start of cultivation can improve the accuracy of the model. To ensure a proper amount of training samples of each class, a threshold of 100 data samples for the action was set for the class to be included with the training data. Thus, based on Table 3, there were three decision making models for each stage: incubation (no action, start shock stage), shock (no action, start fruiting stage), and fruiting stage (no changes, stimulate pinning).

Because of the small amount of data, a bootstrapping sampling technique has been selected to evaluate the performance of the decision-making model. In each experiment, data from 5 random cultivations was selected as the validation set. To analyze the trend of the model performance as the amount of training data increases, 5 to 11 cultivations were used for training. The model was implemented in the Python programming language using a Random Forest classifier from the scikit-learn library. To avoid model overfitting, the max depth of the trees was limited to 3. The loss function with balanced class weights was employed to deal with disbalance of class instances. Thus, 50 experiments of creating a decision-making model with each number of cultivations selected as training data were performed. The results of the analysis are presented in Figure 15 (that is, weighted accuracy for the models and true positive (TP) rates of classes in incubation (Figure 15a), shock (Figure 15b), and fruiting (Figure 15c) stages). The averaged importance of the variables if the data of 11 cultivations is used for training is provided in Figure 15d–f for incubation, shock, and fruiting stages, respectively. The Fourier mean low/medium/high corresponds to the mean intensity value after applying low/medium/high frequency filter, respectively. Similarly, the Fourier std corresponds to the standard deviation of the intensity value after filtering. To monitor the dynamics of mushroom growth, the difference between the number of pins of various sizes is calculated in the 4-, 12-, and 24-h period. For example, the parameter “pins 11 20 24” means a 24-h difference in the number of pins that had a diameter from 11 mm to 20 mm. The analysis results show that as the amount of training data increases, the weighted accuracies of the models increase as the models better predict the class other than taking no action. Simultaneously, the decision to take no action becomes less accurate for the models in shock and fruiting stages, whereas no significant difference was noticed for the true positive value of taking no action class in the model of the incubation stage. The analysis of parameter importance shows that features extracted using morphological and Fourier analysis have high importance in the incubation stage since the number of pins is usually close to zero in the beginning of the growth process. In the model of the shock stage, features extracted using the Fourier analysis and object detection approach have high importance. In the fruiting stage, the most important features are related to the number of pins of various diameters.

The ANOVA analysis for the model accuracy with the training data of 5 and 11 cultivations for the incubation, shock, and fruiting stages are provided in

Table 5. ANOVA analysis of the model accuracy for models trained with 5 and 11 cultivation data for the incubation stage.

Groups	Count			Sum		Average			Variance
5	50			42.52598		0.85052			0.002383
11	50			43.46301		0.86926			0.002113
Source of variation		Sum of squares	Degrees of freedom		Mean squares		F	p-value		F-crit
Between Groups		0.008780279	1		0.008780279		3.905938712	0.050926745		3.938111078
Within Groups		0.220297187	98		0.00224793

,

Table 6. ANOVA analysis of the model accuracy for models trained with 5 and 11 cultivation data for the shock stage.

Groups	Count			Sum		Average			Variance
5	50			37.45403134		0.749080627			0.012167794
11	50			40.68343016		0.813668603			0.005823261
Source of variation		Sum of squares	Degrees of freedom		Mean squares		F	p-value		F-crit
Between Groups		0.104290167	1		0.104290167		11.59355723	0.000960477		3.938111078
Within Groups		0.881561729	98		0.008995528

, and

Table 7. ANOVA analysis of the model accuracy for models trained with 5 and 11 cultivation data for the fruiting stage.

Groups	Count			Sum		Average			Variance
5	50			27.47229322		0.549445864			0.005562777
11	50			32.68785236		0.653757047			0.005669442
Source of variation		Sum of squares	Degrees of freedom		Mean squares		F	p-value		F-crit
Between Groups		0.272020571	1		0.272020571		48.4357684	3.89436 × 10−10		3.938111078
Within Groups		0.550378715	98		0.005616109

, respectively. All statistics were calculated with $\alpha =0.05$. The p-value for the incubation stage shows that the hypothesis about the equal means of the groups cannot be rejected at the significance level of 0.05. However, for the other stages, the p values are significantly lower than the significance level. The calculated least significant difference (LSD) for the shock and fruiting stages are 0.037643 and 0.029743. It is obvious that the mean difference of the groups exceeds the LSD with a difference of 3.2294 and 5.21556 for the incubation and fruiting stages, respectively.

Although the set of samples used for training and testing is small, the models adequately define the situations that make up the largest part of the training set: when the current climate does not need to be changed and when it transitions to the next stage.

Finally, if the predicted action requires modification of the climate parameters, the changes in the values of the climate parameters are obtained using the method of K-nearest neighbors (KNN). Thus, for each such action, a separate dataset of vectors with standardized and normalized features has been prepared. As in the decision-making model, only visual indicators and their changes in time were investigated as features. A decision is made after calculating the Euclidian distance of the existing data instances and selecting K closest ones. The number of neighbors K used in the model was equal to 3 due to the limited number of samples. Then, for each managed parameter, the averaged change value is calculated. In this case, the change of the parameter is defined as the difference of value instead of the absolute value of the parameter. The action to stimulate pinning in the growing stage was selected as an investigation object because it had the largest number of samples (108 in total) for an action, which requires change in the climate parameters. The decision to stimulate pinning can result in changes in the CO₂ set point, room RH set point, and room temperature set point. In the dataset, changes for this action vary from 0 to −1500 ppm for the CO₂ set point, from 0 to −2% for the room RH set point, and from 0 to −1.5 °C for the room temperature set point (see Table 4).

The same bootstrapping sampling methodology was used to separate data instances by cultivation and, therefore, to construct training datasets for the analysis of the performance of the KNN method as the number of training samples increases. The results were demonstrated for the changes of CO₂ because of the largest number of distinct values of change (see Table 4, there are 7 distinct values for the changes for CO₂, 3 for the RH set point, and 4 for the room temperature set point). The boxplots of experimental mean square and absolute errors of CO₂ set point values with a different number of cultivations used for training are provided in Figure 16. The results show that the performance of the model improves as the larger number of cultivations are used for the training.

In conclusion, although the set of samples used for training and testing is rather small, the models adequately define the situations that make up the largest part of the training set: when the current climate does not need to be changed and when it transitions to the next stage. The suggested AI-based climate management process demonstrates promising results. Increasing the number of instances used for training would result in a model with higher accuracy. In addition, including additional variables could improve the decisions of the models. For example, in the current models, the time series of parameters extracted from the visual data are analyzed in the periods of 4, 12, and 24 h. More frequent points of time series better represent the dynamics of the mushroom growing process and lead to more accurate decisions. At the current moment, the AI-system is used in the mushroom growing process under the supervision of the technologists, and its components are modified according to the experts’ feedback.

4. Discussion

In this study, the actions performed by the technologists were used as examples of good practice and, thus, used to generate a training dataset. In general, the climate changes in the specific environment depend on the physical properties of each mushroom growing hall. Thus, different values of the climate parameters can be assigned by the technologist to get similar results in different environments. Moreover, in some cases the actual values of the parameter cannot meet the prescribed ones due to the physical restrictions. The suggested approach to use monitored and evaluated technologists’ actions as a training set and later make decisions on the action level enables the reuse of the methodology in different halls where the climate management procedure may differ and ultimate optimal climate management cannot be guaranteed due to physical restrictions.

In addition, the climate management procedure depends on the objective of the cultivation. For example, it is important to adjust the cultivation process to human resources available for harvesting, or the objective is to have the production of the specific size prepared for the agreed date. If the final objective differs, model tuning should be performed to obtain suitable decision-making models, and the model for predefined objective should be used in the cultivation process.

On the one hand, implementation of such a layer and collection of fully structured data is expensive due to the preparation of the physical infrastructure for the acquisition of synchronized visual and climate data, human resources, and the time necessary to review collected data. On the other hand, besides implementing automatic climate control, such an approach simplifies knowledge sharing between the technologists working in the same growing halls or using collected data for the new technologist as learning materials.

The suggested approach is based on the manual review of the technologists’ actions. The decisions made by the technologists are subjective and depend on the technologists’ experience. However, it is always possible that the decision was incorrect. In general, if the training dataset is large enough, incorrect decisions may be detected using statistical analysis in relation to the outcomes of the cultivation. As the dataset in this research was limited, the identification of unsuitable changes and automatic determination of the period in which the action is appropriate need further investigation.

The main objective of this approach was to show that monitoring visual information may be useful in the decision making process. Of course, the proposed solution is still in the stage of proof of concept, and there are many options to improve the automatic decision-making process, such as the use of more visual indicators; the inclusion of values measured using sensors, such as compost temperature and room temperature; the use of changes of the indicators as timeseries instead of several selected time points; and similar actions. However, such improvements require more structured data and are the objects for further research.

5. Conclusions

The additional AI-based layer for the existing climate management system in the white button mushroom growing hall was presented in this article. The layer combines technologists’ experience, climate management data, and indicators extracted from visual information. The experts’ knowledge and shortage of training data for the complete cultivations led to three different AI models, which represent different stages of mushroom cultivation (that is incubation, shock, and fruiting). Due to variety of data formats, the AI-based layer employs various AI methods. Moreover, the application depends on the cultivation stage. For example, in the early stages, the features extracted conventional signal processing methods; a Fourier transformation and morphological analysis were identified as most important. In the later stages, object detection using convolutional neural networks had a stronger influence on the model results. The AI-based layer provides recommendations on what changes should be made for the climate parameters. The state of required climate change was detected using a decision tree, and the recommended parameters were chosen using a method of K-nearest neighbors.

It should be noted that the dimensions and constructional material of the hall can have an impact on what climate is considered optimal for mushroom cultivation. Thus, the models should be fitted according to the mushroom growing hall properties if they change significantly.

The standard climate management in the white mushroom growing process depends on the technologist’s level of expertise and skills to evaluate various parameters related to the mushroom growing state. The suggested AI-based layer enables experts’ knowledge sharing and helps to move the mushroom growing process towards the unified climate management rules.