High-Throughput Plant Phenotyping System Using a Low-Cost Camera Network for Plant Factory

Abstract

Plant phenotyping has been widely studied as an effective and powerful tool for analyzing crop status and growth. However, the traditional phenotyping (i.e., manual) is time-consuming and laborious, and the various types of growing structures and limited room for systems hinder phenotyping on a large and high-throughput scale. In this study, a low-cost high-throughput phenotyping system that can be flexibly applied to diverse structures of growing beds with reliable spatial–temporal continuities was developed. The phenotyping system was composed of a low-cost phenotype sensor network with an integrated Raspberry Pi board and camera module. With the distributed camera sensors, the system can provide crop imagery information over the entire growing bed in real time. Furthermore, the modularized image-processing architecture supports the investigation of several phenotypic indices. The feasibility of the system was evaluated for Batavia lettuce grown under different light periods in a container-type plant factory. For the growing lettuces under different light periods, crop characteristics such as fresh weight, leaf length, leaf width, and leaf number were manually measured and compared with the phenotypic indices from the system. From the results, the system showed varying phenotypic features of lettuce for the entire growing period. In addition, the varied growth curves according to the different positions and light conditions confirmed that the developed system has potential to achieve many plant phenotypic scenarios at low cost and with spatial versatility. As such, it serves as a valuable development tool for researchers and cultivators interested in phenotyping.

1. Introduction

Stressful situations cause morphological, physiological, and biochemical changes in crops. Thus, research on crop phenomics has been conducted to measure and utilize these changes in order to increase productivity, improve quality, and reduce energy and resource consumption through optimal control [1]. In the early stage, a specific stimulus was provided in a controlled environment, such as a smart farm, and response analysis and growth modeling were performed. In particular, to obtain characteristic information about crops, direct measurement, observation, or destructive methods are used. However, although these methods are generally simple and reliable, they require considerable time and labor because of the small number of activities that can be performed at one time [2,3]. In addition, when destructive methods are applied, the sample is damaged, and subsequent analysis is difficult.

To overcome these difficulties, various analyses using non-destructive methods have been conducted. Data available for analysis include images, environmental (temperature, humidity, light intensity, and nutrients) values, and physical (length and weight) values to quantify the genotype and phenotype [4]. Among them, image-based methods are used to analyze the external characteristics or growth of crops using two- (2D) or three-dimensional (3D) images or to indirectly explain internal phenomena through spectral images [5]. As several samples can be easily and automatically obtained using a camera, the time and cost required to acquire feature information can be reduced [6]. In addition, a large amount of information can be obtained using several images, broadening the scope of the analysis. This development coincides with the creation of integrated and mass crop production facilities; moreover, with the development of information communication technology (ICT) convergence technology, high-throughput analysis has been developed into a high-speed, mass-phenotypic method with high efficiency [7]. A high-throughput phenotyping platform refers to a system that integrates the storage of crop data, data analysis, robotics, and decision making. Growing demand, along with the development of numerous types of applicable data and technologies, has led to the requirement of several research studies for this platform and improvement of standardization, experimental methods, and analysis [8,9,10].

The process of stably collecting and analyzing images is important for analyzing the overall growth of crops through images. The process of acquiring the image should place no additional stress on the crop, and there should be no other influences such as noise or color distribution changes on the image. Several cameras can be utilized to acquire images. In addition, automatically moving and saving the images according to the conditions in a specific storage space can be advantageous because shooting is conducted over several days depending on the growth process. A system with continuity in which the characteristics of crops are automatically analyzed using various image processing and machine learning algorithms for each image and where model development and updates are performed is considered ideal. Thus, the entire process from image acquisition to analysis must be organically connected to enable high-speed mass-phenotypic analysis. However, most of the previously developed systems are very expensive and use algorithms applicable only to a specific environment, hindering their broad application. For example, in a previous study, when examining the number and area of leaves in the growth chamber to investigate the development of Arabidopsis, there were differences in the absence of various application; in addition, the study was soil-based [11]. Similarly to this study, in a 2D-based phenotyping system using a fixed camera, automatic water supply adjustment and area and color characteristics were analyzed. This study was conducted in a soil environment and applied to relatively small crops [12]. In some studies, 3D images and spectroscopy were applied by performing growth monitoring through individual moving ports. However, a limited number of parameters were used, and there were difficulties in system configuration using built-in software [13,14].

A plant factory is a mass crop production facility that can provide a more uniform growth environment than the open field through artificial lighting and complex environmental control in an enclosed space for the year-round production of crops [15]. However, even within these plant factories, spatial variations exist in micro-environmental conditions, such as light, temperature, and humidity, resulting in growth differences [3,16]. Moreover, it is necessary to capture and analyze at minimum hundreds of plant images per day to follow the rapid and intensive progression of growth in plant factories [4,17]. If the analysis is not accomplished in a real-time manner, the data will be accumulated and delay the response to the crop status. In the context, high-speed mass phenotyping approach can be a practical solution to provide location-specific information on crops growing in the cultivation bed, thereby allowing precise environmental control and growth management in plant factories such as yield estimation, crop harvest, or disease treatment.

Methods for acquiring image information according to location include acquiring multiple images while moving one camera and acquiring multiple data at a time by arranging multiple cameras at a certain location. When using a single mobile camera system, it is possible to use a high-cost camera with good performance rather than purchasing several cameras. Good performance provides high resolution and facilitates obtaining images of depth sensors or other wavelength bands rather than simple colors. Because the image is acquired with one device, there is no need to consider the error of the hardware itself. However, this method is relatively expensive and has limitations in repeatability, data acquisition time, and movement section restrictions because the camera position must be moved manually unless an additional automation device is employed in a situation where multiple crops need to be photographed. For example, Jiang et al. [18] developed an on-the-go monitoring platform to measure crop-growth variation according to the location; however, this system could only be moved for the first applied bed structure, and the configuration of the system became more complicated as the cultivation area increased. Yeh et al. [19] created a moving-image monitoring system using a moving arm. Its effectiveness was verified only on a small vertical bed, and rapid analysis was difficult because the growth information was confirmed via image processing through post-processing. Rossi et al. [20] reported a low-cost sliding platform for high-resolution plant phenotyping, but the processing time of more than 1 h would limit the application of the system to the level of laboratory analysis. A greenhouse-based low-cost phenotyping platform reported by Yassue et al. [21] is noteworthy since it proved its effectiveness in a greenhouse-level environment. However, it would not be applicable for vertical plant factories due to height problems. Bari et al. [22] applied a moving-cart embedded camera to the rails for monitoring growing peas in greenhouse, but the large area of the greenhouse meant that it was necessary to analyze the image via post-processing after the collection of images.

By contrast, as the method using several inexpensive cameras can secure a large amount of simultaneous image data, it is easy to analyze changes in the same location without additional correction processes. In addition, it can be flexibly configured for plant factories with relatively large and diverse production structures. However, there is a non-uniformity of information depending on the type and quality of images and the use of various hardware; therefore, it is necessary to develop an image processing technique to overcome this problem. Although An et al. [23] adopted the calibration of multiple-camera array and utilized it for plant phenotyping, the post-data collection and -processing made it difficult to access and analyze data in real time. Thrash et al. [24] also proposed a low-cost wireless phenotyping system, but it was only developed for a single index and a horizontal space. Gang et al. [25] developed a novel convolutional neural network-based model that can estimate various morphologic indices for plants grown in greenhouses, but its feasibility was not validated in an integrated phenotyping platform.

Considering the aforementioned limitations, it is necessary to develop a low-cost phenotyping system capable of collecting and processing data in real time while being structurally more flexible for crop monitoring in vertical plant factories. Furthermore, for phenotyping modeling aimed at analyzing the behaviors of various crops, it is necessary to simplify the determination of new indices or code modifications for correlation analysis.

For the purpose, in this study, a multi-camera network using an inexpensive image sensor and processor was developed. It was designed to have flexible space applicability through a wireless, miniaturized camera node configuration, and to make it easy to collect and analyze various phenotypic indices through a modularized image processing algorithm structure. Based on this, a system capable of performing high-speed mass-phenotypic analysis was constructed. Specifically, the developed system continuously and stably collected crop growth images from camera devices installed at different locations, classified them according to location and time, and stored them in an independent space of the computer server. The stored images were configured to analyze diverse crop characteristics through various image processing algorithms and to output the results. Additionally, the analyzed crop characteristics were used as data for a growth model for a specific situation, and the model was updated according to the data generation. The feasibility of the developed system was evaluated by collecting 2D images of lettuces for the entire growth cycle at various locations. To verify the applicability of the obtained phenotypic indices, growth information of lettuces growing under two different photo-environmental conditions within the plant factory were analyzed and compared.

2. Materials and Methods

2.1. Plant Growth Conditions

The camera system could be applied to any leafy vegetables with low heights, which are usually cultivated in the commercial plant factories [26]. In this study, Batavia lettuce (Lactuca sativa L.) was cultivated for the growth experiment, which is a type of lettuce that originated in Europe. Lettuce is one of the most important leafy vegetables in plant factories because of its popularity, with the highest consumption and economic importance throughout the world [27,28]. Individuals with similar weight, size, and shape were selected by direct sowing, and a monitoring experiment was conducted on the growth of the entire cycle from the second day after planting.

Cultivation was conducted in a container environment system located in Yongin, Gyeonggi-do, Republic of Korea (Figure 1). This container, a system that has an environment independent of the outside environment, is suitable for conducting reaction experiments of crops to a target factor. Crops were grown independently in a sponge medium serving as artificial soil, and the necessary nutrients were provided through a closed nutrient film technique system where the electrical conductivity and pH values without a separate nutrient solution supply device. Generally, EC and pH values below or above the optimal range can impede water and nutrient uptake in crops, thereby inhibiting growth and potentially inducing diseases [29]. Therefore, the nutrient solution was manually managed to maintain an appropriate concentration by measuring the EC and pH values at 2-day intervals through standard sampling. During cultivation, the temperature inside the container plant factory was maintained at an average of 22.1 °C in the range of 21–23 °C, and the humidity was maintained at an average of 65% in the range of 50% to 80%. CO₂ was maintained at 400–700 ppm as a factor relating to plant respiration.

2.2. Photoperiod Condition

Photoperiod is one of the most important variables affecting the plants’ growth process, thereby inspiring researchers or industries to explore the effective light/dark cycle for leafy vegetable productivity [30,31]. Specifically, it was reported that differing pulse width modulation of light would be an important factor in the photosynthesis. Chen and Yang [32] reported that intermittent light irradiation could enhance the biomass and taste of lettuces. Therefore, not only to evaluate the applicability of the developed system but also to confirm the effect of different light/dark cycles on the growth of lettuces, an experiment was conducted to establish a camera network for two cultivation beds and to check the growth difference by varying the photoperiod conditions for each zone. The control (con) group was irradiated with light for 16 h and maintained in the dark for 8 h; for the pulse (pul) group, light–dark conditions were set at intervals of 2 h and 1 h, respectively. Accordingly, the daily light irradiation time of the two groups was 16 h, and the total irradiation time and the total incident light amount were the same under the two photoperiod conditions. However, it was expected that improved growth would appear for pul group due to the different light/dark cycles. A total of 14 Batavia lettuces were cultivated for each group, and 4 lettuces were used for acquiring destructive test data (Figure 2). Specifically, 12 camera systems were installed for each control and experimental group to acquire images, while each camera was installed to be located in the center of each target crop. With regard to the extra two cameras for each group, they were used to validate the phenotypic analysis during the cultivation. Growth characteristics such as fresh weights of shoots, leaf length, leaf width, and leaf numbers were measured for four samples every week. Fresh weights of shoots of lettuces were measured using a digital scale (Si-234, Denver Instrument, Denver, CO, USA), and the length and the width of leaves were measured using a ruler. The measured values were used to analyze the differences in data and to compare means among the treatments, respectively.

2.3. Low-Cost Camera Network

For high-speed, mass-phenotypic analysis targeting plant factories, the narrow cultivation interval between crops and the restriction of height should be considered. Accordingly, the proposed system was configured in the form of a multi-point camera network that could freely attach and detach small camera modules and provide integrated access to information on each image collection node through a web connection. Moreover, the building cost of the camera network system is about 5% level compared to the on-the-go monitoring system by Jiang, Kim, and Cho [18] (

Table 1. Comparison of the camera network system with the on-the-go monitoring system by Jiang, Kim, and Cho [18].

Item	Camera Network System	On-the-Go System
Cost	~USD 50 per node	~USD 2200
Setup	Light and compact (~50 g L96 mm $\times$ W71 mm $\times$ D26 mm per node)	Bulky (~8 kg for conveyer belts, controllers, motors, and actuators)
Scan area (height of 0.45 m)	0.54 m $\times$ 0.4 m	0.94 m $\times$ 1.8 m
Plant scan time	<1 ms	~127 s
Ease of installation	Easy (due to the node’s compact size and individual modularity)	Difficult (due to the complexity of conveyer belts and frames)
External equipment	Wi-Fi router	Linear actuators, a motor servo driver, and step motors

). Although the cost will be increased as the number of camera nodes increases, it is still more reasonable than the on-the-go monitoring system when considering the scan area.

A Raspberry Pi 3B+ (Raspberry Pi Foundation, Cambridge, UK) was used as a low-cost microprocessor that can acquire images of crops for the experiment and transmit them to the server. It was selected because it is small, light, and inexpensive, has good compatibility with various equipment, and has good accessibility compared to other commercial products.

Various cameras, including a USB webcam, can be connected to the Raspberry Pi to take pictures. The Raspberry Pi Camera v2 (Raspberry Pi Foundation, UK) was selected and used to compare the stability, convenience, and image quality. This camera is composed of a module with a resolution of 3280 × 2464 and a performance of approximately eight megapixels. For the stability of data collection, the board and camera modules were configured in a one-to-one connection. The Raspberry Pi and camera module used in this study are not essential parts and can be replaced with other products depending on the experimental environment conditions.

Because the monitoring of crops is conducted remotely, the information obtained on the local Raspberry Pi has to be grasped remotely. Moreover, as mentioned above, an external system is needed to solve the problems associated with the use of multiple processors and limited local capacity. To this end, a computer acting as a server and multiple Raspberry Pi devices were connected to form a system to automatically import data. The computer acting as a server played an overall role in storing and analyzing data. The computer was configured with a Linux operating system environment of core i7-7700k CPU @4.2 GHz (Intel Corp., Santa Clara, CA, USA) and 32 GB RAM for continuity with Raspberry Pi.

Figure 3 shows the network diagram for system setup. This network diagram is an example for the photoperiod experiment of this study, and it is possible to add or remove a server, router, Raspberry Pi, etc., depending on the researcher’s experimental environment. Several Raspberry Pi devices for shooting were connected to a single local router via Wi-Fi. A port forwarding method replaces the internal IP between the local router and the Raspberry Pi connected to the lower internal network and creates a new address that can be accessed remotely. Therefore, it was possible to remotely access the Raspberry Pi containing the image information for each location and transmit and receive real-time monitoring and image information. The external server was connected to several Raspberry Pi devices through the local router and used the secure shell (SSH) protocol to access and issue remote commands. Thus, the SSH class was implemented, and the image data were transmitted to the server at a specific period after creating a session using the port forwarding access address. In the server, there were folders designated for each Raspberry Pi, and images were saved in each folder with a name set using date and time information.

2.4. Crop Image Collection

As described in the following passages, image data taken according to time conditions were saved in the Raspberry Pi and configured to be transmitted to the server at a specific point in time to enable continuous image shooting. When an image acquisition signal was transmitted to the Raspberry Pi according to the time condition (20 min intervals), a process for stabilizing the image sensor was performed, and the image shooting was performed after acquiring the local time required for phenotypic analysis. To stably acquire image data, the data transmission operation was interrupted during the acquisition process. Since it did not take much time to transmit one image, the data were transmitted when a certain number of images were accumulated in consideration of the system load in the transmission process. Accordingly, data collection to the server was performed once daily. The code for image acquisition was written using Python, and camera control was performed with a Python library called “PiCamera.” The use of crontab provided a reduction in the system load, as the system was run only when necessary, rather than continuously.

As the images acquired have high resolution, one file has a size of approximately 4 MB. This is not a problem in short-term monitoring, but in the case of long-term experiments, there may be a problem of insufficient capacity inside the processor. Accordingly, flags for the number of consecutive shooting days, or the number of stored files can be configured to enable continuous monitoring by deleting previous files when the limit is exceeded. In addition, in the process of transmitting the image to the server, it was implemented to record and display logs such as connection errors, image errors, and transmission completion of the local Raspberry Pi to promptly respond to users.

The imaging system consisted of a top-view image acquisition system that could secure crop-growth images in container environmental conditions. The system was configured so that only one crop was placed in the center of one image. Camera modules were installed in the space between the LED lamps to obtain an accurate top-view image and to prevent light intensity interference. Each camera module was calibrated using a classic color checker (X-Rite, Grand Rapids, MI, USA) to compensate the spatial variability. Image acquisition was performed for additional 10 s to stabilize the shooting. Figure 4 shows the multi-camera network installed for multi-point growth image monitoring.

2.5. Phenotypic Index Processing Algorithms

An automatic extraction process of the phenotypic index required for analysis was performed on crop images stored in each folder of the server. Crop images acquired by our system consisting of a camera network contain various noises depending on the environmental conditions of the plant factory. Depending on the system configuration conditions, an image contains only one crop, but the image also includes several types of surrounding environments, such as planting bed, media, and other modules, as noise. The image processing steps to counter these issues are shown in Figure 5. Several algorithms were combined to extract the region of interest (ROI) corresponding to the crop parts required for analysis. ROI are initially extracted from the acquired images using the vegetation index and Otsu threshold method. Because small errors appear in these results, the final binary ROI was extracted through the intersection of colorimetric changes and small space filling methods. For the extracted ROI, the contour and pixel area were obtained, the center was found, the shortest and longest lengths were measured, and the average color information was identified. The extracted index formed a data frame with the structure of the crop, date, time, index 1, index 2,…, index n, which was stored in a comma-separated values (csv) format. It was also possible to check the image for specific results. The image processing was configured to be performed on the entire folder when a certain number of image data were added, including files that were not analyzed. The default value was set at daily intervals; thus, when the daily data transmission was completed, index extraction was performed only on new data. The newly created indicator data were stored cumulatively in the previously saved csv file to enable continuous analysis. For the storage and management of data, it is possible to consider a method of using a database rather than a simple csv format storage. In this study, a database was not considered because a simple configuration was sufficient. However, if the system grows in size, the use of a database would be necessary. OpenCV and PlantCV libraries were mainly used in the image processing.

For accurate analysis of crop image phenotypic indices, some aspects were considered on the images obtained. First, image distortion was considered [33]. Usually, in the case of an image containing several crops, errors occur in the center and edges of the image owing to the hardware characteristics of the camera lens. This phenomenon is called distortion. In the case of the configured system, only one crop was placed in the center of one camera to avoid distortion. Second, even by precisely positioning the camera and crop, physical skew occurs [34]. This can cause problems in image tracking because movement may occur as the crop grows; in addition, it depends on the characteristics of hydroponics. In this regard, an analysis method not affected by skew was used. Specifically, two methods were applied to avoid the effect of skew: first, one camera was placed for each crop so that the crop was located in the center of the image, thereby minimizing the effects of skew, which typically occurs in the corners of the image. Second, the projected area viewed from the top instead of the actual leaf area to extract phenotyping features because it could reduce skew interference caused by leaves growing in various directions. Lastly, because multiple cameras were used, the colors appeared slightly different for each camera, even when shooting the same subject. This was solved by performing a calibration process using a separate color panel as a hardware characteristic.

The first step of the analysis of crops is the extraction of a ROI through background separation. If ROI extraction is not performed adequately, values calculated in the subsequent process become useless. Accordingly, several algorithms were combined and utilized. The target part was a crop, and the image to be analyzed was in the form of 2D red–green–blue (RGB). Thus, Excess Green (ExG), an image extraction method [35], was applied among the available vegetation indices [36]. As shown in Equation (1), R, G, and B color channels were used, normalized values divided by the maximum values ($R_m$, $G_m$, $B_m$) of each channel were employed, and the equation was derived by assigning weights to the green part.

$$\begin{gathered} R^{\mathrm{*}}=\frac{R}{R_m}, G^{\mathrm{*}}=\frac{G}{G_m}, B^{\mathrm{*}}=\frac{B}{B_m} \\ r=\frac{R^{\mathrm{*}}}{R^{\mathrm{*}}+G^{\mathrm{*}}+B^{\mathrm{*}}}, g=\frac{G^{\mathrm{*}}}{R^{\mathrm{*}}+G^{\mathrm{*}}+B^{\mathrm{*}}}, b=\frac{B^{\mathrm{*}}}{R^{\mathrm{*}}+G^{\mathrm{*}}+B^{\mathrm{*}}} \\ ExG=2g-r-b \end{gathered}$$

(1)

ExG uses only RGB colors and is an index affected by green; therefore, large and small noises could be included, as shown in Figure 6. Although the noise of small particles is also an issue, the artificial medium, which is usually used in plant factories, has a larger role in noises. The artificial medium is artificial soil constructed to enable crops to absorb and support the nutrient solution. In commercial plant factories, an empty artificial medium is sometimes placed to control the planting distance, but it could be recognized as a crop part in the image and included as noise. These noises cannot be distinguished by using ExG, so additional image processing is required.

In this study, the hydroponic system in plant factory was considered as the target field. Therefore, noise filtering for plant factory condition was established to extract the crop parts, excluding noise from the image extracted through the vegetation index. After converting the image masked with ExG to the Lab color system, the green-magenta and blue-yellow channels were extracted, and a threshold value determined based on the trial-and-error method was set to remove the noise. Since the lab color space reflects the uniformity of colors, thereby allowing more robustness for the illumination effect [37,38]. Relating to binarization for crop segmentation, by combining the two methods and performing an additional fill process on a small part, a sophisticated binary image of the ROI was extracted. The final target image was obtained by masking the original image on the binary image (Figure 7).

Although more sophisticated segmentation techniques such as a multilevel thresholding or combination of other color spaces such as hue–saturation–value (HSV), and hue–saturation–intensity (HSI) used for detecting the diseased leaf area could be used, the techniques require more processing time for one image, thereby inducing considerable delay for entire growing bed monitoring [37,38,39]. Therefore, in this study, the proposed image processing algorithm was simply constructed to achieve the real-time analysis of the morphological growth characteristics.

A process was performed to determine a phenotypic index that could be used for analysis with respect to the extracted ROI image. The center, contour, length, central axis, color space distribution, growth distortion, and color histogram were selected by referring to the useful phenotypic indices related to crop morphology in previous studies [19,40,41,42].

Table 2. Phenotypic index description and computation method.

Phenotypic Index	Description	Equation	Equation Number
Center	Obtained by using the moment value to find the coordinates of the center of mass $p_{cm}$ Reference point for radius and growth skew calculation	$m_{ji}={\displaystyle \sum _{x,y}}\left(array\left(x,y\right)·x^j·y^i\right)$ $\overline{x}=\frac{m_{10}}{m_{00}},\hspace{1em}\hspace{1em}\overline{y}=\frac{m_{01}}{m_{00}}$	(2)
Contour	Point data acquired using the boundary of the binary mask image through ROI extraction Used for projected leaf area calculation	$l_{contour}=\sum \left|\stackrel{⃑}{{p_c}^i-{p_c}^{i-1}}\right|$ ${p_c}^i$: i-th point of the contour	(3)
Radius (length)	Minimum and maximum radii (length) from the central index to the contour using Euclidean distance Available for estimation of leaf length and volume index	$l_{\mathit{max} radius}=\mathit{max}\left\{\left|\stackrel{⃑}{p_{cm}-{p_c}^i}\right|\right\}$ $l_{\mathit{min} radius}=\mathit{min}\left\{\left|\stackrel{⃑}{p_{cm}-{p_c}^i}\right|\right\}$ $p_{cm}$: center point	(4)
Central axis	Vertical axis in space with respect to the center of the crop and the axis showing maximum growth Alignment of the obtained crop image; evaluation of the crop growth behavior	${Axis}_{\mathit{max} growth}=\mathit{max}\left\{\left|\stackrel{⃑}{{p_c}^i-{p_c}^j}\right|\right\}$	(5)
Growth skew	Degree of deviation of the center of the crop from the center of both the circumscribed circle and the circumscribed rectangle with respect to the crop Evaluation of the crop growth behavior	$l_{growth skew}=\left|\stackrel{⃑}{p_{cm}-p_{cc}}\right|$	(6)
Color space distribution	Degree of color intensity for the crop image (depth image: spatial map) Available for estimation of crop height, volume index	-	-
Color histogram	Histogram for each color coordinate system of RGB, Lab, and HSV for the masked crop part Color indices-based plant disease or stress evaluation	-	-

shows a description of each indicator and the automatic acquisition algorithm.

All processes such as image acquisition, save, image processing, and phenotypic index extraction were modularized through classes and functions. Modularity facilitated the modification and addition of content. Although some parameters needed to be modified according to the experimental environment conditions, it was possible to easily change them according to the user environment through the modularization of the image processing algorithm. Since it was possible to remove only a specific part and add a sub-list of new functions and classes, it could be flexibly applied according to various experimental purposes and growing environments.

Although these various phenotypic indices can have meaning by themselves, they enable further analysis through correlation with actual crop characteristics or addition of an algorithm. To check this possibility, the manually measured values were used for analysis. In the process of conducting the experiment, crop characteristics such as fresh weight, leaf length, leaf width, and leaf number were manually measured and compared with the extracted phenotypic indices. After identifying the correlation, a trend line was drawn through a regression equation, and the coefficient of determination and root mean square error (RMSE) were obtained to verify the indices.

3. Results

3.1. Evaluation of Camera Network

It was confirmed that the developed system can easily identify and solve various problems that may occur in the process of image acquisition and processing through the log generated during the image transmission process. This detected several errors in the artificially generated data port, capturing data, and in the transmission process. Accordingly, it was possible to respond appropriately to the problems that arose, and the validity of subsequent analysis of the transmitted data could be secured.

The phenotypic index extraction algorithm was evaluated by performing the analysis on nine items, including center, contour, minimum radius, maximum radius, spatial vertical axis, maximum growth axis, color space distribution, growth distortion, and color histogram. As a result of performing automatic index extraction on a total of 35,000 image data, the total analysis time was approximately 2.43 s per image and included index values and image result output. The time required for analysis may vary depending on the number and type of indicators to be analyzed.

Figure 8 shows the analysis results using the phenotypic indices selected by the developed system for one crop. ROI extraction was performed through background separation, and the phenotypic index was automatically extracted according to a previously defined algorithm. The background was adequately removed using the vegetation index, color system conversion, and additional algorithms and by providing visualization information for all indices such as center, contour, and maximum/minimum radius. As a result of applying the original image and the phenotypic index, the entire growth cycle can be analyzed for approximately two weeks with an interval of two days for the image. This can provide a better understanding of the acquired information, such as the growth level of crops, growth status, and product quality. In addition, it is possible to access the saved analysis result and analyze it in the form of a data frame. In the developed system environment, the data frame can access each index value by using a “key” corresponding to the column, and it is possible to separate data for each crop by specifying a “value” condition of the key. Accordingly, it is possible to analyze individual crops to perform intensive analysis on crops of different growth by location.

3.2. Growth Analysis for Lettuces under Different Light Conditions

Figure 9 shows the area index changes for the pul group with a change in the photoperiod and for the “con group” with no change. In the pul group, as the light period during image acquisition and the dark period when acquisition was interrupted were short, the data appeared continuously, whereas the con group showed a step-like graph because the dark period was long. The two groups had the same duration of the entire light period; thus, the total number of images acquired in one day was the same, even though there were differences in the acquisition time. That is, the data of the two groups were compared for each day, not for the point value corresponding to the x-axis value of the graph.

In the growth curve, the growth pattern according to the growth location was different in both groups, indicating that there was spatial variation. Approximately 13 days after transplantation, the maximum area of lettuce was approximately 18% larger than the minimum area in the control group, whereas in the pulse group, it was 38% larger, confirming that the spatial variation was larger in the pulse group.

Among the ten phenotypic indices that can be extracted from our system, the indices that can be analyzed meaningfully in light condition experiments include contour, length, minimum radius, and maximum radius in addition to area. These phenotypic indices were closely analyzed to examine the growth difference according to the light period (Figure 10). Thus, a new data frame was created by calculating the average value and standard deviation of each indicator in the developed system interface, and it was expressed as a graph. In the graph, there was no significant difference in the minimum and maximum radius, but the growth was higher in the control group in both area and contour length.

Figure 10 shows the index change for one crop during the experimental period. As observed, it is possible to monitor the growth status, including the phenotypic indices selected for the crop growth of the entire period, using the developed system. However, based on the indices, additional considerations were identified in the analysis. In the case of the contour, it was possible to find a part showing a difference due to the complexity expressed in the growing process of the crop or the change in the crop shape according to the light/dark period. Accordingly, the maximum length from the center was measured by moving the position according to a specific period rather than a single point according to the passage of time. Although it was shown continuously on the graph, the actual measurement position changed significantly. The minimum length was considerably affected even by slight differences in these morphological characteristics; thus, this index showed the greatest change.

In order to verify the phenotypic index, the analysis was performed using the area index as a representative. Figure 11 shows the results of performing predictions using area index for the four characteristics of manually measured crops. Each regression formula was set to have the highest coefficient of determination. The area index showed the best correlation with fresh weight (R² = 0.9602 and RMSE = 2.23 g) (Figure 11A). In the early stages of small size, it was small with RMSE = 0.83 g, but it was confirmed that the error also increased as the size increased (Figure 11B–D). A comparison with the other three characteristics showed a low coefficient of determination compared to the fresh weight, and no specific characteristics were found.

4. Discussion

Among the phenotypic indices, the area that was the most intuitive index for crop growth was a part of the projected pixel points obtained from the top view. Although the area index would be different from the actual area because the index was calculated based on the top projection image without considering the lateral information, it can be used to provide spatial–temporal information of growing lettuces. Specifically, the site-specific growth monitoring might be feasible for the small number of plants because the variations in growth are displayed from the images for all growth periods [19,42]. In fact, this index enabled us to clearly identify both spatial variation in growth and differences in the photoperiods (Figure 9 and Figure 10). In particular, looking at the control condition graph in Figure 10A, it can be seen that the y-axis value decreases at the beginning of the change from the dark period to the light period, and then it gradually increases thereafter. This is considered to be a phenomenon that occurs because the leaf area is expanded due to the horizontal decrease in the leaf during the dark period [31,43]. It can be seen from the graph that it decreases again as the light period begins, but it gradually increases again with plant growth. Meanwhile, another analysis was possible. Under the experimental conditions, when the light period was twice that of the dark stage, and considering the length of the x-axis expressed in the graph, it can be judged that the growth of the crop mainly occurred during the dark stage (Figure 10A). At the beginning of the experiment, the difference was less than two times, but on day 7, the difference in area was slightly larger. Initially, the control group showed similar or slightly smaller values, but 7 days after planting, it showed a larger value than the pulse group. In fact, even in the destruction data, the group with a change in the photoperiod had poor growth, and the final weight showed a difference of 13.89 g when comparing the average value based on the fresh weight of the above-ground part; thus, it was interpreted that this part was revealed by the area index. The initial hypothesis was that the efficiency could be increased by changing the photoperiod, but it was judged that the efficiency was rather low due to the short light period. The length of the contour was similar to that of the area, indicating a difference between the two groups. However, unlike the area index, where the difference gradually increased, the length of the contour showed an initial large difference, which gradually decreased (Figure 10B). As the crop grew, the edge of the leaf flattened, and the curvature seemed to disappear due to overlapping. Therefore, a different tendency from the continuously increasing area index was observed. In particular, a relatively fast saturation phenomenon was observed in the control group. The minimum and maximum radii from the center were also projected values, and the minimum length index was not suitable for analysis of the photoperiod (Figure 10C,D). This was because the minimum radius showed a large change depending on the growth of new leaves and the effect of the light period. The maximum radius showed a tendency similar to that of the area at the beginning, but with saturation at a relatively early period, and the two groups reached the same value in the second half of the experiment. Unlike the area and contour length indices that could increase in various directions on the image, the maximum radius index could not be measured beyond the measurable range; therefore, this point is shown in the graph. Moreover, researchers or growers can add new phenotypic indices based on the camera network system. Then, the system could be used for building a new phenotyping model by building a new model between the phenotypic indices and plant physiological behaviors such as nutrient deficiencies, diseases, or pests.

When referring to studies relating to fresh weight, it was found that the analysis using the area index extracted from this study has explanatory power, and it can be said that the low error shows the accuracy of the measurement (Figure 11A) [3,44]. Similarly, it was confirmed that the maximum radius index and the contour index had a high coefficient of determination in each leaf length and leaf width (Figure 11B,C). The results show the usefulness of the system for monitoring and determining the phenotypic indices of growing plants.

From the index, the growth of different light/dark cycle were observed and compared. The growth of the pul group was lower than that of the con group, indicating the longer light/dark cycle would be more beneficial for vegetative growth of lettuces. Specifically, lettuce biomass did not change unidirectionally with the extension or shortening of the light/dark cycle, but only specific light/dark cycle can strengthen root growth and water absorption of lettuces [32,45]. Therefore, it is important to find the optimal cycle of the illumination on growing lettuces. The results show the developed system would provide the continuous monitoring of growing lettuces, thereby allowing more effective and rapid analysis on such variables.

Table 3. Comparison of the camera network system with the on-the-go monitoring system by Jiang, Kim and Cho [18].

System	Plant	No. of Plants	Phenotypic Characteristics	Code Availability	Cost	Ref.
Camera array HTPP *	Arabidopsis	1050	3	Commercial + Not open	~USD 43,200 (including camera)	[46]
Greenhouse-based HTPP *	Maize	756	3	Open	~USD 5000 (including camera)	[21]
An automated growth measurement system	Boston lettuce	45	4	Not open	~USD 2200 (including camera)	[18]
Sliding phenotyping platform	Maize, Tomato, Olive	1	8	Commercial	~USD 1300 (without camera)	[20]
Our system	Batavia lettuce	24	10	Open	~USD 1200 (including camera)	-

* HTPP: high-throughput plant phenotyping.

shows the comparison of the high-throughput plant phenotyping system developed in this study with other existing systems. The most expensive system using a 3D camera array showed a highly accurate measurement performance with a slope of 1.01 and R² of 0.99 in the comparison with the actual values [46]. The automated growth measurement system using a conveyer also showed a highly linear predictability for the fresh weight, with a slope near 1.0 and R² of 0.95 [18]. Although the predictability of the developed system is lower than those systems (Figure 11), it is comparable to other systems with R² of 0.6–0.8 [20,21]. Considering the performance, a total cost of USD 1200 is quite reasonable. The system also can measure various phenotypic characteristics and has a high degree of freedom in analysis compared to other systems. Moreover, the use of other devices might be limited due to their high costs or spatial reasons in densely growing environment such as plant factories. In contrast, the proposed system has excellent space efficiency through a small-sized camera module and a network-based multi-connection protocol. Continuous monitoring of phenotypic indices relating to height, leaf area, volume and biomass are effective parameters that could allow farmers to conduct better fertilizer and water management as well as scheduling of their harvest [47,48]. In plant factories, leafy vegetables are usually harvested when the leaves reach full vegetative growth; thus, it is important to monitor the growth during the vegetative stage. Accordingly, in this paper, lettuce observation was continued from two days after transplantation until the end of the first harvest, and usable phenotypic indices were successfully presented.

From the above results, it is demonstrated that the developed system can be applied to the real-time analysis of growth differences by group for various growth condition changes in plant factories, as well as spatial variation analyses according to the visualization of growth distribution. In future research, we will improve the measurement performance of the system by adding a side-view camera to collect information about crop height and areas that cannot be observed from the top view. In particular, we intend to apply a machine learning algorithm to the various phenotypic indices collected from the system to understand more complex crop-growth information and to develop a web-based interface, such that users can freely select and modify functions for each process.

5. Conclusions

In this study, we developed an analysis system for high-speed mass phenotyping based on a multi-point camera network using low-cost cameras and processor boards. To ensure its applicability to various cultivation structures and environments, a server computer was configured for several image collection modules to automatically import data from multiple points. Transmission stabilization conditions were applied for data collection stability, and a large amount of image information was transmitted from the local to the main server at daily intervals. When the transmission was completed, an ROI image was obtained through background separation, and 10 phenotypic indices were calculated using an image processing algorithm. The processing time of approximately 2.43 s per image confirmed the real-time analysis performance of the developed system. By applying the system to a commercial plant factory environment, it was possible to determine the growth distribution information and the influence of the growth environment for each individual based on various phenotypic indices through the data frame, confirming its feasible application. The projected area index-based estimation for fresh weight, leaf length, leaf width, and the number of leaves shows a high R² over than 0.9, indicating the system would be feasible for monitoring the growth information of growing lettuces. In addition, the high scalability of easily removing or adding programming parts through modularization of the calculation and analysis structure of the phenotypic index can be useful for growers or researchers who are concerned. In future research, we intend to introduce a side-view camera for crop height analysis and strengthen the field usability by adding machine learning algorithms. Overall, the developed system is expected to effectively help crop-growth management, as it can respond to changes in the dense plant factory cultivation environment and structural changes through low-cost camera installation and network connection configuration.