Nutrient Stress Symptom Detection in Cucumber Seedlings Using Segmented Regression and a Mask Region-Based Convolutional Neural Network Model

Abstract

The health monitoring of vegetable and fruit plants, especially during the critical seedling growth stage, is essential to protect them from various environmental stresses and prevent yield loss. Different environmental stresses may cause similar symptoms, making visual inspection alone unreliable and potentially leading to an incorrect diagnosis and delayed corrective actions. This study aimed to address these challenges by proposing a segmented regression model and a Mask R-CNN model for detecting the initiation time and symptoms of nutrient stress in cucumber seedlings within a controlled environment. Nutrient stress was induced by applying two different treatments: an indicative nutrient deficiency with an electrical conductivity (EC) of 0 dSm⁻¹, and excess nutrients with a high-concentration nutrient solution and an EC of 6 dSm⁻¹. Images of the seedlings were collected using an automatic image acquisition system two weeks after germination. The early initiation of nutrient stress was detected using a segmented regression analysis, which analyzed morphological and textural features extracted from the images. For the Mask R-CNN model, 800 seedling images were annotated based on the segmented regression analysis results. Nutrient-stressed seedlings were identified from the initiation day to 4.2 days after treatment application. The Mask R-CNN model, implemented using ResNet-101 for feature extraction, leveraged transfer learning to train the network with a smaller dataset, thereby reducing the processing time. This study identifies the top projected canopy area (TPCA), energy, entropy, and homogeneity as prospective indicators of nutritional deficits in cucumber seedlings. The results from the Mask R-CNN model are promising, with the best-fit image achieving an F1 score of 93.4%, a precision of 93%, and a recall of 94%. These findings demonstrate the effectiveness of the integrated statistical and machine learning (ML) methods for the early and accurate diagnosis of nutrient stress. The use of segmented regression for initial detection, followed by the Mask R-CNN for precise identification, emphasizes the potential of this approach to enhance agricultural practices. By facilitating the early detection and accurate diagnosis of nutrient stress, this approach allows for quicker and more precise treatments, which improve crop health and productivity. Future research could expand this methodology to other crop types and field conditions to enhance image processing techniques, and researchers may also integrate real-time monitoring systems.

1. Introduction

As the global population continues to grow, new and creative management strategies are required to maintain the agricultural industry’s sustainability. By 2050, there will be over 9.7 billion people on the planet [1]. Meeting the challenge of feeding such a large population will require the application of several concepts and methods from other fields.

Cucumbers (Cucumis sativus L.), an essential horticultural crop, are highly valued for their nutritional content and economic importance [2]. Cucumbers are the second most widely cultivated cucurbitaceous crop in the world, following watermelons [3]. However, like other plants, cucumber seedlings are susceptible to various environmental stresses that can significantly affect their growth and yield [4]. Nutrient stress, in particular, is a critical factor that can hamper the development of cucumber seedlings, leading to reduced crop quality and productivity [5]. Effective and timely detection of nutrient stress is thus crucial for mitigating its adverse effects and ensuring optimal plant health.

Abiotic stresses such as water scarcity, nutrient deficiency, temperature fluctuation, and light intensity variation significantly affect crop physiology and morphology, impacting plant health and productivity [6]. Similarly, insufficient irrigation can lead to water stress, severely threatening plant survival [7]. Nutrient deficiency presents another critical challenge, manifesting in the form of symptoms like blossom end rot (BER) from calcium deficiency, mottled maturation due to potassium shortage, and chlorosis from nitrogen inadequacy, which collectively compromise the plant’s growth and yield [8]. Compared to the optimum temperature, higher or lower temperatures can cause harm to the plant’s growth, pollination, and reproduction [9,10]. When considering the impact of the nutrient composition on plant growth, imbalanced concentrations of calcium and potassium, which may be caused by the detrimental effects of salinity, can be identified as impactful. Accordingly, extensive research has shown that salinity can cause nutritional stress in plants [11,12,13,14]. If these potential causes of yield losses are not properly handled in the very early stages of plant growth, such as in the seedling stage of plants, then they may have an impact on a number of aspects of food security, including accessibility and affordability [15].

Traditional methods, such as visual assessment of the plant’s color, size, and shape, lack the precision needed for an accurate, quantitative analysis [16]. Moreover, lab-based leaf analysis, another common approach, is not only time-consuming and destructive but also requires complex processes to yield accurate results [17]. Additionally, using a chlorophyll meter (SPAD) to gauge nutrient stress may be impractical for many farmers [18]. As a result, much effort has been directed toward the development of new technologies for detecting and estimating nutritional deficiencies in crops. For instance, the effective early detection of crop stress through computer vision is popular in agriculture.

Current research into online plant stress sensing techniques focuses on spectroscopy and image processing approaches. There has been a lot of interest in automatically detecting and classifying plant disorders [19]. Several technologies, such as electrical impedance spectroscopy [20], Fourier-transform infrared micro-spectroscopy [21], and chlorophyll fluorescence spectroscopy [22], have been utilized to collect the information needed for automatic nutrient detection. Moreover, various types of digital images are being used for plant nutrient stress monitoring approaches including chlorophyll fluorescence [23], thermal [24], multispectral [25], and hyperspectral [26]. Among these images, red–green–blue (RGB) images are commonly used due to their low cost, portability, and the availability of the necessary cameras.

Deep-learning-based segmentation is an excellent solution for anomalous leaf segmentation [27,28], disease area identification [29,30], color distortion [31], shape detection [32], and disease quantification [33]. Moreover, early detection and prediction of plant stress can be achieved using these methods, and compared to traditional methods that are time-consuming, labor-intensive, and often destructive, they are advantageous [34]. Recently, advanced methods have been developed to identify crop water stress [35] and nutrient stress [36], enabling the differentiation between stressed and non-stressed plants. An analysis of color and textural features is required to distinguish the characteristic differences between healthy and symptomatic plants [37]. By employing various regression analysis techniques, researchers can accurately assess and predict the onset of plant stress via color features, like leaf pigmentation changes, signal nutrient deficiencies or water stress (e.g., yellowing for nitrogen deficiency). Meanwhile, textural features such as leaf surface variations, also aid in stress detection, with healthy plants displaying a uniform texture and stressed ones showing irregularities like wilting [38].

In recent years, the integration of computer vision and deep learning techniques has emerged as a promising method for automating the detection and analysis of plant stress. Machine learning (ML) approaches collect relevant features from large datasets, learn the data using algorithms, and generate predictions and decisions. A non-destructive nutrient recognition technology for rapeseed has been proposed [39]. The multifractal detrended fluctuation (MF-DFA) method provides twelve textural parameters, comprising six generalized Hurst exponents and six relative multifractal parameters. These parameters were used to identify two nitrogen levels in rapeseed leaf images: N-mezzo and N-wane. The diagnostic accuracy was calculated using five classifiers: Fisher’s linear discriminant algorithm (LDA), an extreme learning machine (ELM), the support vector machine and kernel method (SVMKM), random choice forests (RFs), and KNN. The average discriminant accuracy levels for the SVMKM and RF techniques with 10-fold cross-validation were 97.12% and 97.56%, respectively. A study to identify nutrient deficiency in maize leaves through a laboratory test was conducted [40]. The Gray-Level Co-Occurrence Matrix (GLCM), hu-histogram, and color histogram were employed as parameters for the classification method. Random forest classifiers achieved a 78.35% accuracy. Meanwhile, given the volume and complexity of application data, the method of manually extracting picture features from the data using ML could not solve the problem quickly and easily. The fundamental disadvantage of these classic ML techniques is that the hand-crafted features suit a particular kind of data, and hence the applications of the algorithms are limited to specific jobs. This shows that a strategy that works effectively for one task might not be effective for another. This level of specificity associated with traditional methodologies severely limits their applicability in situations requiring a certain level of generality, prompting researchers to seek approaches that are highly generic and thus have a high applicability.

The convolutional neural network (CNN) is a deep learning system that specializes in image recognition and has been employed to meet a range of computer vision challenges across many fields [41]. A mask-RCNN model to predict geometric size of soil was proposed and was compared with the multiple instance segmentation algorithm for ground-penetrating radar images [42]. A phenotyping approach, involving a 23-layered deep learning technique, was proposed and compared with the classic ML techniques as well as a few other deep architectures [36]. The results showed that a simple 23-layered deep learning architecture could achieve ceiling-level nutritional stress classification from plant shoot images in the same way as advanced deep learning architectures such as ResNet18 and NasNet Large (with millions of trainable parameters). Furthermore, the suggested model outperformed existing ML techniques, with an average accuracy increase of 8.25%. RGB images of sugar beet were used to recognize nutrient deficiency [43]. Gathered dataset was Deep Nutrient Deficiency for Sugar Beet (DND-SB), which contained 5648 images of sugar beets growing under a long-term fertilizer experiment with nutrient deficiency plots that included N, P, and K deficiency, as well as an absence of liming Ca, full fertilization, and no fertilization at all. Then, the dataset was used to compare the performance levels of five convolutional neural networks in detecting nutritional shortage symptoms. The nutrient status of aquaponics lettuce was diagnosed using color imagery and deep convolutional neural networks (DCNNs) [44]. The method comprised several stages, including the detection of plant objects and the classification of nutritional deficiencies. The technique’s robustness and diagnostic capabilities were assessed using a total of 3000 lettuce photos, divided into four nutritional classes: full nutrition (FN), nitrogen deficiency (N), phosphorous deficiency (P), and potassium deficiency. The performance of the DNN was compared to typical ML techniques. The findings showed that the suggested deep segmentation model had an accuracy of 99.1%. Nutrient stress symptoms are often shown in plants through slight and intricate changes in leaf morphology, such as variations in size, shape, and edge characteristics. Sophisticated detection tools are necessary to capture and analyses these fine details. Object detectors excel in identifying and localizing objects within images, making them particularly suitable for examining individual leaf characteristics. A Mask R-CNN is especially effective because it not only detects objects but also generates high-quality segmentation masks for each instance. This capability is crucial for precise morphological analysis, enabling the isolation and examination of specific leaf regions affected by nutrient stress.

In that context, the present study explored the application of segmented regression with a Mask Region-based Convolutional Neural Network (Mask R-CNN) model to create a robust framework for the early detection and monitoring of nutrient stress in cucumber seedlings. Segmented regression is a statistical method that identifies changes in the trends in the data, which can be particularly useful in pinpointing the initiation of nutrient stress symptoms. Meanwhile, the Mask R-CNN is a state-of-the-art deep learning model designed for image segmentation tasks. The Mask R-CNN can effectively identify stress symptoms in cucumber seedlings images, providing detailed and accurate visual diagnostics. The objective of this study was to propose a segmented regression model and a Mask R-CNN model to detect the stress initiation time and the symptoms of nutrient stress in cucumber seedlings grown within a controlled environment.

2. Materials and Methods

2.1. Seedling Preparation and Growth Condition

The plant factory used in this experiment is located in the Department of Agricultural Machinery Engineering, Chungnam National University, Daejeon, Republic of Korea, as shown in Figure 1a. The plant factory aims to produce high-quality agricultural products with stable, year-round, mass production in a controlled environment, minimizing resource inputs and environmental impacts [45]. As per the guidelines for growing horticultural crops in a controlled environment, a wireless sensor and control network were used for controlling and monitoring the ambient environmental conditions, as shown in Figure 1b. The plant factory was equipped with evaporative cooling fans and air conditioners. An automatic environment control system, embedded with a relay, temperature sensor, humidity sensor, carbon dioxide sensor, and Raspberry Pi (model: Raspberry Pi 4B, Raspberry Pi Foundation, Cambridge, UK) microcontroller, was used to maintain the desired environmental conditions.

Cucumber seeds, specifically the Cucumis sativus var. ‘Begdadagi’, were sown in a 104-cell hydroponic germination sponge (LxWxH: 508 × 248 × 38 mm) in a controlled environment (Figure 2a). The seeds were added to the sponge and covered with paper to germinate the seeds (Figure 2b), and after their germination, the seedlings were grown under a uniformly distributed fluorescent lamp (TLD 32W/865RS, Phillips, Amsterdam, The Netherlands) (Figure 2b) with a photosynthetic photon flux density (PPFD) of 150 ± 10 µmolm⁻²s⁻¹, temperature of 24 ± 1 °C, photoperiod of 18/6 h (day/night), relative humidity of 65 ± 5%, and carbon dioxide (CO₂) concentration of 600–800 ppm. The seedling beds were placed in a single tray kept under fluorescent light, maintaining a distance of 0.345 m, as shown in Figure 2b. Starting from the seventh day after germination, the cucumber seedling trays were subjected to three distinct EC levels: 0.0 dSm⁻¹ (indicative of nutrient deficiency), 3.0 dSm⁻¹ (control, representing the optimal nutrient availability), and 6.0 dSm⁻¹ (excessive nutrient level), to span a range of potential nutrient stress conditions commonly encountered in precision agriculture [46]. These levels were selected based on preliminary studies indicating their significance in affecting cucumber seedling growth and health, aligning with the objective of simulating nutrient stress conditions within a controlled environment [47].

In a plant factory, effective water and nutrient application is important for seedling growth. Three beds for the three nutrient levels were prepared for cucumber seedling growth. For each bed, 1 L/tray/day of water with a nutrient solution of 0, 3, and 6 dSm⁻¹ was applied. These variations aimed to understand stress effects while following practices to ensure proper nutrient delivery and prevent over or under watering. Commercial Hoagland nutrient solutions A and B (Daeyu Co., Ltd., Seoul, Republic of Korea) were used, containing essential and trace nutrients as described by Islam et al. [47]. To achieve the desired nutrient level, 5 mL of nutrient solutions A and B were mixed with 5 L of water, and 1 L of this mixture was provided daily to each bottom irrigation tray, positioned beneath the seedling growing tray. The targeted nutrient level was maintained by checking the EC and pH levels daily during the watering practice. The seedling roots absorbed the water from the bottom irrigation tray, facilitating water uptake by plants.

2.2. Image Acquisition

Image data collection was performed using a multi-camera system shown in Figure 2c. The system specifications are detailed in

Table 1. Specifications of the microcontroller and RGB camera used in this study.

Raspberry Pi 4B	Raspberry Pi Camera
CPU: Quad core Cortex-A72, 64-bit@ 1.5 GHz RAM: 8 GB LPDDR4-3200 Connection: 802.11ac wireless, Bluetooth 5.0, BLE, Gigabit Ethernet, 40-pin GPIO header OpenGL ES 3.0 graphics Micro-SD card slot: 32 GB; operating system and data storage Power: 5V DC; USB-C connector and GPIO Operating temp range: 0 to 50 °C	Sensor model: Sony IMX 219 PQ CMOS Sensor size: 3.68 × 2.76 mm Lens size: 1/4” Resolution: 8 MP Image resolution: 3280 × 2464 pixels Video resolution: 640 × 480 pixels Pixel size: 1.12 × 1.12 µm Video/Image mode: 1080 p: 30 fps; 720 p: 60 fps Image control: Automatic Connection: 15-pin MIPI CSI-2

. A cheap, portable, and high-quality RGB camera (model: Raspberry Pi Camera, Raspberry Pi Foundation, Cambridge, UK) was used for the seedling bed to capture seedling images from the top view. To ensure a uniform image quality, the camera was installed at a vertical height of 0.315 m above the seedlings, optimized for maximum field of view (FOV) coverage (Figure 2c). Recognizing potential challenges such as variations in natural light throughout the day, a controlled lighting system using fluorescent lights, same as the fluorescent light used to provide light to the seedlings was implemented to maintain consistent illumination during image capture. This mitigation strategy was critical for minimizing shadows and reflections that could interfere with the image analysis.

The placement of the camera and the automation of the image capture process were carefully prepared with the microcontroller and integrated display to achieve a comprehensive coverage of each seedling bed, while ensuring minimal disturbance to the natural growth environment (Figure 2c). Automated image capture through a microcontroller was conducted consistently during the developmental stages of the seedlings at predetermined intervals, ensuring a rich dataset for analysis. The automation protocol was developed to capture images every 30 min, a frequency determined to optimally balance between capturing significant growth changes and managing the data volume. To reduce the effects of camera jitter or potential image blurring, three images were taken for each seedling bed and then averaged for subsequent analysis. The images were saved in the JPG format with a resolution of 3280 × 2464 pixels on a microSD memory card connected to the microcontroller. The imaging system was built for efficient remote monitoring, featuring a Virtual Network Computing (VNC) viewer for remote control via a graphical user interface (GUI), as elaborated by Islam et al. [48]. VNC offers a remote desktop technology with open-source code under a General Public License (GPL), with commercial versions.

2.3. Dataset Preparation

2.3.1. Image Feature Extraction

This study investigated the stress symptom initiation time and detected stress symptoms in cucumber seedlings. A total of 720 images from each treatment were acquired for analysis. An image of a cucumber seedling acquired from each treatment is shown in Figure 3. To perform a piecewise regression analysis to detect the stress induction time, a feature extraction process using RGB images was performed, as shown in Figure 4. Before feature extraction, the data were preprocessed to optimize the image quality for analysis, applying a Gaussian blur filter for noise reduction and improved clarity, and normalizing the brightness and contrast to offset lighting variations during image capture. Subsequently, we precisely extracted the seedlings’ top canopy area using sophisticated image segmentation, effectively separating the plant foreground (Figure 4) for targeted analysis. The resulting image represented the top projected canopy area (TPCA) of the plant. This area of a plant was represented by the number of green pixels in the image characterized as morphological features. Image analysis was performed by extracting textural features according to the statistical distribution of pixel intensity.

The selection of textural features for stress detection was based on a comprehensive review of the plant physiology literature, which indicated that changes in textural properties could be indicative of nutrient stress. Specifically, GLCM method was used for robustness in capturing textural variations [49]. When using the GLCM method, an orientation pattern angle is derived from characteristics of the pixel matrix value [50]. This allows for entropy, energy, contrast, and homogeneity features to be calculated, which has proven to be useful in identifying subtle changes in plant health [51]. These features were specifically chosen for their sensitivity to variations in the leaf surface texture and pattern, which are affected by nutrient availability, thus providing a reliable basis for detecting nutrient stress. In addition, the GLCM method analyses the spatial relationship between pixel intensities in an image, capturing information about texture patterns. Nutrient stress can alter these textural properties by changing the internal structures of leaves. The use of GLCM textures indicates that nutrient stress will impact the leaf structural patterns, making these features particularly useful for stress detection.

2.3.2. Stress Symptom Initiation and Detection

To detect early environmental stress induced in the seedlings, a separation point indicating the initiation of stress due to nutrients was determined by calculating the mean difference between the treatment and control seedlings at each measured time for all collected data.

To accurately segment the dataset into stressed and non-stressed groups, piecewise-segmented regression analysis, a robust statistical method capable of identifying change points in temporal datasets, was used [52]. This approach was instrumental in pinpointing the exact moment when the physiological parameters of the seedlings began to deviate significantly due to nutrient stress. By comparing the mean differences in textural features between the treatment and control groups over time, a clear demarcation point could be established [53]. This change point, validated through rigorous statistical analysis, enabled the precise classification of images into the stressed and non-stressed categories, forming the basis for our subsequent model training and validation. The separation point in the graphs showing the mean difference between treatment and control seedlings at every measured time for all morphological and textural criteria was used to determine early nutrient-deficit stress [54,55]. The change point can be described as shown in Equation (1) [54]:

$$\mathrm{Z}\left({\mathrm{t}}_{\mathrm{i}}\right)={\mathrm{T}}_{\mathrm{d}}\left({\mathrm{t}}_{\mathrm{i}}\right)-{\mathrm{T}}_{\mathrm{h}}\left({\mathrm{t}}_{\mathrm{i}}\right)$$

(1)

where T_d and T_h are the average values of stressed and healthy seedling parameters at time t_i of the experiment.

At some specific point, the values of the retrieved parameters for the treatment seedlings began to diverge from those of the control seedlings. The time was divided into two intervals, and each interval was fitted with a different regression line by minimizing the sum of squares of the difference between the observed response and the calculated dependent variable [54]. The change point, t_c, estimating the breakpoint between the two regression equations, can be determined with Equations (2) and (3):

$${\mathrm{Z}}_{\mathrm{b}\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{k}}={\mathsf{\beta }}_0+{\mathsf{\beta }}_1\mathrm{t},\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{n}\mathrm{t}<{\mathrm{t}}_{\mathrm{c}}$$

(2)

$${\mathrm{Z}}_{\mathrm{b}\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{k}}={\mathsf{\alpha }}_0+{\mathsf{\alpha }}_1\mathrm{t},\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{n}\mathrm{t}<{\mathrm{t}}_{\mathrm{c}}$$

(3)

Consequently, two regression lines were constructed to closely match the observed data by minimizing the sum of squared differences between the observed response (Z) and the predicted dependent variable (Z_break). Thus, segmented regression analysis was used to analyze the profile difference and establish when stress symptoms were initiated. The statistical computations were carried out with SAS software (Ver. 9.4; SAS Institute Inc., Cary, NC, USA). Based on the statistical analysis, the dataset before the change point was considered the non-stressed dataset, and the dataset after the change point was considered the stressed dataset.

2.3.3. Dataset Description for the Detection Model

The seedling images obtained from the regression analysis were utilized as training and validation datasets in this experiment. The original image was 3280 × 2464 pixels in size, and the seedling count per image was around 70, resulting in a low pixel count per seedling. We chose 800 images at random and adapted them to the Mask R-CNN model and parameters, with 70% of the images being used for training and 30% being used for validation. To train the model, the seedling images were randomly selected based on the seedling age, size, and overlap between seedlings.

Due to the uneven morphology of seedlings at various growth stages, seedling annotation was performed manually. The annotation process was carried out in this study using a web-based application MakeSense.ai (“https://www.makesense.ai (accessed on 23 November 2023)”), which is freely available and distributed under a General Public License (GPL) version 3. One of the main advantages of this tool is that it can be accessed and used directly through a standard web browser, without the need for installing any specialized software. The application of manual annotation of the seedling images, as shown in Figure 5, served the purpose of clearly delineating and identifying cucumber seedlings in various stages of their growth.

2.4. Mask R-CNN Model

The Mask R-CNN (Region-based Convolutional Neural Network) is a sophisticated deep learning architecture designed for instance segmentation tasks, which involve detecting and delineating individual objects within an image. Developed as an extension of the Faster R-CNN framework [56,57], the Mask R-CNN introduces an additional branch for generating segmentation masks alongside the existing components for object detection. This architecture enables the precise pixel-level identification of object boundaries, allowing for more accurate and fine-grained segmentation. By incorporating a Region Proposal Network (RPN) to propose candidate object regions, followed by a refinement process that includes both bounding box regression and mask prediction, the Mask R-CNN excels in simultaneously handling object localization and segmentation tasks. Widely adopted in computer vision applications, particularly in areas such as image segmentation and object recognition, the Mask R-CNN has proven to be an effective and versatile solution for complex visual understanding tasks. The image to be processed was first fed into the pre-trained backbone network ResNet50 and the feature pyramid network model in Mask R-CNN, which extracted features and generated feature maps. By applying the created feature maps as inputs into the Regional Proposal Network, a large number of candidate frames were obtained (regions of interest or ROIs). The SoftMax classifier was used to perform binary classification of the foreground and background, and frame regression was used to obtain more accurate candidate frame position information [58]. Afterward, the ROIs that were clipped from feature maps were passed through the ROI align layer. As a last step, the flow splits into two: one branch goes to the fully connected layer for object detection and bounding box regression, and another branch goes to the full convolutional network (FCN) for pixel segmentation as well as mask creation. The Mask R-CNN architecture is shown in Figure 6.

2.4.1. Mask R-CNN Backbone Network and Feature Pyramid Network (FPN)

In this study, the ResNet-101 architecture was chosen as the foundational framework for the model, and the input size was 1024 × 1024. To determine the most effective model combination, performance evaluations were conducted across various permutations of backbones and segmentation mask predictors. Given the substantial size of the network in comparison to the relatively limited dataset, pre-trained weights from backbones trained on the extensive ImageNet dataset were harnessed for the purpose of transfer learning [59]. Deep learning models tend to exhibit greater robustness when exposed to larger and more diverse datasets. Previous studies have established that leverage pre-trained backbones, particularly those trained on vast datasets like ImageNet, can yield significant performance improvements when fine-tuned for specific tasks. Furthermore, this practice of transfer learning, where knowledge gained from extensive datasets is adapted for use with smaller datasets, enhances the model’s ability to generalize learning. The architecture of the feature extractor is shown in Figure 7, encompassing five primary convolution modules that collectively made a total of one hundred convolutional layers. Additionally, this structure incorporates an average pooling layer and a fully connected layer. The features extracted by this component are subsequently employed for training the classifier, as implemented with a Softmax configuration. The specific convolutional layers and their respective configurations within the ResNet-101 backbone are comprehensively detailed in

Table 2. ResNet-101 backbone configurations and major layers used in this study.

Layer	Conv1_x	Conv2_x	Conv3_x
Output size	112 × 112 × 64	56 × 56 × 56	28 × 28 × 128
Filters	7 × 7, 64 stride 2	$\left[3\times 3, 64 \begin{array}{c}1\times 1, 64\\ 1\times 1, 256\end{array}\right]\times 3$	$\left[3\times 3, 128 \begin{array}{c}1\times 1, 128\\ 1\times 1, 512\end{array}\right]\times 4$
Layer	Conv4_x	Conv5_x	Pooling
Output size	14 × 14 × 256	7 × 7 × 512	1 × 1 × 512
Filters	$\left[3\times 3, 256 \begin{array}{c}1\times 1, 256\\ 1\times 1, 1028\end{array}\right]\times 23$	$\left[3\times 3, 512 \begin{array}{c}1\times 1, 512\\ 1\times 1, 2048\end{array}\right]\times 3$	Average

.

An FPN employs a horizontally connected top-down hierarchical structure to build a network feature pyramid from a single-scale input. It can solve multi-scale challenges when extracting target objects from images and it requires fewer parameters. An FPN is constructed through a combination of both bottom-up and top-down pathways, along with lateral connections. An FPN adopts the structure of feature maps found in a single-shot multibox detector (SSD). It efficiently combines these feature maps, resulting in a significant boost in accuracy without adding to the detection time. Unlike SSD, the FPN does not solely rely on deep feature maps; it also leverages shallower feature maps in its architecture [60]. The FPN generates four sets of feature maps through a bottom-up approach, as shown in Figure 8. These feature maps, labeled C1 to C5, have varying levels of detail, with C2 containing more textural information and C5 containing more semantic content. FPN then uses a top-down and lateral connection method to merge these feature maps. Since C1 to C5 have different down-sampling rates, their sizes differ. To enhance the computational efficiency, the FPN initially reduces dimensionality using 1 × 1 convolutions to create P5, which is then up-sampled to match the size of C4 using bilinear interpolation. P4 is similarly dimensionally reduced and adjusted to match the size of P5. This process continues down the network layers, updating features through unit addition, known as the horizontal connection. After fusion, each result undergoes a 3 × 3 convolution to mitigate aliasing effects from up-sampling. The aim is to ensure that P2, P3, P4, and P5 all have the same number of feature maps [61].

2.4.2. FPN with Regional Proposal Network (RPN)

In seedling image extraction, handling various seedling sizes is a challenge for a single convolutional neural network (CNN). To address this, an FPN is used to create different-sized feature maps. The Region Proposal Network (RPN) then takes these feature maps as input and extracts region of interest (ROI) features based on the specific seedling size. This modified network significantly improves the detection of small objects with a high speed and precision.

The RPN operates as a sliding-window-based target detector, employing a convolutional neural network structure, as shown in Figure 9. It uses sliding frames to generate anchor points with varying sizes and areas, enabling the detection of the target within each sliding window. The choice of anchor point sizes and the required areas’ overlap (IOU) directly impact the classification accuracy. The algorithm adjusts anchor point zoom ratios to {1:2, 1:1, 3:1} and aspect ratios to 2 to adapt to different seedling types. This approach maximizes coverage for a wide range of seedling images.

Intersection over Union (IoU) quantifies the overlap between actual and predicted boxes. It is calculated by dividing the intersection of the two boxes by their union. IoU in the RPN closely matches the empirical IoU, remaining unchanged in the network model. In this work, an overlap ratio greater than 0.5 between the anchor frame and the actual target area designates it as foreground, while an overlap ratio less than 0.5 marks it as the background. Values in between are disregarded. This reduction in calculations streamlines the model, saving computation time, and boosts its effectiveness. To compute IoU, Equation (4) was used, and the sample IoU scores are shown in Figure 10.

$$\mathrm{I}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{s}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n} \mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r} \mathrm{U}\mathrm{n}\mathrm{i}\mathrm{o}\mathrm{n} (\mathrm{I}\mathrm{o}\mathrm{U})={\displaystyle \frac{\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a} \mathrm{o}\mathrm{f} \mathrm{O}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{l}\mathrm{a}\mathrm{p}}{\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a} \mathrm{o}\mathrm{f} \mathrm{U}\mathrm{n}\mathrm{i}\mathrm{o}\mathrm{n}}}$$

(4)

2.4.3. RoIAlign and Loss Function

Mask R-CNN employs a modified region of an interest alignment layer (RoIAlign) mechanism, introduced by Gui et al. [62], to enhance pixel-level object detection and segmentation. Unlike the traditional RoI pooling, RoIAlign does not quantize ROI boundaries, using bilinear interpolation to accurately position sampling points. The process concludes with maximum or average pooling to extract key information from the region of interest. This refinement significantly improves the current network precision and spatial information preservation.

The loss function of Mask R-CNN consists of three parts: classification error, regression error, and segmentation error, as shown in Equation (5) [63]:

$$\mathrm{L}={\mathrm{L}}_{\mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}}+{\mathrm{L}}_{\mathrm{b}\mathrm{o}\mathrm{x}}+{\mathrm{L}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}$$

(5)

where L represents the total cost loss function of the model, L_class represents the classification loss of the prediction box, L_box represents the regression loss of the prediction box, and L_mask represents the average binary cross-entropy loss.

Classification, regression, and the segmentation layer can be described with Equations (6)–(8), as follows:

$${\mathrm{L}}_{\mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}}\left({\mathrm{p}}_{\mathrm{p}},{\mathrm{p}}_{\mathrm{v}}\right)=-\mathrm{l}\mathrm{b}[{\mathrm{p}}_{\mathrm{p}},{\mathrm{p}}_{\mathrm{v}}+(1-{\mathrm{p}}_{\mathrm{p}})(1-{\mathrm{p}}_{\mathrm{v}})]$$

(6)

where p_p represents the predicted probability for the target anchor point, p_v represents the predicted value of the corresponding ground truth label, and lb represents the log loss function.

$${\mathrm{L}}_{\mathrm{b}\mathrm{o}\mathrm{x}}\left({\mathrm{t}}_{\mathrm{p}},{\mathrm{t}}_{\mathrm{v}}\right)=\mathrm{R}({\mathrm{t}}_{\mathrm{p}}-{\mathrm{t}}_{\mathrm{v}})$$

(7)

where R stands for the robust loss function, t_p represents the coordinate vectors of the predicted frame, and t_v represents the coordinate vector corresponding to the ground truth label.

$${\mathrm{L}}_{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}}={\displaystyle \frac{1}{\mathrm{x}}}{\sum }_{\mathrm{i}}\left[{\mathrm{x}}_{\mathrm{v}}\mathrm{l}\mathrm{g}\mathrm{p}\left({\mathrm{x}}_{\mathrm{v}}\right)-\left(1-{\mathrm{x}}_{\mathrm{v}}\right)\lg \left(1-\mathrm{p}\left({\mathrm{x}}_{\mathrm{v}}\right)\right)\right]$$

(8)

where x represents the number of pixels, x_v indicates the category label where the pixel is located, and p(x_v) represents the probability of the predicted category.

2.5. Training Configuration

The model training process for the custom dataset, which focuses on detecting objects of interest like seedlings, primarily involves a transfer learning approach. This technique leverages the knowledge learned by a Mask R-CNN model pre-trained on the MS COCO dataset [7], which is a widely used benchmark for object detection tasks. By utilizing these pre-trained weights, we may benefit from the model’s prior understanding of various object classes, allowing us to fine-tune it for our specific task.

The training process was conducted using the cloud platform (Google Colab; “https://colab.research.google.com (accessed on 15 December 2023)”), which provides access to a Tesla T4 GPU. However, training deep learning models with a complex architecture like Mask R-CNN can be computationally intensive and may require significant memory resources. Due to the memory constraints of the Google Colab, the number of training epochs was limited to 100. A learning rate of 0.001 was used during training. Additionally, a learning momentum of 0.9 was applied to update the model weights after each epoch. These settings control how much the model adjusts its weights during training, ensuring convergence to the optimal solution. To prevent overfitting and maintain model generalization, a weight decay of 0.0001 was employed. Weight decay imposes a penalty on large weights, effectively discouraging them and promoting a more balanced model. The training stability was ensured through hyperparameter tuning. To achieve the optimal model performance within the constrained 100 epochs, various hyperparameters were fine-tuned. This included adjusting the batch size, learning rate, choice of optimizer, padding settings, and selection of filters in the model configuration. Such hyperparameters play a critical role in the convergence and performance of deep learning models and were carefully optimized to maximize the model’s effectiveness.

2.6. Evaluation Matrices

A test was conducted to evaluate the effectiveness of the seedling identification and segmentation method using the training and test datasets. Precision, recall, the F1 score, and accuracy were used to quantify the overall performance of the Mask R-CNN method in this study. The parameters were calculated with the following Equations (9)–(12) [64]:

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}}$$

(9)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(10)

$$\mathrm{F}={\displaystyle \frac{2\times \mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}}$$

(11)

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}}}$$

(12)

where TP is the true positive detection, FN is the false positive detection, FP is the false negative detection, and TN is the true negative detection.

The mean average precision (mAP) refers to the mean of the average precision for different classes, which can be calculated from Equation (13) as follows:

$$\mathrm{m}\mathrm{A}\mathrm{P}={\displaystyle \frac{\sum _{\mathrm{i}=1}^{\mathrm{m}}{\mathrm{A}\mathrm{P}}_{\mathrm{i}}}{\mathrm{m}}}$$

(13)

In this study, seedlings were defined with an Intersection of Union (IoU) larger than 0.5 between the predicted and ground truth bounding boxes. The IoU value was set to be modest to reflect the diminutive size of a seedling in comparison with the image resolution.

3. Results

3.1. Detection of Stress Initiation

After the application of environmental stress on the seventh day of the germination process, our experiment was carried out for a total of ten consecutive days. The human visual stress detection event took place on the 6.5th day after the deployment of nutritional stress. During the course of the experiment, the plants in the treatment group looked to be a lighter shade of green, almost yellow, as a direct result of the lack of nutrients. In addition to this moderate deterioration of the leaves, along the margins of the leaves, there were also small spots that were dark brown in color.

The sequences of the retrieved plant properties (TPCA, entropy, energy, and homogeneity) as average values obtained from the control and treatment groups are shown in Figure 11. This demonstrates that the difference in the TPCA value between the control group of seedlings, which were developing normally, and the treatment group of seedlings, whose growth rate was stifled, increased. This occurred because the treated seedlings lacked the necessary nutrients.

Variations in the surface, texture, and interior leaf structures of plants are also known to result from the effects of stress [65]. In this study, textural analysis of the canopy of seedlings was carried out. In the textural analysis, the concept of entropy was understood to refer to the unpredictability of the gray-level distribution. Greater entropy values were discovered in the control seedlings in comparison with the treatment seedlings, with the control seedlings having vibrant and healthy leaves. Grayscale brightness indicates energy, and so we inferred that the energy value of the healthy control seedlings fell progressively, whereas the yellowish leaves of the treatment seedlings indicated that they gained energy. Meanwhile, there was homogeneity in the gray-level pixel distribution in the canopy images; here, as the control became greener, the gray-level pixel distribution dropped, while the treatment seedlings showed greater gray-level pixel values.

Using piecewise regression analysis, stress was determined statistically. Figure 12 and Figure 13 show the change point and regression line for each collected metric, indicating seedling stress before it became detectable by the human eyesight, for the 0.0 dSm⁻¹ and 6.0 dSm⁻¹ nutrient stress conditions. Four parameters changed on day 4.2 (97.5% confidence interval [3.9, 4.9] and R² = 0.98). Based on this statistical research, four of the indicators (TPCA, entropy, energy, and homogeneity) show promise as early warnings of stress due to nutritional insufficiency. Incorporating these four metrics into a statistical technique allowed us to identify nutritional deficit stress on day 4.2. In our study, this happened 1.5 days before human visual detection. Figure 11 shows that the breakpoint was on day 4 for cucumber seedlings. This not only shows the effectiveness of the method in identifying early stress but also highlights the applicability of the findings across a range of plant types. Moreover, stress and non-stress seedling classification also arises from the deviation point. Therefore, nutrient-stressed datasets were prepared containing images of 4- to 10-day-old seedlings after germination.

3.2. Training and Validation of Data

Model studies have shown that the number of iterations used in models has an impact on the outcomes of the training results. Training and validation loss are also essential metrics for understanding the performance and training progress of a Mask R-CNN model, or any ML model for that matter. Figure 14 shows the training and validation loss curve for the model. The model was run with 100 epochs and 1000 steps in each epoch. The training loss value decreased within the range of 1.983 to 0.46 when the epoch increased, and the same was found for the validation loss, it decreased within the range of 1.5623 to 0.323 when the epoch increased. The training loss decreased from 1.983 to 0.46 over 100 epochs, which shows that the model improves at fitting the training data, and the model was stable after 80 epochs. This suggests that the model learns to make more accurate predictions for the data it is trained on. However, the extremely low training loss values (close to 0) may indicate overfitting. Similarly, the decrease in the validation loss from 1.5623 to 0.323 with the increase in the epoch showed that the model performance improves over time. Lower validation loss values suggest that the model is improving at making accurate predictions on new, unseen data. This is a crucial indicator of the model’s overall performance.

The precision–recall (P-R curve) is an effective metric for assessing the prediction performance when the classes are unbalanced. The P-R curve presented in Figure 15 indicates that our proposed model attained a high seedling detection accuracy. The low loss values indicate that the model used in our study was capable of effectively learning the features and achieved better linkage, indicating that it had the potential to accomplish the intended objectives.

3.3. Performance Detection on Stressed Seedlings

The proposed nutrient stress detection model using the Mask R-CNN model was evaluated for its effectiveness in detecting and segmenting cucumber seedlings and detecting stress conditions in RGB images. The results of this evaluation, which included segmentation and stress detection for cucumber seedlings, are illustrated in Figure 16. In this figure, the segmented seedlings are displayed with distinct colored masks, and bounding boxes are used to indicate the detected stress conditions of the seedlings within the test images, along with the associated probability scores.

In the images, the bounding boxes represent each seedling that was anticipated to be detected by the method. For the final output of the experiment, only seedlings with a detection probability of at least 50% were selected and segmented. This threshold ensures a reliable identification and segmentation of the seedlings, providing a dependable output for further analysis or application. The segmentation masks and bounding boxes effectively illustrated both the presence and the stress conditions of the cucumber seedlings, offering a comprehensive visual summary of the method’s performance in handling various types of seedlings and detecting their stress states.

Table 3. Evaluation of nutrient stress detection of cucumber seedlings using the Mask R-CNN model.

Stress Condition	Condition	Precision Rate (%)	Recall Rate (%)	F1 Score	Accuracy	mAP
0 dSm−1	Average	0.91	0.89	0.90	0.90	0.927
	Best Fit	0.93	0.94	0.934	0.92
6 dSm−1	Average	0.88	0.87	0.874	0.87	0.88
	Best Fit	0.90	0.89	0.894	0.90

shows the result of the nutrient stress detection in cucumber seedlings using the Mask R-CNN model. In our findings, the nutrient stress detection algorithm achieved an average F1 score of 90%, with an average precision of 91% and an average recall of 89%. These indicated that it had a high level of accuracy in both correctly identifying seedlings and capturing all relevant instances, which comprised 90% of the instances for the nutrient stress condition of 0 dSm⁻¹. Moreover, the best-performing image achieved an F1 score of 93.4%, with a precision of 93% and a recall of 94%. The mean average precision for this analysis was found to be 0.927%. For the nutrient stress condition of 6 dSm⁻¹, the average F1 score was 87.4%, with an average precision of 88% and an average recall of 87%. These findings indicate a high level of accuracy in both nutrient conditions, while 6 dSm⁻¹ showed a lower accuracy and mAP for the detection of stressed seedlings. These metrics emphasize the robust capability of the algorithm in accurately detecting nutrient-stressed seedlings. The segmentation masks generated by the algorithm were highly effective in distinguishing all seedlings from the image background. They also successfully identified the structure of overlapping seedlings, which presents a significant challenge in image processing. This capability is particularly useful for practical applications such as harvest planning for robot harvesters or transplanting machines, where the precise identification of overlapping seedlings is critical.

Despite these successes, the algorithm did exhibit some inaccuracies, as shown by the false positive and false negative errors in Figure 16. These errors indicate instances where the algorithm either incorrectly identified non-seedlings as seedlings (false positives) or missed detecting actual seedlings (false negatives). To address these issues, various images were used to train the algorithm to recognize overlapping and closely situated seedlings, as well as smaller seedlings that might be more challenging to detect. Although the training improved the algorithm’s performance, there were still a few instances where seedlings were either not detected or were incorrectly identified. These occasional inaccuracies highlight areas for further refinement and optimization of the algorithm to enhance its overall reliability and accuracy in different scenarios.

4. Discussion

This study has investigated the impact of nutrient stress on cucumber seedlings over ten days. Human visual detection of stress occurred after 6.5 days, whereas the proposed regression model showed the symptoms within 4.2 days after stress introduction. During this time, plants showed visible signs of nutrient deficiency, such as lighter green or yellow leaves and tiny dark brown spots along the leaf margins, indicating moderate deterioration. Similar studies showed early detection of nutrient deficiency symptoms. A dual regression model was used to identify calcium deficiency in lettuce one day before it is visually detectable by humans [54]. The study found that TPCA, energy, entropy, and homogeneity are the most promising markers for the early detection of this deficiency. A similar approach was applied to detect nitrogen deficiency in cucumber, employing the same features (excluding top projected canopy area) as inputs for a dual-segmented regression model [55]. A computer vision system was used to extract color and textural features from leaves using methods like intensity histograms, Fourier transform, and wavelet packet, with a Genetic Algorithm for optimal feature selection, showing an 82.5% accuracy, diagnosing plant diseases 6–10 days before experts [66]. Gabor Wavelets and an artificial vision system (AVS) were used to detect nitrogen deficiency in maize leaves early [67]. The convolutional neural networks (CNNs) provided better feature representations for nitrogen-induced stress classification than the traditional ML methods, achieving an 8.25% higher accuracy with fewer trainable parameters [36].

Image processing and computer vision were used to analyze nutrient availability in rice leaves through feature extraction [65]. Six feature values were identified and used with the Learning Vector Quantization (LVQ) method, effectively detecting deficiencies in nitrogen, phosphorus, and potassium, with contrast, energy, and entropy as key markers. Additionally, multiple linear regression models were developed to prevent an underestimation of the nitrogen contents in rice leaves [68]. A non-destructive method was developed to monitor crop growth and the nitrogen status using digital image analysis [69]. Digital images of rice canopies, grown under various nitrogen treatments, were captured and analyzed to calculate the canopy cover and ten color indices.

The GLCM approach characterizes texture by analyzing the intensity value dispersion and the relationship between the location and orientation of pixels, making it superior to other methods for textural feature extraction [70]. An automatic identification of nutritional deficiencies was proposed using coffee tree leaf images which were segmented using Otsu’s method [71]. Subsequently, they applied the Blurred Shape Model (BSM) and GLCM descriptors to extract shape and textural characteristics from the resulting images. Using the color space in analysis studies also offers a promising way to detect nutrient deficiencies. The HSV color space was used to estimate the green pixel proportion in wheat canopy images, with a coefficient of determination of 0.19 for the nitrogen content [72]. RGB color and Sobel edge detection, along with artificial neural networks, was proposed for nutritional differentiation identification in cucumbers [73].

Figure 17 shows that our model achieved a high detection accuracy for the datasets under high IOU thresholds. Figure 16 shows the normalized confusion matrix (%) plot for the model, which may be used to visually evaluate its performance in stress detection. The x-axis in the matrix reflects the ground truth class for each image, while the y-axis shows the class predicted by the model. For example, one tray image had 80 seedlings, and, out of those 80 seedlings, 73 seedlings were correctly detected by the model for the treatment with EC 0 dSm⁻¹ as there were few overlapping seedlings. In the case of EC 6 dSm⁻¹, the accuracy declined due to leaves overlapping and there being a more complex condition with the development of seedlings.

This study has introduced a first approach to non-destructive nutrient stress detection in cucumber seedlings grown under different nutrient conditions within a controlled plant factory, utilizing a Mask R-CNN model. The nutrient detection model was compared to existing studies that used Mask R-CNN. An improved Faster R-CNN framework with focal loss was proposed for automatic detection of hydroponic lettuce seedlings, achieving 86.2% mAP on their dataset [74]. A mask R-CNN model was introduced to identify nutrient deficiencies in chili plants with an accuracy of 82.61% and the best mAP value of 85.57% [75]. Further research was suggested to develop mobile apps for the real-time identification of nutrient deficiencies in plants. A Mask R-CNN was applied to detect the optimum light conditions for lettuce leaves, yielding lower statistical metrics [76], and a Mask R-CNN model was used to identify cucumbers on a leafy branch, achieving an accuracy of 88.72% [77]. A strawberry fruit detection algorithm was proposed based on the Mask R-CNN, achieving an instance segmentation accuracy of 89.85% using ResNet-50 combined with the FPN architecture [78]. The Mask R-CNN and YOLO models were employed for a dataset from tomato plants in Sri Lanka for deficiency classification, achieving 92% and 98% accuracies, respectively, with deficiency dispersion levels expressed as a percentage, when using Mask R-CNN followed by image processing techniques [79].

Detecting nutrient-stressed cucumber seedlings in complex growing environments, such as those with varying light, occlusion, and overlap, is challenging. This research focused on visual representations of such environments, comparing images of seedlings grown without nutrients to those grown with high levels of nutrient stress. The model was capable of detecting stressed seedlings in more difficult and complex backgrounds with overlapping seedlings. Moreover, the model demonstrated accurate positioning and detection of target seedlings, even in challenging conditions, fulfilling the requirement for precise stress detection. Seedlings can be segmented accurately despite issues like camera haze. Although the proposed model showed a high detection accuracy, future research can be expanded to other crop varieties, and transitioning from controlled environments to open-field conditions will test the model’s robustness under varying natural conditions. Besides, studies should include direct analysis of the nutrient content at various applied nutritional levels, alongside EC measurements. Integrating multispectral and hyperspectral imaging with ML models can improve agricultural crop stress detection. Developing larger datasets, user-friendly applications and interfaces and IoT integration will streamline data collection and accessibility. Additionally, economic analyses will evaluate the cost-effectiveness and feasibility of the technology in agricultural settings.

5. Conclusions

In this study, we successfully applied segmented regression and Mask R-CNN models to detect the nutrient stress initiation time and stress symptoms in cucumber seedlings within a controlled environment. Nutrient stress was applied in a form indicative of nutrient deficiency with EC 0 dSm⁻¹ and excess nutrients as a high concentration of nutrient solution with EC 6 dSm⁻¹. All the seedling images were collected from the plant factory using an automatic image acquisition system. After pre-processing and background removal, morphological and texture features were extracted from the seedling images using the GLCM method. Segmented regression analysis was used to identify the early stress initiation time, and then a dataset for the Mask R-CNN model was prepared based on the stress initiation and stressed seedlings. The annotation of 800 seedling images was performed by an online annotation tool called MakeSense.ai. The Mask R-CNN was implemented using ResNet-101, which provides a feature extractor. Transfer learning was used to train the model network with a small dataset and reduce the processing time. The bounding box and annotations were set as inputs for the training model. Training and test datasets were used for the model evaluation. The segmented regression approach detected the nutrient stress initiation 1.5 days prior to the human eyesight. TPCA, energy, entropy, and homogeneity are prospective indicators of a nutritional deficit in cucumber seedlings. Multiple indicators are better than a single marker for stress detection. The total number of training epochs for Mask R-CNN was 100, and the proposed Mask R-CNN method showed a training loss of less than 0.46%. Our findings on the identification of stressed cucumber seedlings from the test images revealed that the best-fit image had a performance F1 score of 93.4%, a precision of 93%, and a recall of 94%. This study has successfully demonstrated the efficacy of the Mask R-CNN model in detecting nutrient stress in cucumber seedlings. Through rigorous experimentation and validation, we have established the model’s ability to accurately identify and classify nutrient stress symptoms, providing a robust tool for early detection and intervention in cucumber cultivation. The integration of Mask R-CNN not only enhances the precision of nutrient stress detection but also streamlines the monitoring process, allowing for timely and targeted corrective measures. This collaborative approach will contribute to the ongoing evolution of precision agriculture, ensuring the resilience and productivity of crop yields in the face of changing agricultural landscapes and global food security challenges.