Image Analysis Artificial Intelligence Technologies for Plant Phenotyping: Current State of the Art

Abstract

Modern agriculture is characterized by the use of smart technology and precision agriculture to monitor crops in real time. The technologies enhance total yields by identifying requirements based on environmental conditions. Plant phenotyping is used in solving problems of basic science and allows scientists to characterize crops and select the best genotypes for breeding, hence eliminating manual and laborious methods. Additionally, plant phenotyping is useful in solving problems such as identifying subtle differences or complex quantitative trait locus (QTL) mapping which are impossible to solve using conventional methods. This review article examines the latest developments in image analysis for plant phenotyping using AI, 2D, and 3D image reconstruction techniques by limiting literature from 2020. The article collects data from 84 current studies and showcases novel applications of plant phenotyping in image analysis using various technologies. AI algorithms are showcased in predicting issues expected during the growth cycles of lettuce plants, predicting yields of soybeans in different climates and growth conditions, and identifying high-yielding genotypes to improve yields. The use of high throughput analysis techniques also facilitates monitoring crop canopies for different genotypes, root phenotyping, and late-time harvesting of crops and weeds. The high throughput image analysis methods are also combined with AI to guide phenotyping applications, leading to higher accuracy than cases that consider either method. Finally, 3D reconstruction and a combination with AI are showcased to undertake different operations in applications involving automated robotic harvesting. Future research directions are showcased where the uptake of smartphone-based AI phenotyping and the use of time series and ML methods are recommended.

1. Introduction

Food insecurity is a pertinent challenge faced in modern society despite the various initiatives implemented to mitigate the problem. The second United Nations Sustainable Development Goal (SDG) advocates for universal access to safe and nutritious food to end world hunger in all its forms and ensure food security is attained by 2030 [1]. However, the United Nations (UN) also reported that there was a significant increase in the percentage of the world’s population suffering from world hunger by 391 million between 2019 and 2022 to reach 2.4 billion [1]. Subsequently, tackling food insecurity has remained a pervasive problem that has received increased attention from different scientists and researchers who seek more effective solutions to sustainable food production.

Two key challenges are identified to hinder food security and the attainment of UN SDG 2 concerning ending world hunger. First is the low effectiveness of current food production systems where available arable land does not meet the current demand. A study by Meraj et al. [2] revealed the arable land used in food production demonstrated increasing usability deficits following the rise in the population within urban regions. Sharma et al. [3] aligned with [2] and reported that in other cases, water scarcity hindered the production of different crops in arid and semi-arid areas, for example, wheat (Triticium aestivum L.) which relies on the high availability of water. The study by [3] revealed that insufficient water supply to wheat reduces the available yield by closing stomata, decreasing leaf pigments, and reducing the rate of photosynthesis. Chiuyari and Cruvinel [4] also reported that food security was challenged by climate change and economic recessions, hence the need to increase agricultural production by 70% to feed the plant inhabitants by 2050. As such, food production systems are still not able to meet the current demand.

The second challenge to food security pertains to the lack of timely knowledge on crop growth and phenological development in evidence-based decision-making in agriculture. Graf et al. [5] accentuate that the accurate estimation of current and historical growth conditions is important in increasing the efficiency of using resources and inputs, including water and pesticides. However, without such timely knowledge, farmers are likely to use inaccurate inputs at the wrong time, causing low yields from arable land [5]. Zahid et al. [6] add to Graf et al. [5] and posit that precision agriculture utilizing information technology is important in the precise management of crops based on their environmental conditions, requirements for management, and physiological traits. As such, farmers who employ precision agriculture, where real-time monitoring of crops is done coupled with precision adjustments of the production elements, can achieve high yields from arable land. Waqas et al. [7] resonated with [6] where they reported that growing maize (Zea mays L.) in conditions where the temperature was increased by 1 °C above seasonal temperature led to a reduction in crop yield between 3–13% with higher temperatures also deteriorating the quality of the maize grains. Figure 1 illustrates the impact of cold stress on the leaves of maize at the late reproductive phase.

In Figure 1, the impact of cold stress on the late reproductive phase in the growth of maize was showcased.

Cold stress is identified to prolong seedling duration and cell cycles leading to repressed chlorophyll. Additionally, cold stress prolongs the harvest maturity. As such, timely knowledge about the management requirements and environmental conditions through precision agriculture facilitates increasing the yields from crops.

Plant phenotypes describe the externally visible crop traits that provide a comprehensive reflection about the information of crops, such as their growth status and genetic features [5]. As such, plant phenotyping refers to a science associated with agronomy and ecophysiology of plant genomics that facilitates understanding the complex structure of plant traits [1]. Mangalraj and Cho [8] add to [1] and reveal that scientists rely on plant phenotyping to characterize crops used in selecting the best genotype for breeding. In modern agriculture, plant phenotyping is adopted to enhance other applications, including precision agriculture and crop monitoring. Different plant phenotyping methods have also been widely adopted throughout history. Ref. [2] reports that traditionally, plant breeders relied on manual, laborious methods that were also time-consuming. The shortcoming of such processes was that they were not reliable or precise, hence leading to the uptake of automatic methods that are more accurate.

Different scholars have also demonstrated the effectiveness of these plant phenotyping techniques in characterizing plant traits. Pappula-Reddy et al. [9] used a variety of image-based plant phenotyping techniques, including RGB imaging, near-infrared, infrared, and chlorophyll fluorescence imaging, to examine the agro-physiological features of chickpea (Cicer arietinum L.) genotypes under drought stress conditions such as plant phenology, yield components, yield, and physiological parameters. The results revealed a strong positive correlation between manually recorded and image-based traits, hence underlining the importance of embracing image-based screening techniques in breeding methods [9]. Geng et al. [10] also demonstrated the effectiveness of high-throughput phenotyping and deep-learning methods to extract image-based traits of rice where the presence of cloned genes such as TACI, Ghd7.1, Ghd7, and Hd1 was reported, and their effects in different stages of development. A different study by [5] outlined various image phenotyping techniques, such as satellite remote sensing, which were also used to quantify the growth conditions of wheat, where the Green Leaf Area Index (GLAI) and Canopy Chlorophyll Content (CCC) were extracted. Such varied studies [5,10] reveal the wide applications of image phenotyping techniques in plant breeding, where they provide information about the traits of plants, hence improving breeding operations.

The author observes that while many disjointed studies have been carried out to demonstrate the use of image analysis phenotyping of different plant traits, review articles that examine a broad range of studies are not available. The examination of various review articles focusing on the research topics also indicates existing gaps that are addressed in the current review article. For example, Ref. [11] only focused on studies using deep learning convolutional neural networks (CNNs) for plant phenotyping and considered studies published between 2015 and 2020. A different review by [12] was also limited in scope where the focus was summarizing the features, benefits, and shortcomings of optical sensing and data-processing methods applied in different plant phenotyping scenarios. As such, the review article was only directed to guiding the selection of optical sensors and methods of data processing that acquired plant leaf phenotypes cost-effectively and rapidly. A third review article by [13] further focused on UAS-based phenotyping platforms and examined the state of the art in their deployment, collection, curation, storage, and data analysis. The synthesis of the current review articles on plant phenotyping indicates existing gaps in the scope of the methods and the publication period. The core focus of this review article is to investigate the latest developments in image analysis for plant phenotyping using AI, ML, software, and other algorithms by limiting literature from 2020. The article also examines the latest image analysis technology developments, including three-dimensional (3D) approaches. The novelty of this research is that it is the first review study that undertakes a comprehensive approach to examine the current state of the art in using artificial intelligence (AI), machine learning (ML), and other algorithms for image analysis in plant phenotyping from 2020 to date. The exclusion of studies published before 2020 ensures that only current information is reported in the review article.

The objectives of the review article are:

i.

To investigate the latest developments, benefits, limitations, and future directions of image analysis phenotyping technologies based on AI, ML, 3D imaging, and software solutions.

ii.

To investigate the challenges associated with the use of different image analysis phenotyping technologies based on AI, ML, 3D imaging, and software solutions.

The rest of the article is organized into four sections. The methods and materials are showcased in the subsequent section, and results collected from the review of articles are presented. The discussion is then undertaken where the research objectives are reexamined. The conclusion finally showcases the findings from the review article.

2. Materials and Methods

This study employed the systematic literature review method to facilitate data collection. The choice of systematic literature review (SLR) arose from its emphasis on a transparent and reproducible process in reviewing literature to address a specific problem [14]. To undertake the SLR, an explicit and stepwise process was adhered to as showcased in the subsequent section.

2.1. Literature Search

The second phase of the SLR involved the development of a search strategy where parameters guiding the data search were outlined. First, appropriate scientific research databases were selected to facilitate the search for different articles. The databases were preferred as they provided access to different high-quality peer-reviewed articles that were appropriate [15]. The selected database publishers included Science Direct, Scopus, Springer Nature, and MDPI.

Second, keywords were generated from the research question to facilitate the search for relevant articles. The selected keywords included the following: “advanced”, “AI algorithms”, “ML algorithms”, “image analysis”, “plant phenotyping”, “challenges”, and “future directions.” The keywords were further combined using Boolean operators AND/OR to create search phrases. The phrases were the actual queries used in the literature databases and included the following;

• “advanced” AND “AI algorithms” AND “plant phenotyping” AND “challenges”
• “future directions” AND “AI algorithms” AND “image analysis” OR
• “plant phenotyping”
• “challenges” AND “AI algorithms” AND “image analysis” OR
• “plant phenotyping”

The use of the search phrases expanded the scope of the search from the various databases [16]. As a result, a wide range of resourceful articles were identified and examined in this study.

2.2. Study Selection

In the third phase, the literature was selected. The use of keywords and Boolean operators led to the generation of 840 articles from the selected databases. To narrow the search, inclusion and exclusion criteria were defined. The research considered studies published in the last four years (2020–2024) to ensure current and updated information was evaluated in the review article. Additionally, it was important to limit the studies to focus on addressing the formulated research question in the study. The inclusion and exclusion criteria considered in the search are showcased in

Table 1. Inclusion and exclusion criteria.

Focus	Inclusion	Exclusion
Scope	Studies focused on the latest developments, benefits, limitations, and future directions of image analysis phenotyping technologies based on AI, ML, 3D imaging, and software solutions	Studies not focused on algorithms used in plant phenotyping
Period	2020–2024	Before 2020
Language	English	All non-English languages
Design	Primary experimental studies	Secondary reviews
Type	Peer-reviewed journal articles	Grey literature, blogs

below.

The inclusion criteria also focused on primary studies and excluded all secondary reviews and critical review papers to ensure the findings were supported by empirical findings. Preference was also given to articles that were published in English to eliminate the need for third-party translation, which would require more time. The emphasis on full-text articles was also prioritized to ensure comprehensive findings were selected.

The exclusion criteria specified in this study excluded articles published in non-English languages such as Italian, Spanish, German, and Chinese. Studies that were beyond the scope of the research and which did not elaborate on the current state-of-the-art in using AI algorithms for image analysis in plant phenotyping were not considered.

The adoption of the inclusion and exclusion criteria reduced the search results further. During the sorting and filtering processes, 340 duplicate articles were excluded. The remaining 500 articles were further screened to ensure they adhered to the publication period. As a result, 200 articles published before 2020 were excluded, and 300 articles remained. The next phase focused on the scope of the study to facilitate addressing the research question, which led to the elimination of 160 articles that did not cover the current state-of-the-art use of AI algorithms for image analysis in plant phenotyping. In the subsequent process, 140 articles were screened to ensure they were full-text and complete studies. A further 56 articles were eliminated as they only provided abstract and review information. In total, this study reviewed 84 articles.

2.3. Reporting the Findings

The screening process of the articles using the inclusion and exclusion criteria led to a total of 84 articles that met the requirements to address the research question. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart demonstrating the filtration process of the articles is showcased in Figure 2 below.

As showcased in the PRISMA flowchart, this review article considered 84 experimental studies that detailed various AI, ML, and 3D methods in image analysis.

3. Results

3.1. Overview of Methods Used in the Selected Articles

The studies reviewed in the research are showcased in Appendix A.

The evaluation of the selected studies in the review indicated that the quantitative experimental method was employed in all studies. However, individual differences emerged in the state-of-the-art techniques adopted for image analysis in plant phenotyping. The analysis demonstrated that different techniques including deep learning algorithms, unmanned aerial vehicles (UAV) image analysis, high-throughput imaging, hyperspectral imaging, convolutional neural networks, 3D reconstruction using multi-view images, transfer learning, digital imaging, and multiple image sensors. The findings related to the different studies are detailed in the subsequent section.

3.2. Latest Techniques Using AI and ML Algorithms in Plant Phenotyping

The focus of the review article was to examine the latest developments, benefits, limitations, and future directions of image analysis phenotyping technologies based on AI, high throughput analysis (HTP), and 3D image reconstruction. The AI category further includes classifications of unspecified algorithms and software solutions.

3.2.1. AI (ML) Algorithms

Random Forest (RF) algorithms were adopted in plant phenotyping. Ref. [17] employed the RF algorithm to extract crop leaf pixels from multi-source RGB imagery captured using different platforms such as cameras, smartphones, and UAVs. Ref. [18] demonstrated how ML algorithms such as random forests (RF), gradient boosting (GB), and deep neural networks (DNN) could enhance soybean yield prediction for different climate and growth conditions. The noteworthy finding from [18] was the highlight of classification machine learning algorithms and a transfer learning approach in genotype selection and the categorization of soybean yield for screening high-yield varieties. As such, Ref. [18] argued that the adoption of classification ML techniques was the most useful in crop breeding applications due to their effectiveness in identifying high-yielding genotypes and improving yield prediction. Ref. [19] also used RFs in the hyper-spectral imaging (HIS) pipeline to detect chlorophyll distribution hence enhancing crop management strategies.

The evaluation of the results also indicated that other classification algorithms, such as support vector machines (SVMs), were also essential in different phenotyping tasks, such as crop counting [18]. The research by [18] reported that problems were faced in counting corn plants (Zea mays L.) using manual methods, especially when large tracts of land were considered. To address the problem, multispectral images were acquired using embedded cameras in unmanned aerial vehicles (drones), and the dataset was processed using digital imaging processing techniques. The use of SVMs was also identified to facilitate the classification of multispectral images, where highly accurate results, higher than 84%, were reported when counting maize plants [18]. The studies indicated that classification image analysis methods using SVMs were applied in large-scale farms to undertake crop counting and enhance crop yield based on different climates and growth conditions. Ref. [20] employed SVMs coupled with Recursive Feature Elimination (RFE) and Boruta algorithms for high light tress phenotyping of tomato genotypes using chlorophyll fluorescence and reported high reliability. Ref. [21] reiterated [20] and used a combination of SVMs, CNN, and RF algorithms in UAV-based imaging to estimate plant height. The results indicated that the techniques were effective in monitoring plant features such as height, yield, and biomass. Ref. [22] demonstrated the use of regression trees and ANNs in improving crop yield by correlating hyperspectral data with photosynthetic parameters hence leading to sustainable crop development. In Ref. [23], the AGSS algorithm was used in detecting root length and diameter where a 90% higher accuracy was reported hence enhancing the repair of root images in small embedded systems. In Ref. [24], the DFU-Net model was adopted in the automated detection of seed melons and cotton plants hence providing valuable technical support for intelligent crop management at 92% accuracy.

The CNN algorithm was adopted in [25] plant phenotyping to obtain maize seedling emergence rate and speed of leaf emergence where a 99.53% accuracy was reported. Ref. [26] added to [25] and demonstrated the effectiveness of R-CNN in the accurate reconstruction of fruits and branches in guava plants. In Ref. [27], CNN was combined with a 3D instance segmentation algorithm to estimate apple counting and volume segmentation to extract traits from orchards. Ref. [28] reiterated [27] and revealed that the R-CNN algorithm generated a high accuracy of 97.3% in determining fruit numbers. A similar case was [29], where non-destructive measurement phenotype technology was adopted in watermelon seedling breeding, management, and quality breeding. Ref. [29] showed that the Mask R-CNN algorithm delivered good measurement performance for the watermelon plug seedlings from the one true-leaf to 3 true-leaf stages. Ref. [30] added to [29], where the use of deep convolutional neural networks (CNN) was identified to enhance the high-spatiotemporal resolution of UAV imagery to characterize the effects of genetics and the environment on key traits of the miscanthus crops in [31], the CNN algorithm was adopted with 3D imaging to enhance crop breeding and precision agriculture capabilities. Ref. [30] added to [29] and revealed the use of 3D-CNN in the diagnosis of yellow rust infection in spring wheat.

In Figure 3, deep learning algorithms were showcased in classifying large flowers, including canola and peas, where rectangles identified the noise, whereas ovals represented missed flowers.

3.2.2. Unspecified AI Algorithms

The review of different studies indicated that unspecified deep and machine learning algorithms were adopted in undertaking image analysis and plant phenotyping. The first image analysis plant phenotyping approach regarded the use of deep and machine learning algorithms, as demonstrated in numerous studies. The evaluation of the studies was undertaken to identify how various techniques were employed to derive plant traits in unique phenotyping contexts. The results underlined the importance of deep learning in monitoring and managing plant growth, where issues in different growth cycles were addressed [18]. The study by [33] developed a predictive model for lettuce growth and also trained it to improve the accuracy of the lettuce growth images and phenotypic indices. The insight from [18] showed high accuracy in predicting lettuce phenotypic indices with errors of less than 0.55% for geometric indices and less than 1.7% for color and texture indices. The role of deep learning in predicting the various issues that would be faced in each growth phase of the lettuce life cycle was reported to enhance quality and efficient lettuce production [18]. The insights also indicated that deep learning algorithms could enhance crop sustainability by enabling scientists to address issues faced in different phases of growth, hence promoting higher productivity. Ref. [34] also demonstrated the use of successive projections algorithm (SPA) classification algorithms in classifying tea plant stresses with a 98% accuracy. The uniqueness of [34] was the adopted algorithm where the continuous wavelet projections algorithm (CWPA) was developed using the continuous wavelet analysis (CWA) and SPA to generate an optimal spectral feature set for crop detection. In Ref. [35], the Otsu algorithm was used to accurately obtain the target region of maize stems hence demonstrating feasibility in the assessment of field conditions.

In Ref. [36], VGG16, VGG19, and ResNet-50 were used in classifying different types of weeds including deep, corn, tomato, and cotton weeds leading to significant performance improvement. In Ref. [37], Generative Adversarial Network (GAN) and U-Net were used in segmenting plant regions and biomass areas to detect emergence counting and estimate crop biomass leading to high yield. Ref. [38] further combined U-Net with DeepLabv3+, PSPNet, and HRNet networks to segment soybean images hence improving the reconstruction of robustness. The low-light image-based crop and weed segmentation network (LCW-Net) was used in segmenting crops and weeds in sugar beet, carrot weeds, grass weeds, and dicot weeds. Ref. [26] used PointNet and YoloV5 where they demonstrated the use of deep-learning regression techniques for predicting grain counts in sorghum panicles. The study combined RGB images and the deep learning regression framework for point clouds where 147 sorghum panicles were considered. The results obtained showed a mean absolute error with a percentage of 6.5% and 6.8% in low-resolution point cloud datasets [26].

Further work in [28] demonstrated clustering algorithms developed for corn population point clouds where an accurate extraction of the 3D morphology of individual crops was done. The study by [28] applied a similar approach as [18,26], where multispectral image data was first obtained using drones, and thereafter, corn plant models were generated using SfM algorithms. The subsequent phase involved simplifying the point clouds and extracting crop point clouds using color and distance thresholds. The use of clustering algorithms demonstrated the segmentation of corn point cloud data, hence segmenting the individual corn plants in the field and providing an efficient and low-cost solution to accurately measure information related to crop phenotypes. The results from [28] revealed an accuracy of 93.1% in the segmentation of corn plants, hence providing a cost-efficient, automated, and low-cost solution to measure the phenotypic information in corn accurately.

A further application was identified by [32], where C-RGB, MS1, and MS2 ML classification algorithms were adopted in gathering flowering information to facilitate escaping abiotic stresses such as heat and drought. In their study, ref. [32] used multispectral imaging sensors such as RGB and multispectral cameras, as well as both supervised and unsupervised techniques, to monitor the flowering intensity of canola, chickpea, pea, and camelina. The results from [32] showed that the features extracted from RGB images from drones were able to accurately detect and quantify large flowers of the spring canola, winter canola, and pea, while they were unable to process the data for the chickpea and camelina. The findings indicate both limitations in extracting flower features using classification algorithms where only some type of data is suitable. Figure 3 showcases the use of clustering algorithms to detect winter canola and pea flowers.

The findings from studies indicated that using classification and deep learning algorithms facilitated addressing different laborious tasks such as manual counting of maize crops, grain counting from sorghum panicles, classification of tea plant stress, and prediction of best climates and conditions of crop growth. The results underscored the relevance of ML and deep learning in applications such as crop yield prediction, estimating plant density, monitoring the growth of crops such as lettuce throughout their life cycles, and measuring the performance of miscanthus perennial crops.

In addition to the traditional ML and deep learning algorithms adopted for plant phenotyping, the review also examined emerging and recently published alternatives. In Ref. [32], the use of high-resolution and time series data from field soybean populations was obtained using UAVs where the red, green, and blue (RGB) infrared images were used as inputs to construct the multimodal image segmentation model. The results of the developed RGB and infrared feature fusion segmentation network (RIFSeg-Net) indicated higher performance than traditional deep learning segmentation networks at a precision of 0.94 and an F1-score of 0.93. As such, the proposed method generated high practical value in resource identification in germplasm and could lead to a practical tool to facilitate genotypic differentiation analysis and select target genes. The findings by [32] were reiterated in a study by [39] where multi-source, time-series, and deep learning methods were used in the classification of poplar seed varieties and drought stress. The obtained results showed that the LSTM method adopted had a high classification accuracy of 96.56% while the ResNet18-LSTM and ResNet18-CBAM-LSTM generated accuracies of 98.12% and 99.69% [39]. The findings indicated that combining time series and deep learning methods improved the classification performance when monitoring multiple varieties of poplar seedlings under different drought conditions.

In further studies, the use of smartphone-based applications for plant phenotyping was also identified. In Ref. [40], a geographic-scale monitoring approach for counting coffee cherries was developed using an AI-powered citizen science approach. The method used basic smartphones to capture images of coffee trees and a model was trained and validated using the YOLO (You Only Look Once) approach to detect cherries. The results in [40] showed an R² of 0.59 in Peru and 0.71 in Colombia where coffee was grown under different biogeoclimatic conditions. The findings indicated that the smartphone-based method could be applied on broader scales and could be transferred to different regions and countries. The insights indicated the potential for photo-based phenotypic monitoring using smartphones and was suitable in low-income regions. A different study by [41] supported [40] and revealed the use of smartphone-based whole-plant and on-device phenotyping Android applications that could assess 25 leaf traits, 5 stem traits, and 15 plant traits from images. The study used a DeepLabV3+ model for image segmentation and revealed that maize phenotyping could be measured in real time to benefit crop breeding in the future. Another study by [42] Rockel reiterated [35] where an open-source Android app, PhenoApp, was developed to guide digital recording of phenotypical data in greenhouses and in the field. The findings from [42] indicated that using PhenoApp led to customizing descriptors and also improved the efficiency of digital data acquisition in genebank management hence contributing to breeding research. The insights from the studies indicated that using smartphones for plant phenotyping traits was a promising future trend.

3.2.3. AI Algorithms (Software)

The AI analysis also involved the use of software applications where 2D image reconstruction was undertaken in deep learning algorithms. The synthesis of the studies indicated that applications were based on ML and high-throughput phenotyping methods.

In Ref. [43], vision software known as AIseed was developed based on machine learning to automatically capture plant traits such as shape, texture, and color of different seeds, hence guiding the assessment of seed quality to improve breeding and yield. Further inspection of [43] indicated that the motivation for adopting AIseed software was the need to assess the quality of seeds on a large scale basis automatically, where 54 seed features were evaluated to enhance seed quality detection and prediction. Ref. [43] demonstrated that the AIseed vision software was associated with numerous advantages, including its non-destructive nature, high performance in the extraction of phenotypic features, and high speed and precision levels in performing analysis. Figure 4 showcases the AIseed software interface used in the analysis of seed phenotyping traits.

In Figure 5, the original and processed seed images using AIseed software were illustrated. The software generated satisfactory results that facilitate plant phenotyping traits.

In a further study, Ref. [44] added to [43], where they demonstrated that software systems focused on high-throughput phenotyping (HTP) techniques could also be developed based on computer vision algorithms. Ref. [44] developed Cottonsense, an advanced HTP system to address the challenges of deployment across different growth phases of cotton crops, including the square, flower, closed boll, and open boll, yielding a 79% accuracy score in segmenting different fruit categories. The similarity between [43,44] was the use of computer vision algorithms in developing software to automatically assess plant traits. The studies also showed that software was an effective tool in plant phenotyping, especially in large-scale operations. In Ref. [27] the seedscreener software was used in phenotyping seeds automatically leading to a high accuracy rate of 94%.

3.2.4. HTP Analysis

Further analysis also showed that researchers relied on high-throughput image analysis methods for plant phenotyping. The analysis showed that most studies employed diverse image analysis phenotyping models. In Ref. [45], a high-throughput field phenotyping method using a process-based biophysical model was adopted in the detection of early vigor of 231 cereal oat lines in a commercial breeding program. The results in [45] showed that the phenotyping method was important in identifying multi-metric evaluations of spatial and temporal variability in the cereals, hence improving crop breeding applications. Ref. [46] used NDVI where aerial phenotyping techniques were adopted in enhancing sugarcane breeding to identify genotypes with superior yield traits in different watering conditions. Ref. [20] added to [46] and showed that 3D and NDVI techniques had a high utility in evaluating the lifespan of pepper crops. Further study showed that other high-throughput imaging methods adopted included spatial aggregation approaches [47]. In the study, Ref. [47] demonstrated that spatial aggregation approaches were effective in airborne measurement of sun-induced chlorophyll fluorescence (SIF) to monitor different plant functions at different scales.

In Ref. [48], it was shown that low-light image-based crop and weed segmentation networks were accurate in late-time harvesting applications and outperformed other state-of-the-art methods. In a further study, a large-scale-cable-suspended field phenotyping system was proposed to measure bidirectional reflectance distribution [49]. The result in [49] showed that hot spots were visible in the backscatter direction at visible and near-infrared bands at the largest sensor zenith angle. Ref. [50] demonstrated the high accuracy of image-based semi-automatic root phenotyping to examine root morphology and branching features.

The results in [50] revealed that interspecific advantages for maize occurred within 5 cm from the root base in the nodal roots, and soybean inhibition occurred within 20 cm from the root base. Ref. [32] also demonstrated the effectiveness of a multimodal image segmentation model comprising RGB and infrared feature fusion segmentation network (RIFSeg-Net) in the assessment of the rate of soybean canopy development on a large scale, which was laborious and time-consuming. The application of the various image analysis phenotyping models facilitated diverse applications, including monitoring soybean canopies for different genotypes, root phenotyping of field-grown crops, and late-time harvesting of crops and weeds.

In Ref. [51], the CIAT Pheno-I image analysis framework was used to extract plot-level vegetation canopy metrics of cassava-based on its effectiveness and speed. In Refs. [26,27], CCO imaging techniques were used to obtain canopy features of different types of crops from UAV images. In Ref. [38], the watershed algorithm was proposed to perform bean image segmentation where it generated higher performance.

3.2.5. HSI Analysis

The reviewed studies further showcased the use of hyperspectral imaging (HSI) techniques in plant phenotyping. In Ref. [52], the influence of fractional vegetation cover (FVC) on digital terrain model (DTM) reconstruction accuracy was investigated. The study considered UAV imagery to examine plant height features and reported that the accuracy of DTM constructed using an inverse distance weighted algorithm was influenced by fractional vegetation cover (FVC) conditions. The study by [53] reiterated the findings by [52] where UAV imagery obtained at three growth stages using complete and incomplete data was used to compare four rapeseed height estimation methods. The results showed that where complete data was used, optimal results were obtained with an R² of 0.932 and a root mean square (RMSE) of 0.026 m. Further work by [47] developed and tested two spatial aggregation approaches to make airborne sun-induced chlorophyll fluorescence (SIF) data usable in experimental settings when monitoring plant functioning at different scales. The first spatial aggregation approach considered involved aggregating pixel values directly on SIF maps while the second aggregated at-sensor radiance before retrieval of SIF data. The results showed that the second approach generated a better representation of ground truth, R² = 0.61, compared to the first approach, R² = 0.55, when combined with robust outlier detection and weighted averaging [54]. Bhadra et al. [55] showed that UAV-based spectral sensors and advanced ML models could be adapted to accurately estimate leaf chlorophyll concentrations (LCC) and average leaf angle (ALA), hence bridging crop breeding and precision agriculture. The work by [39] demonstrated the effectiveness of a transfer learning-based dual stream neural network PROSAIL-Net in estimating LCC from the UAV images with a high accuracy of R² 0.66 and 0.57 for the LCC and ALA, respectively. Figure 4 illustrates the data collection scenarios using the UAV image setup adopted in the study.

In Figure 4, the data collection setup using UAV images for chlorophyll concentration estimation was showcased.

Further analysis also indicated that combining ML algorithms with high throughput phenotyping image analysis techniques to undertake plant phenotyping led to higher performance. In Ref. [18], the use of multispectral image analysis for maize plant counting and assessment was showcased. The dataset of spectral images was gathered from flights over an agricultural area and processing of orthomosaics was undertaken in spectral channels of red, green, and blue. Further classification of the image data was undertaken using SVM classifiers. The study demonstrated that combining the multispectral image analysis method with the SVM technique led to an accurate and timely counting methodology for maize plants that guided cultivation to ensure high yield was showcased with an accuracy rate of 88.47% [18]. In the study by Zhu et al. [19], close-range hyperspectral imaging (HSI) combined with ML techniques such as random forests (RF) and Savitzky-Golay-standard normal variate (SG-SNV) was adopted to quantify the chlorophyll distribution of basil crops. The insights from [19] indicated that canopy HSI methods presented challenges due to the complex interaction of illumination with the canopy geometry. Combining HSI with the ML techniques led to satisfactory chlorophyll distribution maps consistent with the observed levels of chlorophyll, hence being applicable in the timely monitoring of the chlorophyll status of whole canopies. The findings from [51] also reiterated [19] where they demonstrated the use of multispectral imagery and ML techniques to accurately predict cassava traits. The insights indicated that coupling the two methods could facilitate the extraction of phenotypic information rapidly. The insights from the analysis of the case studies [19,55] indicated that ML techniques were adopted at the image-processing phase to overcome the limitations associated with the hyperspectral imaging techniques. The findings from [51] supported [32], where they demonstrated that combining ML approaches to predict plant traits with multispectral imagery using unmanned aerial vehicles was effective in the measurement of canopy metrics and vegetation indices traits of cassava at different time points during the growth cycle. The evaluation of the studies also highlighted various limitations of the hyperspectral imaging techniques. The limitations included subjectivity in their application [32] and complicated interaction [19]. However, the analysis of the studies revealed that high throughput phenotyping techniques would be combined with ML algorithms to generate higher accuracy in identifying plant traits. The findings suggested that in future applications, more combinations of different high-throughput phenotyping techniques with deep learning and ML algorithms would be expected.

3.2.6. 3D Image Reconstruction

The final application area involved 3D image reconstruction, as demonstrated in diversified studies. The evaluation of these studies showed that relying on 3D reconstruction techniques ensured fast and accurate phenotyping of plant traits by setting the scene of the agricultural land. The results also indicated that 3D image reconstruction was feasible across diverse plant phenotyping applications. Ref. [56] demonstrated a stereo-vision-based system based on the mobile platform to estimate the height of plants. In their research, Ref. [56] argued that computer vision-based systems overcame the manual monitoring approaches of crops that were time-consuming and labor-intensive. In Ref. [20], point-cloud prepossessing coupled with different techniques including mesh generation and optimization of feature-point positions to achieve high accuracy in the reconstruction of maize leaves at high consistency. In Ref. [28], X-ray MicroCT was used in measuring the internal traits of bean seeds to establish their morphological traits. In Ref. [26], neural radiance fields (NeRF) was coupled with Instant-NGP and Instant NSR where insights showed that high reconstruction results were obtained that were comparable to reality capture software. In Ref. [57], a system was developed using Visual SfM and Agisoft Photoscan to map 3D image reconstruction of red pepper plants (Capsicum annuum L.) and undertake automatic analysis. The study used a Kinetic v2 with a depth sensor and high-resolution RGB camera to obtain accurate reconstructed 3D images that were compared with conventional images in aspects such as plant height, leaf number, and width. The results in [57] showed that an error of 5 mm or less was identified when analyzing 3D images, hence its suitability in plant phenotypic analysis. Ref. [58] added to [57] and reported that a 3D reconstruction method based on SfM combined with MVS achieved higher accuracy of 43.3% in wheat phenotyping. Ref. [38] also combined MVS and SfM and reported that 3D reconstruction of dummy and root systems was successful. Ref. [58] supported [38] and revealed high accuracy using SfM in phenotyping vegetables by recording video clips using smartphones. Ref. [27] also adopted SfM to automatically detect and measure plant height, petioles inclination, and single-leaf area. Ref. [26] added to [57], where they showed that 3D reconstruction techniques were important in the development of a garden robot using FCSN that navigated towards rose bushes and clipped them guided by pruning rules. In Ref. [27], 3D reconstruction was adopted to recover the morphology of plants and segment branches, hence leading to a highly accurate method for pruning rose plants.

Ref. [51] supported [26], where they demonstrated the use of multi-vision technology in fruit picking in an orchard, and 3D triangulation techniques identified the positions of fruit clusters and avoided obstacles for the robots. The insights from these studies showed that 3D reconstruction enabled robots to perceive different plant traits, hence allowing them to undertake pruning and fruit harvesting activities with high accuracy. Ref. [28] also adopted 3D reconstruction to create a large-scale agricultural orchard environment, hence facilitating the navigation of robots in the scene. The insights of these findings indicated that 3D reconstruction allowed agricultural robots to undertake pruning and harvesting activities more accurately as they distinguished branches from fruits. In Ref. [28], 3D reconstruction applications were identified as applicable in phenotyping root activity where roots with a diameter higher than 1 cm had higher length similarity between the algorithm reconstructed root system architecture and the simulated RSA at 76.2%. Ref. [27] also reiterated [28], where they showed that 3D reconstruction techniques were cost-effective and cost-efficient in phenotyping water use in rooting systems for sorghum crops. The insights indicate the multiple application areas where 3D reconstruction phenotyping methods could be adopted in plant phenotyping, including root activity, pruning, and harvesting fruits.

In Ref. [26], NDT, intrinsic shape signatures (ISS), and iterative closest point (ICP) algorithms were used in phenotyping tomato canopies using 3D reconstruction where a high correlation was reported. In Ref. [27], an ICP-based 3D registration algorithm for eye-in-hand configuration was adopted to ensure real-time IC-based 3D reconstruction of different types of food items including apples, pears, and bananas. Ref. [59] also used ICP and reported a 93.42% accuracy in the reconstruction of peanut plants hence identifying their plant traits. In Ref. [28], the root topological structure and geometric features reconstruction algorithm was used in the 3D reconstruction of the root system architecture revealing high feasibility and effectiveness.

In Ref. [20], the MTLS image analysis using 3D characterization was used revealing a high correlation with other methods. In Ref. [28], eye-in-hand stereo vision and SLAM systems were used in the construction of a global map to undertake a phenotyping analysis of passion, litchi, and pineapple. Ref. [26] also used SLAM in the reconstruction of a semantic map of citrus orchards by combining the BiSeNetV1 algorithm. In Ref. [28], intrinsic shape signatures-coherent point drift (ISS-CPD) and distance field-based segmentation (DFSP) were adopted to demonstrate a high accuracy of 79.99% in the reconstruction of the soybean canopies.

The use of the motion algorithm in constructing the 3D morphology of maize plants hence extracting phenotyping parameters was also demonstrated [35]. Additionally, throughput 3D phenotyping technology was adopted to understand the association between the canopy architecture and the light environment of maize and soybean [58]. In Ref. [38], cylinder fitting and disk stacking were adopted in extracting 3D point cloud data from UAV images for sorghum panicles. In Ref. [35], the logistic growth model was demonstrated to extract phenotypic traits of soybean plants to identify patterns in phenotypic changes. In Ref. [38], the PCL and computational geometry algorithms (CGAL) were used at high accuracy to measure the mapping of the plant’s environment hence providing higher crop efficiency.

Further work revealed that in other instances, the 3D reconstruction techniques were combined with deep learning methods, as reported in [59]. In Ref. [59], machine vision-based fruit grading systems were adopted where a 96.8% accuracy was reported in measuring mango fruit structures. The insights in [20] showcased that adopting a combination of deep learning and 3D reconstruction led to higher accuracy in complex applications, including detecting branches within orchards where SPGNet and DBSCAN were used. A similarity was identified between [20,26], where it was shown that 3D techniques and mask R-CNN algorithms were important in detecting guava branches and fruits, which were represented using 3D components. As a result, advantages such as obstacle avoidance paths could be generated for harvesting robots. In Ref. [27], PointNett++ was used in combination with 3D modeling to assess the effectiveness of 3D data in measuring cucumber plants with curved growth patterns. In Ref. [20], the U2-Net algorithm was combined with 3D reconstruction to assess the minimum average track length and minimum reprojection error for carex cabbage and kale. In Ref. [27], the probabilistic voxel carving algorithm was combined with 3D reconstruction to extract plant traits automatically from maize. In Ref. [27], the stereo algorithm was combined with GwCNet to quickly and accurately obtain in-depth information about oranges hence suitable in commercial fruit sorting lines.

4. Discussion

In addressing the first objective, the examination of the reviewed studies indicated four categories of emerging developments in image analysis phenotyping. The examination of the findings showcased that a popular approach regarded the use of machine and deep learning techniques in phenotyping different plant traits. Studies such as [18] used predictive deep learning to monitor the different growth cycles of lettuce and identify existing issues that emerged. Close inspection of [18] revealed a similarity to [33], where deep learning methods such as random forests and deep neural networks were adopted in predicting the yield of soybeans under different climate and growth conditions. Further analysis revealed that deep learning approaches were important in undertaking manual and laborious tasks on large-scale farms. A case example was counting maize crops in large farms [18] and grain counts from sorghum panicles [26]. Segmentation of plant traits was a second application area where deep and machine learning algorithms were widely adopted from the reviewed studies. Ref. [28] showcased the use of clustering algorithms in segmenting phenotyping information from corn plants while [32] used ML techniques in extracting flowering information and abiotic stresses, including drought and heat. Ref. [34] also demonstrated that ML algorithms were effective in classifying tea plant stresses at a 98% accuracy. A further application area of plant phenotyping using ML and deep learning algorithms was plant breeding and management, where [59] used them to measure the performance of plug watermelon seedlings.

The discussion further delved into the use of high-throughput image phenotyping techniques in extracting plant traits. The difference between the high throughput techniques and deep learning reviewed earlier regarded the methods adopted in the extraction of plant traits. Ref. [45] demonstrated high-throughput field phenotyping techniques used in multi-metric assessment of the spatial and temporal variability of cereals, hence improving crop breeding applications. Further work in [46] highlighted the efficacy of aerial phenotyping where normalized difference vegetation index (NDVI) was used in identifying whether genotypes were high- or low-yielding based on the canopy temperatures. The discussions also revealed the effectiveness of spatial aggregation techniques in measuring sun-induced chlorophyll fluorescence using airborne mechanisms [47]. Additionally, the techniques were used in roots where [50] demonstrated image-based semi-automatic root phenotyping in the examination of root morphology and branching features.

The discussions also observed that, in some instances, researchers were keen on leveraging the advantages of both deep and machine learning combined with high throughput imaging techniques. The inspection of the results indicated multiple case studies where high throughput imaging techniques were used to extract imaging details, whereas deep learning methods were adopted to classify plant traits. The close inspection of these studies indicated that combining high throughput imaging techniques and deep learning led to higher accuracy in detecting different plant traits and enhancing phenotyping applications.

The final section in the results discussed the importance of using software—3D reconstruction and algorithm-based alternatives—in plant phenotyping. The discussion of the results revealed that different algorithm-based software was available, including AIseed, used in [43] to automatically capture the traits of seeds, including shape and texture. A similar observation was made in [44], where Cottonsense was adopted as a high-throughput software system used in segmenting different plant traits. A vision-based system was showcased in [56] to estimate plant height, hence eliminating the need for intense manual labor.

The second class of software showcased the use of 3D applications to reconstruct different aspects of the plants, such as roots, fruits, and branches, and a combination of deep learning, machine learning, and 3D image reconstruction. The synthesis of these studies reveals that relying on multiple techniques led to higher accuracy in enhancing precision agriculture applications. Ref. [28] demonstrated 3D image reconstruction to scan the roots of a 9-year-old ash tree, extracting the spatial coordinates of the root points and estimating the root point diameter. As such, farmers could leverage the advantages of 3D reconstruction to implement robotics within agricultural farms, such as fruit harvesting robots that can accurately detect fruits and separate them from branches.

To further enhance the performance of 3D reconstruction tools, researchers also incorporated deep and machine learning methods. In Ref. [59], a 3D reconstruction and image processing ML algorithm was adopted to estimate the volume and shape of mango fruits, revealing a 96.8% accuracy compared to other methods that generated 91.7% and 91.5% accuracy. Ref. [28] added to [59] and demonstrated the 3D reconstruction of blueberries as they developed in clusters, hence extracting their cluster traits. The 3D reconstruction generated images of the blueberries, whereas a mask R-CNN was adopted in segmenting individual blueberries at high accuracy, hence facilitating development monitoring, yield estimation, and prediction of harvest times [28]. The studies revealed that researchers could leverage the benefits of both 3D reconstruction and ML and deep learning algorithms to enhance plant phenotyping. The cost-effectiveness and cost-efficiency of the methods also emerged as advantages arising from the adoption of the latest image analysis phenotyping techniques. The findings aligned with the previous literature where studies, such as [5,10,11], had underscored the effectiveness of plant phenotyping techniques in understanding plant traits to improve yield and breeding. Further advantages also arose in real-time monitoring of the plant growth to improve breeding applications, for instance, how plants responded to environmental factors such as water stress. The findings relating the use of deep and machine learning algorithms in enhancing plant phenotyping applications and image analysis were also aligned with [35], where smart agricultural engineering, such as farm machinery optimization, was enhanced by using bio-inspired algorithms such as ant colony and firefly algorithms. The insights indicate the increasing applications of state-of-the-art technologies in improving smart agriculture to promote sustainability as showcased in AI used in smart greenhouses [58,60]. In a further study, Ref. [61] also demonstrated the efficacy of an image processing methodology that combined deep learning and high-resolution imaging to separate disease and senescence in wheat canopies. The findings showed that tracking the necrotic and chlorotic fractions in leaves enabled separate quantification of the contributions of physiological senescence and biotic stress on the overall green leaf area dynamic and investigating the interactions between biotic stress and physiological senescence. As a result, combining high-resolution imaging and deep learning further facilitated genetic studies of disease resistance and tolerance.

The discussion further addressed the second research objective, where the limitations and weaknesses of advanced imaging methods of plant phenotyping, such as deep learning and 3D image reconstruction, were identified. A key limitation was that the deep learning algorithms were only applied to specific crops, hence lowering the generalizability of the findings compared to other crops. Further synthesis of the results showed that the implementation of the latest techniques involved complex processes and the interaction of different systems to identify plant traits. For example, mobile systems used in collecting images were combined with ML algorithms for data analysis and processing, and 3D structures to map the agricultural environment to facilitate extracting plant phenotyping traits. The complexities involved in the individual studies lower the generalizability of the techniques in future applications. As such, future work should focus on the development of off-the-shelf 3D reconstruction and image analysis software that can be adopted in agricultural settings to generate plant phenotyping data. Additionally, future research ought to focus on identifying multiple case studies and diverse crops where the algorithms can be adopted to enhance crop monitoring.

5. Conclusions

The overarching aim of this research was to investigate the latest developments in image analysis for plant phenotyping using AI, ML, software, and other algorithms by limiting literature from 2020. The evaluation of the results from different quantitative experimental studies revealed that significant advancements have been made in image analysis for plant phenotyping, where different technologies are employed to generate plant traits. The synthesis of the findings showed that machine and deep learning methods were effective in diverse phenotyping applications, such as predicting issues expected during the growth cycles of lettuce plants, predicting yields of soybeans in different climate and growth conditions, and identifying high-yielding genotypes to improve yields. The predictive abilities of deep and machine learning have been harnessed in plant phenotyping applications where predictions of yields are showcased.

Furthermore, the insights indicate that farmers can rely on deep and machine learning to ease laborious tasks such as crop and grain counting on large-scale farms using classification algorithms such as SVMs. The adoption of high throughput image analysis techniques is also emphasized as a relevant technique in detecting plant traits to facilitate breeding. Hyperspectral imaging (HSI), UAV image analysis, and spatial aggregation techniques were used to monitor plant functions. The combination of high throughput image analysis with both machine and deep learning has also enhanced plant phenotyping in applications such as leaf chlorophyll concentration estimation. Finally, the research has demonstrated an increasing trend to use 3D reconstruction and software technologies for plant phenotyping. The reviewed software is vision-based and processes seed information to enhance the prediction and detection of seed quality. 3D reconstruction techniques have also been adopted to identify plant traits accurately and in applications involving automated robotic harvesting.

Therefore, the review article underscores that significant development has occurred in the technology space with novel variations of deep learning, ML, 3D reconstruction, and software being integrated into plant phenotyping. Emerging insights also highlight the adoption of smartphone-based plant phenotyping and combining ML and time series data to generate higher performance.

However, a few limitations have been highlighted, which are potential areas for future research. A limitation was that the studies focused on specific crops, which limited the generalizability of the findings across different settings. For example, questions emerged regarding the applicability of deep learning algorithms in classifying other crop traits based on the effectiveness observed with plants such as maize and sorghum. A second limitation was the complexities involved in the implementation of the different technologies, such as 3D reconstruction coupled with deep learning.

To encourage more farmers to adopt these technologies in crop breeding and yield estimation, further work should focus on easing their integration within small-scale farms. The development of off-the-shelf software to estimate yield and count crops based on collected UAV images is a potential research avenue to ease their use in farms. Future directions in the area are expected to involve advancements in developing hybrid solutions that combine these strategies. For example, the development of vision-based software that provides AI functionality to undertake autonomous plant phenotyping in large-scale farms to predict issues in crops at different growth stages.