Research on Morphological Indicator Extraction Method of Pinus massoniana Lamb. Based on 3D Reconstruction

Abstract

Pinus massoniana (Lamb.) is an important plantation species in southern China. Accurate measurement of P. massoniana seedling morphological indicators is crucial for accelerating seedling quality assessment. Machine vision, with its objectivity and stability, can replace human eyes in performing these measurements. In this paper, a measurement method for seedling morphological indicators based on Euclidean distance, Laplacian contraction, PointNet++, and 3D reconstruction is proposed. Firstly, multi-angle sequence images of 30 one-year-old P. massoniana seedlings were collected, distorted, and corrected to generate a sparse point cloud through the Structure-from-Motion (SFM) and dense point cloud through the Patch-Based Multiple View Stereo (PMVS). Secondly, a Dense Weighted Semantic Segmentation Model based on PointNet++ was designed, achieving effective segmentation of the P. massoniana seedling point clouds. Finally, a multi-iteration plane method based on Laplacian contraction was proposed. The new skeleton points were refined by minimizing the Euclidean distance, iteratively generating the optimal morphological skeleton, thus facilitating the extraction of morphological indicators. The experimental results demonstrated a good correlation between the machine vision-extracted morphological indicators (including plant height, ground diameter, and height-to-diameter ratio) and manually measured data. The improved PointNet++ model achieved an accuracy of 0.9448 on the training set. The accuracy and Mean Intersection over Union (MIoU) of the test set reached 0.9430 and 0.7872, respectively. These findings can provide reliable technical references for the accurate assessment of P. massoniana seedling quality and the promotion of digital forestry construction.

1. Introduction

Forests serve as the cradle of human civilization, and artificial forests, as an important component of China’s forest resources, play an increasingly prominent role in maintaining global carbon balance and mitigating global warming [1]. In a series of ecological projects implemented in China, such as the Grain for Green Program, Pinus massoniana (Lamb.), as a crucial pioneer afforestation species in barren mountainous areas, has significantly improved the low forest coverage and scarcity of forest resources [2,3]. P. massoniana, known for its preference for sunlight, drought tolerance, strong adaptability, rapid growth, and high economic value, is widely distributed in China’s subtropical regions, making significant contributions to the development of artificial forests [4].

High-quality and robust seedlings serve as the material foundation and prerequisite for the sustainable development of plantation forestry. The quality of P. massoniana seedlings directly affects the success of afforestation, operational costs, and economic benefits. Therefore, employing scientifically sound methods for evaluating seedling quality is a crucial mean of ensuring the production of superior seedlings [5,6].

Currently, the main indicators used to evaluate the quality of nursery seedlings and establish grading standards are morphological indicators. These indicators primarily include seedling height, ground diameter, height-to-diameter ratio, root system morphology, stem–root ratio, and apical bud, among other comprehensive quality indices. These indicators are closely related to the growth status of the seedlings [7,8]. Traditional measurements of seedling height and ground diameter rely heavily on tools such as vernier calipers and rulers, which are time-consuming, inefficient, and prone to significant subjective errors in manual measurements [9].

To enhance the accuracy of morphological indicator measurements, researchers have integrated computer technology, image processing techniques, machine vision, and Deep Learning (DL) techniques into the extraction of seedling morphological indicators [10,11,12,13,14]. McGuinness et al. [15] proposed three algorithms that effectively and accurately measure the morphological features of the stem, height, and root system of radiata pine seedlings in New Zealand. They also developed a machine vision system for the sorting of radiata pine seedlings. The study employed denoising and segmentation techniques for preprocessing the acquired images and introduced an algorithm for measuring the feature dimensions of the images. The results demonstrated a strong correlation between the measured data of seedling features obtained by the machine vision system and the actual values. Suo Rui et al. [16] proposed a transfer learning approach for apple seedling segmentation based on Blend Mask with ResNet-101. The novel labeling strategy employed in this study contributed to improved accuracy in image segmentation, enabling precise measurement of apple seedling morphological indicators. Hao et al. [17] applied Mask Region Convolutional Neural Networks (Mask R-CNNs) and Residual Neural Network-50 (ResNet-50) to segment Chinese fir, achieving an F1 score of 84.7% and an IoU score of 91.3%.

Accurate and robust plant segmentation based on DL is a key technique in machine vision for measuring morphological indicators [18,19]. Compared to traditional plant segmentation methods, DL approaches exhibit significant advantages in various aspects, including speed and accuracy [20,21,22,23]. Expanding DL from two-dimensional (2D) to three-dimensional (3D) research applications is currently a trending research direction [24,25]. Utilizing machine vision techniques to acquire image-based 3D models of seedlings offers several advantages. It eliminates the need for strict research environments, mitigates costly expenses, and enables the capture of color, texture information, and improved measurement precision [26,27]. Sun et al. [28] proposed a high-throughput, three-dimensional rapid plant point cloud reconstruction method based on autonomous calibration of the Kinect v2 sensor position. The sensor captured two Red–Green–Blue-Depth (RGB-D) images of the turntable surface, automatically calculated the center point and normal vector of the turntable rotation axis, and aligned the coordinate systems of the depth images taken from different viewpoints to achieve coarse registration. Subsequently, the iterative closest point algorithm was employed for precise registration of the multi-viewpoint point cloud, enabling fast three-dimensional point cloud reconstruction and extraction of morphological parameters for greenhouse plants. Turgut et al. [29] employed a DL approach based on a 3D point cloud to achieve segmentation and morphological feature extraction of plants in the Rosa genus. The study utilized six novel point cloud-based DL architectures to segment the structural components of the rose tree models and generate 3D models. The results indicated that among the six methods, PointNet++ exhibited the highest segmentation accuracy. The advantage of PointNet++ lies in its flexibility in handling the hierarchical organization of point cloud data. By iteratively extracting features from the neighborhood of each point, PointNet++ enables the network to effectively capture local features of the point cloud [30].

The aforementioned studies have demonstrated the accuracy and effectiveness of morphological feature extraction methods using three-dimensional reconstruction techniques. However, there is currently limited research on the reconstruction of three-dimensional point cloud models for coniferous seedlings. In order to address this gap, we proposed a viable theoretical approach to further meet the demands of technology-assisted ecological engineering in forestry. In this paper, a series of corrected two-dimensional images of P. massoniana from multiple viewpoints were collected, and these images were reconstructed into a 3D point cloud, which then underwent preprocessing. Subsequently, a dense weighted semantic segmentation network based on PointNet++ was introduced to construct a dense weighted semantic segmentation model. Finally, a multi-iteration plane method based on Laplacian contraction to extract the skeletal points of the stems in a refined manner was designed, enabling the extraction of morphological feature values for P. massoniana. This research provides a novel method for detecting morphological indicators during the process of seedling production and cultivation.

2. Materials and Methods

This study primarily focuses on the non-destructive measurement of morphological indicators of P. massoniana. The research involves the collection of images and the utilization of the SFM [31] and PMVS [32] algorithms for reconstructing P. massoniana seedlings. Various processing techniques were applied to the resulting point cloud of P. massoniana, enabling the extraction of seedling morphological indicators. The experimental workflow is illustrated in Figure 1.

2.1. Sample and Experimental Environment

The P. massoniana seedling samples used in this study were sourced from the Qingyuan Nursery in Yizhou District, Hechi City, Guangxi, China. A total of 30 one-year-old seedlings with diverse morphological characteristics were selected, and a subset of these samples is illustrated in Figure 2. Environment for multi-angle image acquisition is illustrated in Figure 3. Parameters of industrial camera and lens are illustrated in

Table 1. Parameters of industrial camera and lens.

	Name	Parameter
MER-1070-10GC	Resolution	3840 × 2748 px
	Frame rate	10 fps
	Sensor	1/2.3″ MT9J003 electronic Rolling shutter CMOS
	Signal-to-noise ratio	36.67 dB
M1620-MPW2	Focal length	16 mm
	Iris	F2.0–F16.0

.

2.2. Camera Calibration

To achieve high-precision 3D reconstruction, it is necessary to obtain the camera’s parameter matrix and perform distortion correction on the images. Camera calibration involves solving parameters such as intrinsic parameters and distortion parameters. In this study, Zhang Zhengyou’s calibration method [33] was chosen for its simplicity of operation and high accuracy in obtaining these parameters. A chessboard pattern with a size of 4 × 5 and a grid size of 20 $\mathrm{m}\mathrm{m}$ per cell was used as the calibration target. Initially, a total of 60 images capturing the chessboard pattern at various poses were collected. Subsequently, the Zhang Zhengyou camera calibration method was applied to obtain the camera’s parameter matrix. To determine the scaling factor of the P. massoniana point cloud, a known-sized calibration block was placed during the capture process. The scaling factor was calculated by comparing the point cloud height of the calibration block with its actual height.

2.3. Acquisition and Preprocessing of Point Cloud

2.3.1. Acquisition of Point Cloud

After distortion correction, multi-angle images of P. massoniana were processed using the SFM algorithm to obtain an initial sparse point cloud, which is then refined into a dense point cloud using the Patch-Based PMVS algorithm. However, due to the minimal color difference and indistinctive shape features of the leaf at the top of P. massoniana seedlings, the algorithm detected only a few feature points, resulting in scattered and discontinuous point cloud data. To address this issue, paper labels were added to the pots containing the seedlings to increase the number of feature points, thereby enhancing the accuracy of the three-dimensional reconstruction of P. massoniana seedlings.

2.3.2. Point Cloud Denoising

Due to the influence of the shooting environment and image background, the obtained dense point cloud of P. massoniana seedlings contained irrelevant points and abundant background noise. Therefore, further processing of the acquired dense point cloud was necessary to reduce the impact of irrelevant information and noise on subsequent plant semantic segmentation and extraction of morphological parameters.

Firstly, prior to capturing multi-angle images of P. massoniana seedlings, a black background with a significant color contrast to the plants and a turntable were chosen. Thus, unrelated noise points resulting from the image background can be directly eliminated using RGB threshold segmentation. The RGB threshold (Red, Green, Blue) is set to (0,0,0)~(40,45,50).

Additionally, to address the issue of scattered outliers and abnormal points in the point cloud of P. massoniana seedlings, this study employed the Statistical Outlier Removal (SOR) filtering method [34] for removal. The SOR filtering algorithm calculates various metrics such as the average distance and standard deviation of each point’s surrounding k-nearest neighbors. By setting an appropriate threshold, the algorithm identifies and eliminates the noise points from the point cloud. The SOR filtering algorithm is configured with a neighborhood point count of k = 6 and a standard deviation multiplier parameter of nSigma = 1.

2.3.3. Point Cloud Coordinate Calibration

The dense point cloud of P. massoniana obtained using SFM-PMVS was unordered and inconsistent. To facilitate the subsequent experimental measurement of various scalar parameters of the P. massoniana point cloud, it is necessary to redefine the coordinate system.

Due to the intricate and complex stem and leaf structure of P. massoniana, it is not suitable for coordinate calibration. Therefore, the point cloud of the pot was first clipped using CloudCompare software 2.12.3. Then, four points, $P_{x1}, P_{x2}, P_{z1}, \mathrm{a}\mathrm{n}\mathrm{d} P_{z2}$, located on the circular top surface of the pot, were selected which are the minimum and maximum coordinates on the $x$ and $z$ axes, respectively, as shown in Figure 4.

Firstly, two points, $P_{x1} \mathrm{a}\mathrm{n}\mathrm{d} P_{x2}$, are selected. After applying the rotation matrix $R_1$, the transformed points, $P_{x1}^{\prime } \mathrm{a}\mathrm{n}\mathrm{d} P_{x2}^{\prime }$, are obtained, ensuring that the line segment formed by $P_{x1}^{\prime } \mathrm{a}\mathrm{n}\mathrm{d} P_{x2}^{\prime }$ is perpendicular to the $y$-axis. By utilizing the equality of the $y$-coordinates of the transformed points, the rotation angle $\theta$ of the point cloud around the $z$-axis can be determined. The coordinate transformation equation for the point cloud is represented by Equation (1):

$$P_{xi}^{\prime }=R_1·P_{xi}=\left[\begin{array}{ccc}\cos \theta & \sin \theta & 0\\ -\sin \theta & \cos \theta & 0\\ 0& 0& 1\end{array}\right]·P_{xi}$$

(1)

where $i$ represents the index of the $i$-th point, and $\theta$ denotes the angle of rotation of the point cloud around the $z$-axis.

Next, two points, $P_{z1} \mathrm{a}\mathrm{n}\mathrm{d} P_{z2}$, are selected. After applying the rotation matrix $R_2$, the transformed points, $P_{z1}^{\prime } \mathrm{a}\mathrm{n}\mathrm{d} P_{z2}^{\prime }$, are obtained, and the line segment formed by connecting $P_{z1}^{\prime } \mathrm{a}\mathrm{n}\mathrm{d} P_{z2}^{\prime }$ is perpendicular to the $y$-axis. By utilizing the constraint of equal $y$-coordinates for the transformed points, the rotation angle $\alpha$ around the $x$-axis for the point cloud is determined. The point cloud coordinate transformation is given by Equation (2):

$$P_{zi}^{\prime }=R_2·P_{zi}=\left[\begin{array}{ccc}1& 0& 0\\ 0& \cos \alpha & \sin \alpha \\ 0& -\sin \alpha & \cos \alpha \end{array}\right]·P_{zi}$$

(2)

where $i$ represents the index of the $i$-th point, and $\alpha$ denotes the angle of rotation around the $x$-axis for the point cloud.

In conclusion, the corrected point cloud of P. massoniana exhibits a bottom surface point cloud parallel to the $xoz$ plane, while the stem point cloud extends vertically along the $y$-axis. The coordinate calibration equation for the P. massoniana seedling point cloud is represented by Equation (3):

$$P_i^{\prime }=R·P_i=R_2·R_1·P_i=\left[\begin{array}{ccc}\cos \theta & \sin \theta & 0\\ -\sin \theta ·\cos \alpha & \cos \theta ·\cos \alpha & \sin \alpha \\ \sin \theta ·\sin \alpha & -\cos \theta ·\sin \alpha & \cos \alpha \end{array}\right]·P_i$$

(3)

where $i$ represents the index of the $i$-th point, $P_i$ denotes the $i$-th point, and $P_i^{\prime }$ represents the $i$-th point after correction.

2.4. Dense Weighted Semantic Segmentation Model Based on PointNet++

In subsequent experiments aimed at extracting morphological indicators from P. massoniana seedlings, different parts of the plant were used to correspond to distinct morphological characteristics. For instance, the pot containing the P. massoniana seedling represents irrelevant background information and needs to be removed prior to measuring the morphological indicators of the stem and leaf. The stem section of P. massoniana can provide parameters such as stem length and ground diameter, while the combination of the stem and leaf sections enables the extraction of the tree height parameter.

The semantic segmentation using the PointNet++-based dense weighted model allows for the rapid and automated segmentation of different parts of P. massoniana point cloud, facilitating subsequent parameter measurements. Compared to the original PointNet++ model, the PointNet++-based dense weighted semantic segmentation model exhibits higher accuracy. Given the dense leaves of P. massoniana with its close connection to the stem, and the green appearance of the stem surface, traditional clustering and threshold segmentation methods prove inadequate, while the PointNet++-based dense weighted semantic segmentation model successfully accomplishes the segmentation task.

PointNet [35] and PointNet++ [30] are DL networks that directly take point cloud data as input. PointNet initially elevates the $(x,y,z)$ coordinates of each point to high-dimensional features using Multilayer Perceptrons (MLPs), and then obtains global features through max pooling. Finally, fully connected MLPs [29] are employed. However, this design limitation hinders PointNet’s ability to effectively extract local fine-grained features.

PointNet++ consists of two main layers: the Hierarchical Set Abstraction (SA) layer and the Feature Propagation (FP) layer. The SA layer includes the sampling and grouping stages. Firstly, the Furthest Point Sample (FPS) algorithm is employed to uniformly sample a fixed number of points from the input point cloud. Then, each sampled point forms a local neighborhood with a radius of R, resulting in multiple overlapping neighborhoods that partition the input point cloud. Finally, PointNet is applied to each neighborhood to extract features specific to that neighborhood. On the other hand, the FP layer propagates the extracted features from each local neighborhood back to the original input point cloud. Therefore, PointNet++ aims to extract fine-grained features at different scales of the input point cloud, focusing on local rather than global information.

Due to the numerous imbalances in the total number of sample points among different classes in the dataset, training the model directly yields unsatisfactory results. Therefore, to achieve a better balance among the quantities of pot, stem, and leaf samples in the entire dataset, this study incorporated class balance weights into the loss function. When calculating the class balance weights, the total number of each class in the dataset is first computed, and the proportion of each class in the dataset is determined using Equation (4):

$$F_i=\frac{{Point}_i}{Sum}(i=\mathrm{0,1},2)$$

(4)

where ${Point}_i$ represents the total number of points labeled as $i$ in the training set; $Sum$ represents the total number of all points in the training set; 0 denotes the pot, 1 denotes the stem, and 2 denotes the leaf; and $F_i$ represents the proportion of labels $i$ in the dataset.

Subsequently, the average proportion, denoted as the $Median$, is computed for each class. Based on the $Median$ and $F_i$, the balanced weights for each class in the dataset are determined using Equation (5):

$${Weight}_i=\frac{Median}{F_i}(i=\mathrm{0,1},2)$$

(5)

where ${Weight}_i$ represents the balancing weight for each class.

The FP layer of PointNet++ propagates local features to the input point set through only two MLP layers. However, when the input point set contains a large number of points, it may lead to incorrect feature assignment for some points. Therefore, this study introduced a feature point adaptive weighting module that calculates the weights of the features of points output by the FP layer across different channels. These weights are then multiplied with the original features, thereby enhancing the expression of the original features and improving the segmentation accuracy of the input point set.

The feature point adaptive weighting module designed in this study consists of 1 adaptive average pooling layer, 1 adaptive max pooling layer, 4 1D convolutional layers, 2 LeakyReLU activation function layers, and 1 Sigmoid activation function layer. The architectural diagram is shown in Figure 5.

Equation (6) represents the computational formula for the feature point adaptive weighting module:

$$F=\sigma (W_1(LeakyReLU{(W}_0(F_{avg}^c)))+W_1(LeakyReLU{(W}_0(F_{max}^c))))$$

(6)

where $\sigma$ represents the Sigmoid activation function; $W_0 \mathrm{a}\mathrm{n}\mathrm{d} W_1$ denote the weights of the first and second convolutional layers within this module; and $F_{avg}^c \mathrm{a}\mathrm{n}\mathrm{d} F_{max}^c$ refer to the tensors resulting from adaptive average pooling and adaptive max pooling, respectively.

Furthermore, to enhance the classification capability of the fully connected (FC) layer in the network, this study also introduced a dense mapping block that incorporates a feature point adaptive weighting module. By progressively reducing the feature output dimensions and iteratively optimizing the feature weights at each layer, the accuracy of the network is further improved.

The dense mapping block designed in this study consists of 1D convolutional layers, 1D BatchNorm layers, ReLU activation function layers, feature point adaptive weighting modules, and Dropout layers, with a dropout probability of 0.1 for each neuron. The structure of the dense weighted semantic segmentation network, built based on PointNet++, is illustrated in Figure 6.

In this study, a multi-angle image-based approach was employed to perform 3D reconstruction of P. massoniana. Subsequently, a series of data processing steps, including image distortion correction, point cloud denoising, and coordinate calibration, were conducted to obtain high-quality point cloud data. Moreover, to facilitate the rapid measurement of morphological indicators from the acquired P. massoniana point cloud, an improved PointNet++ algorithm was utilized for semantic segmentation. This allowed for the classification of the point cloud into three categories: pot, stem, and leaf.

The dataset preparation was performed using Cloud-Compare software, where each P. massoniana point cloud was manually divided into three parts. The training and testing sets were split in an 8:2 ratio. An example of the created dataset is shown in Figure 7.

The hardware and software studied were system: Windows 10; CPU: Intel I7-11700F 2.50 GHz; GPU: Nvidia GeForce RTX 3080Ti (12GB); and environment configuration: Py-Charm + Pytorch1.8 + Python 3.7.4 + Cuda 12.1, Agisoft Metashape, Cloud-Compare 2.12.3, MATLAB R2021b.

2.5. Multi-Iteration Plane Method Based on Laplacian Contraction

After obtaining the segmented stem point cloud of P. massoniana seedlings, extraction of the stem skeleton points is necessary to measure the stem length, ground diameter, and height-to-diameter ratio. The original Laplacian contraction algorithm tends to produce skeleton points with large intervals, leading to large errors when calculating the stem length. Therefore, this study proposed a multi-iteration plane method based on Laplacian contraction. By slicing the original stem point cloud into layers using the Laplacian contraction algorithm, new skeleton points are generated within each point cloud layer. These new skeleton points are positioned to minimize the sum of Euclidean distances between themselves and all other points in the corresponding point cloud layer. The refinement process improves the measurement accuracy of morphological indicators for P. massoniana seedlings. For complex morphological structures in certain P. massoniana samples, increasing the number of iterations allows for the extraction of skeleton points that accurately reflect the growth status of the stem.

2.5.1. Nearest-Neighbor Sorting

The skeleton points extracted by the Laplacian contraction algorithm are unordered, thus requiring sorting of the skeleton points before the multi-plane iteration.

The skeleton points extracted using the Laplacian contraction algorithm are denoted as the point set $O$, and the sorted skeleton points are denoted as the point set $O^{\prime }$. Initially, the point set $O^{\prime }$ is an empty set. Firstly, the point $P_m$ with the minimum $y$-axis in the point set $O$ is selected as the starting point. $P_m$ is removed from the point set $O$ and appended to the point set $O^{\prime }$. Subsequently, the updated point set $O$ is searched to find the point $P_n$ with the minimum distance ($D$) from the end point $P_m$ of the updated point set $O^{\prime }$. $P_n$ is then removed from the point set $O$ and appended to the point set $O^{\prime }$. This process is repeated, resulting in the correctly sorted point set $O^{\prime }$. The distance $D$ is calculated using Equation (7):

$$D=\sqrt{{(x_m-x_n)}^2+{(y_m-y_n)}^2+{(z_m-z_n)}^2}$$

(7)

where $x_m, y_m, z_m, x_n, y_n, {\mathrm{a}\mathrm{n}\mathrm{d} z}_n$ represent the coordinates of points $P_m \mathrm{a}\mathrm{n}\mathrm{d} P_n$.

2.5.2. Skeleton Refinement of P. massoniana Stem Point Cloud

After obtaining the sorted set of points, $O\prime$, firstly, select two points, $P_1^{\prime }$ and $P_2^{\prime }$, and consider the vector passing through these two points as the normal vector, $N_1$. Construct two planes, $A_1$ and $A_2$, perpendicular to $N_1$ at these two points, respectively. Using $A_1$ and $A_2$ as plane constraints, obtain a subset of points, $S_1$, representing the stem point cloud within these planes. Then, using the minimum bounding box of $S_1$ as a constraint, identify the point, $T_1$, that minimizes the sum of Euclidean distances between $T_1$ and all points in $S_1$. Next, take $P_1^{\prime }$ and $T_1$ as new reference points and repeat the aforementioned steps to obtain the second refined skeleton point, $T_2$. Subsequently, compute the third skeleton point, $T_3$, based on $T_1$ and $P_2^{\prime }$. By iteratively repeating this process, a set of refined skeleton points, $T$, can be obtained. The workflow of the multi-iteration plane method based on Laplacian contraction is illustrated in Figure 8.

2.6. Acquisition of Morphological Indicators for P. massoniana Seedlings

The morphological indicators investigated in this study for P. massoniana primarily include plant height, stem length, ground diameter, and height-to-diameter ratio.

In this study, the true plant height of P. massoniana samples was measured using a measuring tape. The true ground diameter was measured using an electronic caliper. For measuring the true stem length of P. massoniana seedlings, a flexible ruler was placed along the curve of the stem, and then straightened to measure its length. The ratio of plant height to ground diameter is referred to as the height-to-diameter ratio of P. massoniana.

After calibration, the point cloud of P. massoniana exhibits a bottom plane parallel to the $xoz$ plane, while the direction of the stem is vertically aligned along the $y$-axis. Therefore, the difference between the maximum and minimum values in the $y$-axis direction of the stem and leaf point cloud represents the plant height, denoted as $H_c$. By slicing the point cloud using two refined skeleton points at the bottom of the stem point cloud, the sum of the differences between the maximum and minimum values in the $x$ and $z$-axis directions of the sliced point cloud, divided by two, gives the ground diameter, denoted as $W_c$. The sum of the Euclidean distances between the refined skeleton points provides the stem length, denoted as $L_c$. The ratio of plant height to ground diameter is defined as the height-to-diameter ratio, denoted as $R_{HW}$. These relationships are mathematically described by Equations (8)–(11):

$$H_c=P_{ymax}-S_{ymin}$$

(8)

$$W_c=\frac{(s_{xmax}-s_{xmin})+(s_{zmax}-s_{zmin})}{2}$$

(9)

$$L_c=\sum _{i=1}^{n-1}\sqrt{{(x_i-x_{i+1})}^2+{(y_i-y_{i+1})}^2+{(z_i-z_{i+1})}^2}$$

(10)

$$R_{HW}=\frac{H_c}{W_c}$$

(11)

where $P_{ymax}$ represents the point with the maximum $y$-axis in the leaf point cloud; $S_{ymax}$ and $S_{ymin}$ correspond to the points with the maximum and minimum $y$-axis values, respectively, in the stem point cloud; $s_{xmax}$, $s_{xmin}$, $s_{zmax}$, and $s_{zmin}$ denote the points with the maximum and minimum $x$- and $z$-axis values in the stem point cloud slice; $x_i$, $y_i$, and $z_i$ denote the coordinates of the $i$-th point in the refined skeleton points, with $n$ representing the total number of skeleton points; $p_{xmax}$, $p_{xmin}$, $p_{zmax}$, and $p_{zmin}$ refer to the points with the maximum and minimum $x$- and $z$-axis values, respectively, in the leaf point cloud.

2.7. Evaluation Indicators

In this study, the accuracy of the semantic segmentation for the P. massoniana point cloud was evaluated using measures such as $Accuracy$ and Mean Intersection over Union ($mIoU$). The accuracy of skeleton extraction was assessed using the determination coefficient ($R^2$) for the stem length parameter of P. massoniana. The precision of three-dimensional reconstruction was evaluated according to Equations (12) and (13):

$$Accuracy=\frac{{\sum }_{j=0}^k{P}_{ii}}{{\sum }_{i=0}^k{\sum }_{j=0}^k{P}_{ij}}$$

(12)

$$mIOU=\frac{1}{k}\sum _{i=0}^k\frac{P_{ii}}{{\sum }_{i=0}^k{P}_{ij}+{\sum }_{i=0}^k{P}_{ji}-P_{ii}}$$

(13)

where $k$ represents the number of categories in the dataset. $P_{ii}$ represents the number of accurately predicted point clouds. $P_{ij}$ and $P_{ji}$ denote the misclassified point clouds, where $P_{ij}$ refers to the number of point clouds with a true label of $j$ but predicted as $i$, and $P_{ji}$ refers to the number of point clouds with a true label of $i$ but predicted as $j$.

3. Results and Discussion

3.1. Camera Parameter Matrix

The parameters required for camera calibration were calculated and are presented in

Table 2. Camera parameters.

Camera	Intrinsic Matrix	Radial Distortion Coefficient	Average Reprojection Error	Resolution
Front-view camera	$\left(\begin{array}{ccc}\mathrm{10,485.8106}& 0& 1231.4592\\ 0& \mathrm{10,501.9894}& -176.3217\\ 0& 0& 1\end{array}\right)$	$\left(\begin{array}{cc}-0.2158& -0.0870\end{array}\right)$	0.3071	3840 × 2748
Oblique-view camera	$\left(\begin{array}{ccc}\mathrm{10,053.5173}& 0& 2030.3629\\ 0& \mathrm{10,074.7860}& 728.5858\\ 0& 0& 1\end{array}\right)$	$\left(\begin{array}{cc}-0.3832& 3.8786\end{array}\right)$	0.1277	3840 × 2748

. The consistency between the resolution of the calibrated images and the actual resolution indicates the correctness of the camera calibration experiment results.

Figure 9 illustrates both the original images and the images after distortion parameter correction. Figure 9a,b represent the original images captured by the frontal and oblique cameras. Figure 9c,d represent distortion-corrected images. The inconsistency in the proportion of seedlings displayed is due to the fact that the seedlings in the corrected image are not the actual size. In this study, a calibration block was introduced to determine the scaling factor and restore it to its true size.

3.2. P. massoniana Point Cloud

3.2.1. Original P. massoniana Seedling Point Cloud

To validate the significance of maintaining seedling stability during the acquisition of multi-angle images for three-dimensional reconstruction, this section compares the reconstruction results of two image acquisition methods.

Figure 10a shows the point cloud of P. massoniana reconstructed from the multi-angle images captured during continuous rotation of the turntable, depicting one full rotation of the seedling. In Figure 10b, the turntable rotated at a fixed angle of 5°, and the images were captured after waiting for the seedling to stabilize, resulting in the reconstructed point cloud of the seedling. It is evident that the point cloud in Figure 10a exhibited noticeable ghosting artifacts, resulting in significant measurement errors of the ground diameter. This is primarily attributed to the slender and tall stems of P. massoniana seedlings which, even with low-speed rotation, can shake. Conversely, the point cloud in Figure 10b demonstrates that waiting for the P. massoniana seedling to stabilize before image capture effectively reduces the ghosting artifacts caused by these jitters.

A partial point cloud of P. massoniana seedlings obtained through 3D reconstruction is shown in Figure 11.

3.2.2. Point Cloud Denoising

In this section, $RGB$ threshold segmentation is applied to remove irrelevant noise points such as the background with black color. The $SOR$ filtering algorithm was selected to eliminate scattered outliers and abnormal points in the point cloud. After the above preprocessing, a higher accuracy and fewer outliers in the P. massoniana seedling point cloud can be obtained, and the complex branch and leaf structures at the top of the plants were also more clearly visible. Figure 12 illustrates the point cloud before and after denoising for a P. massoniana seedling.

3.2.3. Point Cloud Coordinate Calibration

To facilitate the extraction of morphological indicators from the P. massoniana point cloud for subsequent experiments, it is necessary to perform coordinate correction on the point cloud, aligning the basal plane of the P. massoniana with the $xoz$ plane. A P. massoniana point cloud before and after coordinate correction is shown in Figure 13.

3.2.4. Comparisons with Previous Study

In 2019 [36], our team designed a non-contact morphological parameter extraction system for P. massoniana seedlings based on multi vision, including the establishment of a hardware experimental platform for seedling image acquisition and 3D reconstruction based on Delaunay triangulation algorithm. The reconstructed model diagram is shown in Figure 14. The difference between early surface reconstruction fitting and Figure 13b shows that the early details are clearly not refined enough, and the gaps between the pine needles of the seedlings could not be accurately fitted and segmented. The point cloud reconstruction method based on SFM and PMVS in this study had clearer images, and the subsequent extracted morphological index parameters were more accurate.

3.3. Dense Weighted Semantic Segmentation Model Based on PointNet++

By incorporating the feature point adaptive weighting module in the FP layer, the features are enhanced, enabling better capture of both local and global characteristics of P. massoniana point clouds. The addition of the dense mapping block in the FC layer, with progressively decreasing feature dimensions and adaptive adjustment of the weight for each feature point, further improves the accuracy of the network. The results of the PointNet++ model before and after the improvement are presented in

Table 3. Training results of the model.

Model	Training Set		Test Set
	Accuracy	Loss	Accuracy	mIOU
PointNet++	0.8966	0.1444	0.8915	0.6361
CDC-PointNet++ (our algorithm)	0.9448	0.0814	0.9430	0.7872

.

The Accuracy, Loss, and mIOU curves during model training are illustrated in Figure 15.

The training set accuracy of the CDC-PointNet++ model reached 0.9448, demonstrating a 4.82% improvement compared to the original network. For the test set, the accuracy and mIoU achieved values of 0.9430 and 0.7872, respectively, representing improvements of 5.15% and 15.11% compared to the original network. The proposed feature point adaptive weighting module in this study enables adaptive learning of each point’s features, enhancing the original feature representation and providing a more comprehensive feature representation for the original network. Additionally, the proposed dense mapping block optimizes output features layer by layer, enhancing the network’s non-linearity, stability, and generalization ability, thereby improving the accuracy and mIoU of the final segmentation. The segmentation results of the dense weighted semantic segmentation model based on PointNet++ on the test set are visualized in Figure 16.

3.4. Stem Point Cloud Skeleton Extraction

To obtain the stem length parameter of P. massoniana, this study employs the Laplacian contraction algorithm for skeleton extraction from the stem point cloud. By cumulatively calculating the Euclidean distance between adjacent points, the stem length parameter is determined. The extracted skeleton points are illustrated in Figure 17.

From Figure 17a, it can be observed that the extracted initial skeleton points have relatively large gaps between neighboring points. Moreover, due to the arc-shaped nature of P. massoniana stems, there is a large error when calculating the stem length parameter using Euclidean distances. To minimize this calculation error, this study employed a multi-iteration plane method based on Laplacian contraction to refine the initial skeleton points. The refined skeleton points are illustrated in Figure 17b.

The measured stem length parameters from the original skeleton points and the refined skeleton points are presented in Figure 18.

As one of the challenging parameters to measure in plant morphological indicators, achieving high precision in stem length measurements is difficult. From Figure 18, it can be observed that the stem length parameter obtained from the original skeleton points underestimates the true values, with a coefficient of determination ($R^2$) of only 0.5142. This discrepancy arises from the large gaps between neighboring points in the original skeleton, rendering the direct use of Euclidean distances between neighboring points as a substitute for the arc length of P. massoniana unfeasible. However, after skeleton refinement, the $R^2$ value increases to 0.7191, resulting in a 20.49% improvement in stem length measurements compared to those obtained from the original skeleton points. This improvement is attributed to the fact that skeleton refinement reduces the distances between neighboring points, making the Euclidean distances between them more comparable to the arc length and thereby yielding stem length measurements that closely align with the true values.

3.5. Other Morphological Indicators of P. massoniana Seedlings

From the experiment, the predicted values and the actual values of the ground diameter, plant height, and height-to-diameter ratio parameters for 30 groups of P. massoniana seedlings obtained are shown in Figure 19.

From Figure 19, it can be observed that the predicted ground diameter parameter of P. massoniana exhibited an accuracy of 0.8677 when compared to the ground truth values. The primary cause of inaccuracies in ground diameter measurements is the presence of some holes and outliers in the reconstructed point cloud of P. massoniana, which are difficult to eliminate through point cloud preprocessing. Furthermore, due to the positional relationship between stem and leaf point cloud and errors introduced by the manually annotated dataset, the semantic segmentation network failed to precisely segment the stem and leaf point cloud. In contrast, the predicted plant height parameter of P. massoniana demonstrated an accuracy of 0.8893 when compared to the ground truth values. This is because the plant height parameter measures the difference between the maximum and minimum $y$-axis of the stem and leaf point cloud of P. massoniana. Notably, there were distinctive characteristics between the pot, the stem and leaf (the pot being at the bottom), enabling the semantic segmentation network to accurately segment the pot point cloud. Moreover, the $R^2$ for the predicted height-to-diameter ratio parameter of P. massoniana reached 0.8354. Additionally, during the measurement of P. massoniana morphological indicators, the use of calibration blocks to determine the scaling factor between the reconstructed seedling and the real seedling introduced errors that contributed to the predicted errors of P. massoniana seedling morphological indicators.

4. Conclusions

The aim of this study was to design a measurement method for morphological indicators of P. massoniana seedlings utilizing 3D reconstruction, semantic segmentation models, and skeleton extraction techniques.

(1)

For the 3D reconstruction of P. massoniana seedlings, an image acquisition platform was established to capture a multi-angle sequence of 2D images. After distortion correction, the images are used in SFM combined with PMVS to construct a 3D point cloud of P. massoniana seedlings. Preprocessing methods such as denoising and coordinate correction were applied to the original point cloud of P. massoniana.

(2)

The processed P. massoniana point cloud was used to create a dataset, and a dense weighted semantic segmentation network based on PointNet++ was proposed to enhance the representation of original features and improve the segmentation accuracy of the input point set. The experimental results demonstrated that the improved PointNet++ network accurately segments the pot, stem, and leaf of P. massoniana. The accuracy of the model on the training and testing sets reached 0.9448 and 0.9430, respectively, indicating its applicability and segmentation performance in point cloud segmentation.

(3)

To reduce measurement errors caused by large gaps between neighboring points, a multi-plane iterative method based on Laplacian contraction was employed to refine the skeleton points of the stem point cloud. After skeleton refinement, $R^2$ increased to 0.7191, representing a 20.49% improvement in stem length compared to measurements obtained from the original skeleton points. The predicted parameters of plant height, ground diameter, and height-to-diameter ratio for P. massoniana show good correlation with the ground truth values, with an average determination coefficient of 0.8641.

The proposed method and workflow are also applicable to other tree seedlings. This study replaces traditional manual measurement methods while meeting measurement accuracy and increases the stability of measurement data while reducing manual labor, providing a more accurate approach for rapid non-destructive measurements of seedling morphological indicators, as well as a reference for the precise sorting of P. massoniana seedlings. In the future, combined with measurements of the growth and changes in roots, comprehensive applications will be implemented according to the national seedling quality grading standards.