Phenotype Analysis of Arabidopsis thaliana Based on Optimized Multi-Task Learning

Abstract

Deep learning techniques play an important role in plant phenotype research, due to their powerful data processing and modeling capabilities. Multi-task learning has been researched for plant phenotype analysis, which can combine different plant traits and allow for a consideration of correlations between multiple phenotypic features for more comprehensive analysis. In this paper, an intelligent and optimized multi-task learning method for the phenotypic analysis of Arabidopsis thaliana is proposed and studied. Based on the VGG16 network, hard parameter sharing and task-dependent uncertainty are used to weight the loss function of each task, allowing parameters associated with genotype classification, leaf number counting, and leaf area prediction tasks to be learned jointly. The experiments were conducted on the Arabidopsis thaliana dataset, and the proposed model achieved weighted classification accuracy, precision, and $F_w$ scores of $96.88\%$, $97.50\%$, and $96.74\%$, respectively. Furthermore, the coefficient of determination $R^2$ values in the leaf number and leaf area regression tasks reached 0.7944 and 0.9787, respectively.

1. Introduction

Plant phenotypes are the physical, physiological, and biochemical characteristics of plants that reflect their structure and composition. Traditional phenotype analysis is performed manually, which introduces problems such as high labor and material costs, and low efficiency [1,2]. Therefore, traditional methods are unable to meet the demands of high-throughput plant phenotype research. Establishing a high-throughput phenotype analysis platform could allow plant phenotype data to be objectively, accurately, and quickly obtained, as well as facilitating the analysis of the associations between phenotypic traits, which could provide data-driven decision support for breeding, cultivation, and agricultural practices [3].

Deep learning techniques have became an important tool in plant phenotype research due to their powerful data processing and modeling capabilities [4,5]. Deep learning approaches have became the main technical tools in plant phenotype research and are effective in plant classification, identification, and feature extraction [6,7,8].

Convolutional neural network (CNN) is a widely acknowledged representative deep learning approach, that has achieved great success in many computer vision tasks. In recent years, researchers have proposed a large number of CNN-based models, such as AlexNet [9], VGG [10], GoogLeNet [11], and ResNet [12]. Additionally, these architectures have inspired subsequent contributions, such as Res2Net [13], Condensenet v2 [14], EdgeNeXt [15], and GhostNetV2 [16]. The above-mentioned CNNs have been widely used for plant phenotype data analysis [17,18].

There are many research works focused on single-task learning, for plant classification, leaf number prediction, and pest and disease detection. For example, Lee et al. [19] trained a fine-tuned AlexNet model to automatically extract leaf features. Dyrmann et al. [20] constructed a convolutional neural network to classify seedlings of plants or families. Barre et al. [21] proposed a method for the quantification of plant phenotypes associated with the growth of Arabidopsis thaliana in a controlled environment and conducted categorization by using a convolutional neural network structure for identification. Namin et al. [22] used a CNN for automatic joint feature and classifier learning, and applied a long short-term memory (LSTM) model to study the growth of plants and their dynamic behavior. Roitsch et al. [23] first provided a detailed description of genomics-based methods for plant phenotype feature extraction, followed by the idea of applying plant phenotype studies in conjunction with methods related to deep learning. Kong et al. [24] employed the Mask R-CNN algorithm to effectively identify and count full and wilted grains in rice spikes.

Single-task learning methods mainly focus on a specific phenotypic feature and characteristic of a plant, without considering the diversity and correlations between phenotypic features. On the other hand, multi-task learning mechanisms can bring related tasks together to learn the mutual generalization bias, allowing the use of priori knowledge [25,26,27] to improve the learning ability. For instance, Pound et al. [28] used multi-task learning for wheat spike phenotyping analysis. Wen et al. [29] proposed a multi-task learning network based on Mask R-CNN, which could achieve the multi-task detection of citrus in complex environments. La et al. [30] used a fully convolutional regression network-based multi-task learning architecture to detect individual full-grown tree, tree seedlings, and tree gaps in citrus orchards. Khaki et al. [31] utilized MobileNetV2 as a lightweight backbone feature extractor with two parallel sub-networks to perform density-based counting and localization tasks simultaneously, with a lower number of parameters, superior performance, and higher effectiveness when compared to other methods. Chaudhury et al. [32] proposed a multi-task deep learning framework to segment leaves and count the number of leaves in digital plant images, which showed an improved segmentation mean intersection over union score and difference. Keceli et al. [33] combined CNN features and transferred features to construct a multi-input multi-task learning model to predict the disease in plants. Wang et al. [34] proposed a dual-stream hierarchical bilinear pooling model and optimized the weights of two tasks using homoscedastic uncertainty, which improved the independent identification accuracy of plants and diseases.

In this paper, we propose an optimized multi-task learning model for phenotypic analysis of Arabidopsis thaliana with a hard parameter sharing approach and uncertainty weighting strategy. Based on the hard parameter sharing framework, the loss function of each task is weighed according to the uncertainty of task dependence. We tested three tasks related to Arabidopsis thaliana plant phenotypic analysis: (1) classifying the genotype, (2) counting the total number of leaves, and (3) estimating the leaf area. Different multi-task learning models are constructed and compared for the above three tasks.

2. Multi-Task Learning Architecture for Arabidopsis thaliana Phenotypic Analysis

The architecture of the multi-task model used for Arabidopsis thaliana phenotype analysis in this paper is shown in Figure 1. The hard parameter sharing framework was used as the underlying network, and each task had its own task-specific layer in order to extract different features related to the task. During the joint training of the multiple tasks, task-dependent uncertainty was used to weigh the loss function for each task layer, thus balancing the impact of regression and classification tasks. For this paper, we adopted the hard parameter sharing framework of [27], which describes a multi-task learning model with a sharing network based on ResNet for Arabidopsis thaliana phenotyping. The symbols appearing in the paper and their details are summarized in

Table 1. Important mathematical symbols and their descriptions.

Symbol	Description
$\eta$	Learning rate
$\sigma$	Weighting relationship factors
$P_w$	Weighted precision of the classification task
$R_w$	Weighted recall of the classification task
A	Accuracy of the classification task
$F_w$	Weighted comprehensive performance of precision and recall
$R^2$	Coefficient of determination
$MAPE$	Mean absolute percentage error in the regression task
$MAE$	Mean absolute error in the regression task
$MSE$	Mean square error in the regression task

.

2.1. Modified Hard Parameter Sharing in VGG16 Network

VGG16 [10] is a CNN model, which uses stacked small convolutional kernels with a low number of parameters and more non-linear variations, making the extracted features more adaptable to complex patterns. Thus, VGG16 can be easily modified to achieve better classification performance [10]. For the above reasons, the VGG16 was chosen as the backbone network for the multi-task learning model proposed in this paper.

We added a specific task layer to the VGG16 network. The modified model structure is shown in Figure 2, where the dashed box indicates the specific task layer added to complete the genotype classification, leaf number counting, and leaf area estimation tasks.

As shown in Figure 2, the last maximum pooling layer of VGG16 was removed and replaced with fully connected layers. Then, fully connected layers were also applied to each of the three task-specific layers. The fully connected layers were used for extracting features associated with each of the tasks.

The three tasks were designed as follows: (1) the genotype classification task passes through fully connected layers, and the classification results are finally output through a softmax layer; (2) the leaf number counting task passes through fully connected layers, then outputs the regression prediction results; (3) and the construction of the leaf area estimation-specific task layer is the same as the leaf number counting task. The details of the model parameters for each task with the original VGG16 network are detailed in

Table 2. Network parameters for each task in the modified VGG16 model.

No.	Genotype Classification	Leaf Number Counting Regression	Leaf Area Regression
1	input (224 × 224 RGB image)
2	conv3-64
3	conv3-64
4	maxpool
5	conv3-128
6	conv3-128
7	maxpool
8	conv3-256
9	conv3-256
10	conv3-256
11	maxpool
12	conv3-512
13	conv3-512
14	conv3-512
15	FC-1536
16	FC-512	FC-512	FC-512
17	FC-256	FC-1	FC-1
18	softmax	-	-

.

After the plant phenotype images are input to the network model, the corresponding feature vectors are extracted through the underlying shared network (i.e., VGG16) and the respective task-specific fully connected layers for each task. The genotype classification task involves multiple classifications of the feature vectors through the softmax layer to obtain the probabilities for each category.

The softmax layer applies the $Softmax$ activation function, which transforms the inputs of multiple neurons into a value between 0 and 1, where the sum of the mapped elements in the vector is equal to 1. The softmax value is calculated using Equation (1):

$$Softmax\left(\mathit{x}\right)=\frac{e^{{\mathit{x}}_i}}{{\sum }_j{e}^{{\mathit{x}}_j}},$$

(1)

where i and j are category labels in the multi-categorization problem, ${\mathit{x}}_i$ denotes the $\mathit{x}$ value for the input category i and maps the value to an exponential function, and ${\sum }_j{e}^{{\mathit{x}}_j}$ denotes the exponential sum of all category values.

The leaf number regression task and the leaf area regression task apply the $ReLU$ activation function in the task-specific fully connected layer to obtain the final prediction results. The $ReLU$ function can overcome the problem of gradient disappearance, and is calculated using Equation (2).

$$Relu\left(\mathit{x}\right)=max(0,\mathit{x}),$$

(2)

where $\mathit{x}$ is the input value.

2.2. Loss Functions for Different Tasks

The loss functions used in this paper include the mean absolute error, mean square error, and cross-entropy loss function. The mean absolute error (MAE) loss function is applied to solve the problem of prediction of specific values. Its formula is shown in Equation (3):

$$L_{MAE}=\frac{1}{n}\sum _{i=1}^n|y_i-\widehat{y_i}|.$$

(3)

The MAE measures the average of the sum of the absolute differences between the actual values $y_i$ and the predicted values $\widehat{y_i}$, where n is the total number of actual values.

The mean squared error (MSE) loss function is applied to regression problems, and is shown in Equation (4). The MSE measures the mean value of the square of the difference between the predicted values $\widehat{y_i}$ and the actual observed values $y_i$.

$$L_{MSE}=\frac{1}{n}\sum _{i=1}^n{(y_i-\widehat{y_i})}^2.$$

(4)

The cross-entropy measures the distance between the actual output probability and the expected output probability.The smaller the cross-entropy value, the closer the two probability distributions will be. The cross-entropy loss function is often used in solving the classification problems, and is calculated using Equation (5):

$$L_{CrossEntropy}=-\frac{1}{n}\sum _i\sum _{c=1}^M{y}_{ic}\log \left(p_{ic}\right),$$

(5)

where M is the number of categories, $y_{ic}$ is a symbolic function that takes 1 if the true category of sample i is equal to c and 0 otherwise, and $p_{ic}$ is the predicted probability that observation sample i belongs to category c.

2.3. Loss Balance for Each Task

In this paper, each task-specific layer loss function is weighted training to balance the task-dependent uncertainty [35]. Thus, a joint loss is used during model training, which balances the impact of each task and can mitigate the impact of negative migration on weakly correlated tasks. In Figure 2, the circles represent the joint loss values. In the last task-specific layers for the three tasks, the task-dependent uncertainty is used to balance the loss function for each task, and the joint loss values are used to guide the next training round of the model. Finally, the final joint multi-task loss is evaluated by maximizing the maximum Gaussian likelihood.

2.3.1. Likelihood Function for the Regression Task

For the regression task, the probability is defined in terms of the Gaussian likelihood for output [35], as shown in Equation (6):

$$p\left(y\right|f^W\left(\mathit{x}\right))=N(f^W\left(\mathit{x}\right),{\sigma }^2),$$

(6)

where $\mathit{x}$ is the input, y is the real label, W is the parameter matrix, $f^W\left(\mathit{x}\right)$ is the output, and $\sigma$ is the observed noise parameter of the model.

Based on this Gaussian likelihood, the log-likelihood of the regression task in joint multi-task learning [35] can be obtained, as shown in Equation (7):

$$\log p\left(y\right|f^W\left(\mathit{x}\right))\propto -\frac{1}{2{\sigma }^2}{\Vert y-f^W\left(\mathit{x}\right)\Vert }^2-\log \sigma .$$

(7)

2.3.2. Likelihood Function for Classification Task

In the classification task, the Gaussian likelihood for the probability [35] is shown in Equation (8):

$$p\left(y\right|f^W\left(\mathit{x}\right))=Softmax\left(f^W\left(\mathit{x}\right)\right).$$

(8)

The scaling factor ${\sigma }^2$ is added to Equation (8), which is related to the uncertainty of the distribution and determines the flatness of the discrete distribution. The final log-likelihood of the classification task [35] is obtained as shown in Equation (9):

$$\log p(y=c|f^W\left(\mathit{x}\right),\sigma )=\frac{1}{{\sigma }^2}{f_c}^W\left(\mathit{x}\right)-\log (\sum _{c^{{}^{\prime }}}{e}^{\frac{1}{{\sigma }^2}{f_{c^{{}^{\prime }}}}^W\left(\mathit{x}\right)}).$$

(9)

2.3.3. Multi-Task Joint Loss

The multi-task model in this paper consists of a classification task and two regression tasks. The joint loss was constructed, referring to the literature [35], to maximize the log-likelihood (i.e., minimize the negative log-likelihood) as shown in Equation (10):

$$\begin{array}{cc}\hfill L_{uncertainty}& =-\log p(y_1,y_2,y_3=c|f^W\left(\mathit{x}\right))\hfill \\ \hfill & =\frac{1}{2{{\sigma }_1}^2}{\Vert y_1-f^W\left(\mathit{x}\right)\Vert }^2+\log {\sigma }_1+\frac{1}{2{{\sigma }_2}^2}{\Vert y_2-f^W\left(\mathit{x}\right)\Vert }^2+\log {\sigma }_2-\log p(y_3=c|f^W\left(\mathit{x}\right),{\sigma }_3)\hfill \\ \hfill & =\frac{1}{2{{\sigma }_1}^2}{L}_1\left(W\right)+\frac{1}{2{{\sigma }_2}^2}{L}_2\left(W\right)+\frac{1}{{{\sigma }_3}^2}{L}_3\left(W\right)+\log {\sigma }_1+\log {\sigma }_2+\log \frac{{\sum }_{c^{{}^{\prime }}}{e}^{\frac{1}{{{\sigma }_3}^2}{f_{c^{{}^{\prime }}}}^W\left(\mathit{x}\right)}}{{\left({\sum }_{c^{{}^{\prime }}}{e}^{{f_{c^{{}^{\prime }}}}^W\left(\mathit{x}\right)}\right)}^{\frac{1}{{{\sigma }_3}^2}}}\hfill \\ \hfill & \approx \frac{1}{2{{\sigma }_1}^2}{L}_1\left(W\right)+\frac{1}{2{{\sigma }_2}^2}{L}_2\left(W\right)+\frac{1}{{{\sigma }_3}^2}{L}_3\left(W\right)+\log {\sigma }_1+\log {\sigma }_2+\log {\sigma }_3,\hfill \end{array}$$

(10)

where $y_1$ is the true label of the leaf area regression task, ${\sigma }_1$ is the noise parameter of the leaf area regression task, $y_2$ is the true label of the leaf number regression task, ${\sigma }_2$ is the noise parameter of the leaf number regression task, $y_3$ is the true label of the genotype classification task, and ${\sigma }_3$ is the noise parameter of the classification task. Furthermore, ${\sigma }_1$, ${\sigma }_2$, and ${\sigma }_3$ are equivalent to data-based adaptive weights, which are used for weighting the loss function of each task factor.

The term $L_1\left(W\right)={\Vert y_1-f^W\left(\mathit{x}\right)\Vert }^2$ denotes the Euclidean distance between the output of the leaf area regression task and the true label $y_1$, $L_2\left(W\right)$ denotes the Euclidean distance between the output of the leaf number regression task and the true label $y_2$, and $L_3\left(W\right)=-\log \left(Softmax\left(f^W\left(\mathit{x}\right)\right)\right)$ is the cross-entropy loss for the classification task. Overall, the process of model training is the process of learning the weight relationship factor for each task.

2.3.4. Multi-Tasking Learning Processing

During training, the weight-related parameters are combined together, according to Equation (10), as a joint loss. Then, each weight-related parameter is updated gradually. Finally, a combination of a weight parameter matrix and multiple task weight relationship factors are learned, such that the multi-loss function for the multiple tasks reaches a minimum value and the influence of each task will be well-balanced.

During the model training process, the model optimizer automatically updates the parameters of the network, including the weight value for each task, according to the gradient information in order to reduce the value of the loss function. The update process for the weight-related parameters is shown in Figure 3.

In Figure 3, if the joint loss value of the model is calculated after the previous iteration and the proportion of the leaf area loss in the total loss is larger (i.e., the value of $\frac{1}{2{{\sigma }_1}^2}{L}_1\left(W\right)$ in Equation (10) is larger), then the optimizer will train the model in favor of the leaf area regression task in the next iteration, then increase the weight correlation factor of the leaf area regression task defined in Equation (10) for ${\sigma }_1$, in order to reduce the weight of this task. The adjustment procedure for ${\sigma }_2$ and ${\sigma }_3$ weighting each task’s loss in the total joint loss is calculated in a similar way.

3. Experimental Results

3.1. Experimental Environment

The experiment was conducted on a Windows 11 computer with an Intel i9-12900K CPU, 48 GB RAM, and a GeForce RTX 3060 Ti GPU with 8 GB RAM. The algorithms were implemented using Tensorflow 2.4.0, Python 3.7, Pytorch 1.3.1, and CUDA 10.0.

3.2. Dataset

The experiments were carried out on the Arabidopsis thaliana dataset from Ara2013-Canon [36]. The dataset contains 5 genotypes of Arabidopsis thaliana (col-0, ein2, ctr, adh1, and pgm) and has 165 images in total. Each image has been labeled with information regarding genotype, leaf number, and corresponding annotated color leaf segmentation images, as shown in

Table 3. Sample Arabidopsis thaliana images from the Ara2013-Canon dataset [36].

Genotype	Col-0	ein2	pgm	ctr	adh1
Original image
Leaf segmentation image
Leaf numbers	7	8	5	6	7
Total number of samples	35	39	35	32	24
Percentage of categories	21.21%	23.64%	21.21%	19.39%	14.55%

.

3.3. Evaluation Metrics

3.3.1. Classification Metrics

For classification tasks, the accuracy, weighted precision, weighted recall, and weighted $F_w$ values are typically used for evaluation of the results.

The accuracy is the ratio of the number of correct samples predicted to the total number of samples, as defined in Equation (11):

$$A=\frac{TP+TN}{TP+TN+FP+FN}.$$

(11)

The weighted precision is the ratio of the total number of truly positive cases to the total number of all samples predicted to be positive, as defined in Equation (12):

$$P_w=\frac{{\sum }_{i=1}^{n}T{P}_i}{{\sum }_{i=1}^{n}T{P}_i+{\sum }_{i=1}^{n}F{P}_i}.$$

(12)

The weighted recall is the ratio of the number of correctly predicted positive samples to the actual number of positive samples, as defined in Equation (13):

$$R_w=\frac{{\sum }_{i=1}^{n}T{P}_i}{{\sum }_{i=1}^{n}T{P}_i+{\sum }_{i=1}^{n}F{N}_i}.$$

(13)

The $F_w$ value is a comprehensive performance index based on precision and recall, as defined in Equation (14):

$$F_w=2\times \frac{P_w\times R_w}{P_w+R_w}.$$

(14)

The concepts of $TP$, $TN$, $FP$, and $FN$ used in the above equations are summarized in

Table 4. Descriptions of $TP,TN,FP$, and $FN$.

True Results	Predicted Results
	Positive Example	Negative Example
Positive example	True positive example ($TP$)	False negative example ($FN$)
Negative example	False positive example ($FP$)	True negative example ($TN$)

.

3.3.2. Regression Metrics

For regression tasks, the coefficient of determination ($R^2$), mean absolute percentage error (MAPE), mean absolute error (MAE), and mean square error (MSE) are commonly used as evaluation metrics. The $R^2$, MAPE, MAE, and MSE are defined in Equations (15)–(18), respectively.

$$R^2=\frac{{\sum }_{i=1}^n{(\widehat{y_i}-\overline{y})}^2}{{\sum }_{i=1}^n{(y_i-\overline{y})}^2}.$$

(15)

$R^2$ is a numerical characteristic used to represent the relationship between a random variable and multiple random variables, as defined in Equation (15). The larger the value, the closer the regression value to the true value.

$$MAPE=\frac{1}{n}\sum _{i=1}^n|\frac{y_i-\widehat{y_i}}{y_i}|,$$

(16)

$$MAE=\frac{1}{n}\sum _{i=1}^n|y_i-\widehat{y_i}|,$$

(17)

$$MSE=\frac{1}{n}\sum _{i=1}^n{(y_i-\widehat{y_i})}^2.$$

(18)

In the above equations, n denotes the number of samples, $y_i$ denotes the true value of the sample, $\widehat{y_i}$ denotes the predicted value of the sample, and $\overline{y}$ denotes the average of the true values of all samples.

3.4. Experimental Results

3.4.1. Learning Rate Test

The learning rate is used to control the learning progress of a model by determining how fast it converges to a local minimum. Equation (19) shows the relationship between the current and previous model weight values for controlling the learning rate $\eta$.

$${\omega }_{new}={\omega }_{old}-\eta ·\frac{\partial L}{\partial {\omega }_{old}},$$

(19)

where ${\omega }_{old}$ denotes the network weights before the update, ${\omega }_{new}$ denotes the updated network weights, $\eta$ is the learning rate. The smaller the value of $\eta$, the slower the loss gradient decreases and the slower the network convergence process.

In this paper, the epochs was set to 300 and the batch size was 1. The weighted balance was applied to the three loss functions to learn the multiple tasks model. The $\eta$ was set to the range of $1\times {10}^{-6}$ to $1\times {10}^{-4}$ to obtain the best learning rate for the optimal results. The results with different learning rate $\eta$ are shown in Figure 4.

Figure 4 showed that the weighted classification accuracy, precision, and $F_w$ value were similar under different $\eta$ values, and the best classification results were obtained when $\eta$ = $1\times {10}^{-5}$. The results are detailed in

Table 5. The optimal results for different evaluation metrics for genotype classification tasks.

Accuracy	${\mathit{R}}_{\mathit{w}}$	${\mathit{P}}_{\mathit{w}}$	${\mathit{F}}_{\mathit{w}}$
0.9688	0.9688	0.9750	0.9674

.

From Figure 4, we can see the best value of $R^2$ was 0.8320 when $\eta$ = $8.4\times {10}^{-5}$ in the leaf number regression task, under which the value of $R^2$ was 0.9241 for the leaf area regression task. The best value of $R^2$ was 0.9787 when $\eta$ = $1\times {10}^{-5}$ in the leaf area regression task, while the $R^2$ value was 0.7944 for the leaf number regression task.

The $R^2$ value for the leaf number regression task decreased when $\eta$ increased from $1\times {10}^{-5}$ to $8.4\times {10}^{-5}$, while the $R^2$ of the leaf area regression task increased. Thus, the hard parameter sharing model with VGG16 was used as the base network and $\eta$ was set to $1\times {10}^{-5}$. Results of each regression task are shown in

Table 6. Regression results when learning rate $\eta$ = $1\times {10}^{-5}$.

Task	${\mathit{R}}^2$	MSE	MAE	MAPE
Leaf Number Regression	0.7944	0.5892	0.5846	0.0691
Leaf Area Regression	0.9787	0.3029	0.4551	0.0328

.

Based on the above parameter settings, the confusion matrix for the genotype classification task is shown in Figure 5.

3.4.2. Comparison with Different Networks

We adopted three different basic networks, i.e, VGG16, ResNet50 [12], and Xception [37] for comparison with the proposed hard parameter sharing model. The number of epochs, the batch size, and $\eta$ were set to 300, 1, and $1\times {10}^{-5}$, respectively.

(1)

Classification task

A comparison of the best classification metrics in the training process for the different basic network models is shown in Figure 6. From Figure 6, it can be seen that our method outperformed the other two basic network models, in terms of the achieved classification evaluation metrics. The classification accuracy and recall of our method were 78.13% and 56.25% higher than those based on ResNet and Xception, respectively. For precision of classification, our method achieved 45.71% and 31.15% higher values than the other two models, respectively. The classification $F_w$ value obtained by the VGG16-based network model was 79.64% and 56.75% higher than those of ResNet and Xception, respectively.

(2)

Regression tasks

Figure 7 shows a comparison of the $R^2$ values obtained using the different basic networks in regression tasks. It can be seen that the model using VGG16 as the basic network performed better in both regression tasks, compared to the other two models. In the leaf number regression task, the $R^2$ of our method was 33.92% and 28.32% higher than those of the models based on ResNet50 and Xception, respectively; and the $R^2$ value of our method in the leaf area regression task was 61.46% and 22.43% higher than those of the other two models, respectively.

3.4.3. Weighted Loss Function Results

Based on the above parameter settings, the values of $\sigma$ associated to the loss functions were continuously updated to balance the relationships between the tasks. The logarithmic values of ${\sigma }^2$ ($\log {\sigma }^2$) for each task during the training process are shown in Figure 8. The ${\sigma }^2$ is used as the denominator to balance multiple tasks, with smaller values representing a larger proportion of the total loss for that task.

In Figure 8, it can be seen that the value of $\sigma$ for the classification task gradually increased in the first 50 iterations, then gradually decreased, while the $\sigma$ value for the leaf number regression task gradually increased in the first 200 iterations and then slowly became smaller. Finally, for the leaf area regression task, $\sigma$ kept increasing. In the early stage of model training, each task may dominate the training of the model and, thus, $\sigma$ increased for all three tasks.

By adjusting the proportion of loss in each task during the training process, the tasks are balanced and individual tasks are prevented from dominating the training process, which can allow the optimal solution for all tasks to be obtained.

(1)

Classification task

Table 7. Results comparison for unweighted loss and weighted loss models in the classification task.

	Accuracy	${\mathit{R}}_{\mathit{w}}$	${\mathit{P}}_{\mathit{w}}$	${\mathit{F}}_{\mathit{w}}$
Unweighted loss	0.9063	0.9063	0.9375	0.9096
Weighted loss	0.9688	0.9688	0.9750	0.9674

provides the optimal classification results with the weighted and unweighted loss models. It can be seen that, for the classification task, the weighted accuracy and recall of the weighted loss model increased by 6.25%, the weighted precision increased by 3.75%, and $F_w$ increased by 5.78% over the unweighted loss model.

(2)

Regression tasks

The regression results for the unweighted and weighted loss models are provided in

Table 8. Results comparison for unweighted loss and weighted loss models in the regression tasks.

Task	Unweighted Loss			Weighted Loss
	R2	MSE	MAE	R2	MSE	MAE
Leaf number regression	0.7171	0.8109	0.7238	0.7944	0.5892	0.5846
Leaf area regression	0.9703	0.4209	0.5465	0.9787	0.3029	0.4551

. The comparison indicates that, for the leaf number regression task, the $R^2$ of the weighted loss model increased by 7.73%, the value of MSE decreased by 22.17%, and the value of MAE decreased by 13.92% compared to the unweighted loss model. For the leaf area regression task, the MSE value of the weighted loss model was reduced by 11.80% and the MAE value was reduced by 9.14% compared to the unweighted one. Results showed that the task-dependent uncertainty to weigh the loss function achieved better results in both classification and regression tasks.

3.5. Comparison with Previous Works

Dobrescu et al. [27] constructed a multi-task learning model with ResNet50 as a hard parameter sharing network, trained using the Ara2013 dataset, and obtained the optimal multi-task prediction model when $\eta$ = $1\times {10}^{-4}$. In the case of the size of the dataset used in this paper, the model proposed by Dobrescu et al. (denoted as DMWL) was trained with 300 epochs, a batch size of 1, and $\eta$ = $1\times {10}^{-4}$. Their model was compared with the best model obtained in this paper.

(1)

Classification task

The classification accuracy results for DMWL and our method on the training set are shown in Figure 9. As can be seen from the figure, our method performed better than DMWL, with higher classification accuracy, in the first 140 iterations, while both models achieved 100.00% classification accuracy on the training set in the latter 160 iterations.

The classification accuracy results for DMWL and our method on the test set are shown in Figure 10. From Figure 10, it can be seen that our method outperformed DMWL on the test set, with its classification accuracy after each iteration almost always being higher than that of DMWL.

DMWL achieved the best classification effect at 278 iterations, which was compared with the best values of the classification evaluation metrics achieved by our method. The comparison histogram is shown in Figure 11. In the classification task, the values of each classification evaluation metric for our method were higher than those obtained by DMWL.

(2)

Regression tasks

For the leaf number regression task, the best $R^2$ values achieved by our method against those of DMWL are shown in Figure 12. As can be seen from the figure, the $R^2$ value obtained with our method was higher than that for DMWL.

For the leaf area regression task, a comparison of the two models in terms of MSE is shown in Figure 13. Figure 13 indicates the MSE values obtained by our method were lower than those of DMWL.

In summary, the best model obtained in this paper demonstrated improved accuracy, recall, and other metrics in the classification task, as well as outperforming a state-of-the-art model in both regression tasks on the considered dataset.

4. Conclusions and Future Work

In this study, we used VGG16 as a base network and implemented hard parameter sharing and task-dependent uncertainty trade-off loss functions for genotype classification, leaf number counting, and leaf area estimation tasks to enable phenotypic analysis of Arabidopsis thaliana. Our method demonstrated better prediction results, with a classification accuracy of 96.88%, precision of 97.50%, and $F_w$ of 96.74%, while also obtaining optimal $R^2$ values of 0.7944 and 0.9787 in the leaf number and leaf area regression tasks, respectively. The multi-task learning models using task-dependent uncertainty trade-off loss functions improved the phenotypic result.

The dataset used in this paper was small, in the future, the dataset can be added. For the acquisition of leaf area labels, methods for obtaining leaf area information directly through images without manual annotation. For the leaf number and leaf area tasks, the task-specific layers can be re-defined to segment each leaf with target detection to obtain the leaf number and leaf area information without requiring manually labeled images.