A Semi-Supervised Object Detection Algorithm Based on Teacher-Student Models with Strong-Weak Heads

Abstract

Semi-supervised object detection algorithms based on the self-training paradigm produce pseudo bounding boxes with unavoidable noise. We propose a semi-supervised object detection algorithm based on teacher-student models with strong-weak heads to cope with this problem. The strong and weak heads of the teacher model solve the quality measurement problem of pseudo label localization to obtain higher-quality pseudo labels. The strong and weak heads of the student model are decoupled to reduce the negative impact of pseudo label noise on classification and regression. We reach 52.5 mAP (+1.8) on the PASCAL visual object classes (PASCAL VOC) dataset and even up to 53.5 mAP (+3.2) by using Microsoft common objects in context (MS-COCO) train2017 as additional unlabeled data. On the MS-COCO dataset, our method also improves about 1.0 mAP with the experimental configurations of 10% COCO and COCO-full as labeled data.

1. Introduction

Object detection has made significant progress with the development of deep convolutional neural networks (DNNs). Training an accurate object detector requires a large and well-annotated dataset. There have been many creative outputs for semi-supervised learning (SSL) in image classification but few academic results for semi-supervised object detection (SSOD).

Pseudo-labeling methods [1,2,3,4] have recently gained considerable attention in SSOD. Specifically, a teacher model generates a set of prediction boxes on unlabeled images and filters out possible pessimistic predictions with a confidence threshold. The remaining boxes are named pseudo-boxes and then used as targets to train student models. However, the noisy pseudo labels lead to two critical problems. First, the confidence comes from the output of the classifier, which can be used to filter the classification but should not be used to filter the localization. Second, mixing pseudo-labels and ground truth labels in a branch for training could cancel out the correct parameters learned from the truth data, leading to performance degradation.

To overcome the above-mentioned problems, we propose a semi-supervised object detection algorithm based on the FasterRCNN [5] framework using teacher-student models with strong-weak heads. From the whole framework, we follow the Mean Teacher [6] paradigm. Mean Teacher [6] works with large datasets and improves the speed of learning and the classification accuracy of the trained network. We migrate it from classification to object detection. Specifically, our teacher model generates pseudo annotations for unlabeled images and filters out high-quality pseudo labels. The student model uses these pseudo labels for semi-supervised training. The student model updates its weights by training, and the teacher updates the weights from the student model by exponential moving average (EMA).

In more detail, we introduce dual heads for the teacher and student models, respectively. We regard the strong head inference results of the teacher model as pseudo labels and prompt the weak head inference results to filter the pseudo labels. Other methods [1,2,3,4] using self-training in SSOD have no way to calculate the intersection over union (IOU) of the pseudo boxes. While there is a one-to-one correspondence between the prediction boxes of our teacher model’s dual heads, the IOU between them can determine the localization quality of the pseudo labels. Compared with others, the two-head structure of our teacher model solves the localization quality measurement problem of pseudo labels.

The strong and weak heads of the student model can better utilize the pseudo and ground truth labels for training. Other self-training approaches [1,2,3,4] use a single branch to mix pseudo labels and ground truth data for training. Such a method causes pseudo label noise to interfere severely with the proper weights learned from the ground truth data, especially in the fully-connected layers that directly determine the final classification and regression output. Compared with others, our student model can extract features from pseudo and ground truth labels by the shared network and reduce the negative impact of noise in low-quality pseudo labels on classification and regression by decoupling the strong and weak heads. Furthermore, we introduce the convolutional block attention module (CBAM) [7] to enhance feature extraction capability of the dual-head shared part in both spatial and channel dimensions.

The main contributions of this paper are as follows:

• We propose a novel semi-supervised object detection algorithm with competitive performance in sufficient experiments on PASCAL VOC and MS-COCO datasets.
• In the teacher model, the strong head inference results could generate pseudo labels, with which the weak head inference results can be used to compute the IOUs to determine the quality of pseudo label location to filter pseudo annotations.
• In the student model, the strong head focuses on clean data (ground truth labels and high-quality pseudo labels), while the weak head can extract features from clean data and dirty data (low-quality pseudo labels) without polluting the strong head weights by noise in dirty ones.

This paper is organized as follows: Section 2 illustrates the related works; Section 3 exhibits the details of our method; we present the experimental results and comparisons to other methods in Section 4; and, finally, Section 5 concludes the paper.

2. Related Work

2.1. Object Detection

Object detection [5,8,9,10,11,12,13,14,15] has received considerable attention in recent years. Object detection can be classified into two categories: two-stage and one-stage. Two-stage object detection methods [5,12,14,16,17] divide the detection process into two steps: firstly, generating a region of interest (ROI) through a region proposals network (RPN) and then classifying and locating in the ROI. One-stage object detection methods [10,11,13] do not need to generate regions of interest but directly generate classification probabilities and regression results in one step. Generally, the two-stage algorithms are more accurate, while the single-stage algorithms are faster. Relying on massive labeled data, these detectors perform well in pedestrian detection [18], vehicle detection [19] and other applications [20,21,22,23,24]. In our work, we develop our framework based on FasterRCNN [5], the classical two-stage object detector. Unlike previous approaches that trained the model only on labeled data, we trained our object detector on labeled and unlabeled data with our proposed semi-supervised learning strategy.

2.2. Semi-Supervised Image Classification

In the field of semi-supervised image classification, there are several recent research results, and two research methods have received much attention. One is self-training [25,26], also known as the pseudo labeling method, a popular method for SSL that generates pseudo labels on unlabeled images to guide the model training with a pre-trained model. Another method is the consistent regularization [27,28,29]. The core idea is to make the same image’s predicted output as consistent as possible after perturbations. These perturbations include model dithering [25], data augmentation [27], and feature-based augmentation [30]. Data augmentation is widely used, which includes Mosaic, Mixup, CutOut, and reinforcement learning [31,32], in addition to rotation, panning, flipping, or color dithering. Rational data augmentation improves the robustness of deep neural networks [33] and dramatically enhances the performance on semi-supervised image classification tasks. Mean teacher approaches [2,4,6,34,35,36] use a dual-head structure for teachers and students, where teachers update their weights from the students’ weights by EMA. We follow some ideas in SSL, but our core aim is to further address the localization problem in SSOD.

2.3. Semi-Supervised Object Detection

Semi-supervised object detection [1,2,3,4,37,38,39,40,41,42,43,44] applies semi-supervised learning methods to object detection by training object detectors on labeled and unlabeled data. Recently, several papers have been published on SSOD. CSD [38] utilizes a consistent algorithm that encourages no change in the output of the original image and the image after perturbation. STAC [1] is a self-training approach with a pre-trained teacher model to generate pseudo-labels on unlabeled images, which are used to train a student model. STAC [1] lays down a paradigm for semi-supervised object detection research, but generating pseudo labels at once limits its performance. Instant-teaching [3] generates pseudo labels and designs a teacher-student model interaction learning algorithm. ISMT [4] considers historical pseudo-labels, but its performance is not superior. All the self-training methods mentioned above use a confidence threshold to coarsely filter pseudo labels, which inevitably leads to poor quality of pseudo label localization. Considering this, Unbiased Teacher [2] does not involve pseudo boxes in the regression training. Nevertheless, this method would waste the valid localization information in the noisy pseudo boxes. In this paper, we continue the research paradigm of STAC [1] and propose a new semi-supervised object detection algorithm. The teacher model dynamically generates pseudo labels for the student model. The two-head prediction boxes of our teacher model can calculate IOU to be a basis for localization filtering. The student model fully extracts features from pseudo labels with a CBAM [7] and decouples strong and weak heads to reduce the negative impact of low-quality pseudo label noise on classification and regression.

3. Methodology

We want to solve the object detection problem under semi-supervised conditions, where $D\mathrm{s}={\{x_i^l,y_i^l\}}_{i=1}^{N^l}$ denotes labeled images and $D\mathrm{s}={\{x_i^u,y_i^u\}}_{i=1}^{N^u}$ denotes unlabeled images. $x_i^l,x_i^u$ denote the ith labeled image and the ith unlabeled image, respectively; $y_i^l,y_i^u$ denote the truth labels of the ith labeled image and the pseudo labels of the ith unlabeled image. The ground truth labels and the pseudo labels consist of classification and regression annotations; the difference is that the teacher model generates the pseudo labels. $N^l,N^u$ denote the number of supervised and unsupervised pictures.

3.1. Method Overview

The framework of the algorithm is shown in Figure 1. First, the strong head of the pre-trained teacher model generates pseudo labels for the weakly data augmentation unlabeled images. Their classification confidences distinguish the classification quality of the pseudo labels by a classification threshold. Both the model structures and the input features of the two heads are the same; consequently, there is a one-to-one correspondence between their regression boxes, whose IOUs could be used to determine the pseudo labels localization quality filtered by the IOU threshold. When training the student model, the strong head learns from clean data (ground truth labels and high-quality pseudo labels), while the weak head learns from both clean data and dirty data (low-quality pseudo labels).

Regarding the whole framework, our teacher-student model consists of two training stages, a pre-training stage, and a semi-supervised training stage. In the pre-training phase, we train the teacher model with a small amount of ground truth data. We copy the initialization parameters of the pre-trained teacher model to the student model before the semi-supervised training. A teacher model is trained in STAC [1] to generate pseudo labels at once and no longer updates the generated pseudo labels, which makes the training of the student model all based on poor quality and non-updated pseudo labels, thus limiting its performance. In our approach, the teacher model dynamically generates pseudo labels for the student model, and the student model updates the parameters to the teacher model via EMA. During the semi-supervised training process, the teacher model’s parameters are continuously updated, generating higher-quality pseudo labels, which can better train student models.

In terms of the internal structure of the student and teacher models, unlike the ordinary FasterRCNN [5] model, we designed a decoupled two-head structure with a shared network. This design differs from using only a single head to train both high-quality and low-quality data or using two completely independent models to train both data separately and finally taking the mean value of their predictions. Compared with the former, the strong and weak heads decoupling in our scheme avoids the interference of low-quality pseudo-label data noise on the classification and regression results; compared with the latter, our approach draws on the idea of multi-task learning to mitigate overfitting. Here, high-quality data refers to ground truth labels and high-quality pseudo-labels, and low-quality data refers to low-quality pseudo-labels, which will be elaborated on later. In the shared part of the two, there is a CBAM [7] to further extract the common features from both channel and spatial dimensions.

3.2. Pre-Training Stage

We train the two-head model using a small amount of truth data in a fully supervised form, with all supervised data going through both strong and weak heads. We randomly initialize the strong and weak heads separately. Specifically, we first use the available supervised data $D\mathrm{s}={\{x_i^l,y_i^l\}}_{i=1}^{N^l}$ to optimize our model parameters with supervised losses $L_{\sup }$. The supervised losses as Equation (1) for object detection include four losses: the RPN classification loss, the RPN regression loss, the ROI strong head classification loss, the ROI strong head regression loss, the ROI weak head classification loss, and the ROI weak head regression loss:

$$L_{\sup }^{}=L_{cls}^{RPN}+L_{reg}^{RPN}+L_{cls}^S+L_{reg}^S+L_{cls}^W+L_{reg}^W$$

(1)

3.3. Semi-Supervised Training Stage

After the pre-training phase, semi-supervised training begins.

In the first stage of semi-supervised training, the teacher model infers the unlabeled images after weak data augmentation to generate pseudo labels and filter them. Specifically, first, the teacher model’s strong head generates candidate pseudo-label boxes $B_S={\left\{b_i^s\right\}}_{i=1}^N$ and confidence scores $C_S={\left\{c_i^s\right\}}_{i=1}^N$, and its weak head generates pseudo-label boxes $B_W={\left\{b_i^w\right\}}_{i=1}^N$ and confidence scores $C_W={\left\{c_i^w\right\}}_{i=1}^N$. The IOUs $IO{U}_{SW}={\left\{io{u}_i^{SW}\right\}}_{i=1}^N$ of $B_S={\left\{b_i^s\right\}}_{i=1}^N$ and $B_S={\left\{b_i^s\right\}}_{i=1}^N$ are calculated. We select the pseudo-label boxes $B_S={\left\{b_i^s\right\}}_{i=1}^N$ and confidence scores $C_S={\left\{c_i^s\right\}}_{i=1}^N$ obtained from the strong head inference as the pseudo labels for the next stage of semi-supervised training of the student model. $C_S={\left\{c_i^s\right\}}_{i=1}^N$ are the basis for judging the classification quality of pseudo labels. $IO{U}_{SW}={\left\{io{u}_i^{SW}\right\}}_{i=1}^N$ are the quality basis for judging the pseudo-label regression boxes, which solves the problem of locating the quality measure of the pseudo-label regression boxes that were not addressed in other papers [1,3,34] because their single-head model structure has no way to find IOUs.

We use the following method to filter the pseudo-labels $B_S={\left\{b_i^s\right\}}_{i=1}^N$ and $B_S={\left\{b_i^s\right\}}_{i=1}^N$. We adopt non-maximum suppression (NMS) to filter the candidate pseudo-label boxes to avoid duplicates. First, $C_S={\left\{c_i^s\right\}}_{i=1}^N$ classify the pseudo labels into high-quality and low-quality classification labels by the confidence threshold τ; $IO{U}_{SW}={\left\{io{u}_i^{SW}\right\}}_{i=1}^N$ classify the pseudo labels $B_S={\left\{b_i^s\right\}}_{i=1}^N$ into high-quality regression labels and low-quality regression labels by the IOU threshold ε. When a set of pseudo-labels is both high-quality classification and high-quality regression labels, we call them high-quality pseudo-labels. When a set of pseudo labels is both low-quality classification labels and low-quality regression labels, it is not eligible for the next semi-supervised training. The remaining pseudo labels are called low-quality pseudo labels. The above method completes the generation and quality distinction of pseudo labels.

The second stage of semi-supervised learning is training the student model with clean and dirty data. We sampled the ground truth and pseudo-labeled images equally and then performed strong data augmentation before delivering them into the student model for training. For the network sharing part, the student model’s RPN network generates proposal boxes for the images and then extracts ROI features based on the proposal boxes utilizing ROI Align [12]. These ROI features are extracted further through CBAM [7]. For the two-head structure of the student model, the features obtained in the previous step are fed into the strong-head and weak-head ROI networks, respectively, and the clean data (ground truth data and high-quality pseudo-labels data) are sent to the strong head only to calculate the loss, while both clean and dirty data (low-quality pseudo-labels data) are sent to the weak head to calculate the loss. The total ROI loss during semi-supervised training includes, classification and regression loss in the strong head of ROI, classification loss in the weak head of ROI, and regression loss in the weak head of ROI. The formula is shown below:

$$L_{semi}^{ROI}=L_{cls}^{ROI-S}+L_{reg}^{ROI-S}+L_{cls}^{ROI-W}+L_{reg}^{ROI-W}$$

(2)

The total loss of the student model in the semi-supervised phase is:

$$L_{semi}^{}=L_{cls}^{RPN}+L_{reg}^{RPN}+L_{semi}^{ROI}$$

(3)

At the end of each iteration of the student model, the teacher model parameters are updated by EMA from the students, and α is the update momentum:

$${\theta }_t\leftarrow (1-\alpha ){\theta }_t+\alpha {\theta }_s$$

(4)

The teacher model, after parameter update, could generate new, higher quality pseudo labels during the next iteration, and these pseudo labels are given to the student model for subsequent training. Unlike the static pseudo labels in STAC [1], our pseudo labels are dynamically updated.

The above is the complete training process, and for the inference process, we only use the strong head of the teacher model.

3.4. Strong-Weak Heads

Semi-supervised object detection methods [1,2,3,4] using pseudo labels cannot get around two key issues: how to measure pseudo label quality and train pseudo labels rationally. In the previous section, we solved the pseudo label quality measurement problem by classifying pseudo labels into high-quality pseudo labels and low-quality pseudo labels based on the dual heads structure of the teacher’s model. Next, we use the two-head structure of the student model to solve the problem of reasonable training for pseudo labels and ground truth labels.

Pseudo labels differ from ground truth labels in that they are noisy and large in number. Treating the truth and pseudo labels equally during training biases the gradient updates toward incorrect pseudo labels noise. This approach cancels out the correct parameters learned from the truth data, which leads to performance degradation. We design strong and weak dual heads to address the negative effect of pseudo labels noise on the network parameters.

We combine all the pseudo and ground truth labels and train them with a weak head. This weak head can learn some classification and localization information from the pseudo label and truth data; however, the noise of the pseudo label in the network limits the performance of the coarse weak head. Therefore, we introduce a strong head to learn the truth data and high-quality pseudo labels separately to avoid the weight contamination of the fully connected layer by noise in the low-quality pseudo labels.

Specifically, the strong and weak heads share the rest of the network except for the ROI head: pseudo labels and ground truth labels are sampled in a 1:1 ratio at each iteration, essentially an oversampling strategy for the truth data. This approach can somewhat solve the sample imbalance problem of pseudo labels and ground truth labels. The strong head of the student model trains only with clean data (ground truth data and high-quality pseudo data), and its weak head trains with both clean and dirty data (low-quality pseudo data). Compared with the single-branch structure, the ROI strong and weak dual-head network of the student model computes separate losses for clean and dirty data, preventing the direct interference of low-quality pseudo labels noise on the weights of the strong head network. Compared with the two completely independent model structures, the shared part of the student network can extract common valid information from clean and dirty data to eliminate the supervisory inconsistency of the two independent model structures. Two completely independent model also have a considerably higher computational complexity than our method.

3.5. CBAM

To further enhance the feature extraction capability of the shared network part, we introduce CBAM [7] before ROIHead, as shown in Figure 2. We take the intermediate feature map $F\in {\Re }^{C\times H\times W}$ as the input with height H, width W, and dimension C attributes. The following Equation can express the whole process:

$$F^{\prime }=M_C(F)\otimes F$$

(5)

$$F^{″}=M_S(F^{\prime })\otimes F^{\prime }$$

(6)

The CBAM [7] sequentially yields a channel attention map $M_C\in {\Re }^{1\times H\times W}$ and a spatial attention map $M_S\in {\Re }^{C\times 1\times 1}$. $\otimes$ represents point multiplication, the attention map is broadcasted accordingly during the multiplication process: channel attention values are broadcasted along the spatial dimension, and spatial attention values are broadcasted along the spatial dimension. $F^{″}$ is the final feature output.

As shown in Figure 3, the input feature maps $F\in {\Re }^{C\times H\times W}$ are pooled, respectively, by maximum pooling layers and average pooling layers. Then, pooling results are sent into the shared fully connected layers. The outputs are added together and activated by sigmoid to obtain the channel attention map $M_C\in {\Re }^{1\times H\times W}$.

As shown in Figure 4, in the spatial attention module, firstly, $F^{\prime }$ goes through maximum pooling and average pooling in the channel dimension. Then, the two pooling results are contacted and convolved. After sigmoid activation, the output is the spatial attention map $M_S\in {\Re }^{C\times 1\times 1}$.

3.6. Data Augmentation

The images fed to the teacher model are weakly augmented data, including only random horizontal flipping. The images fed to the student model are strongly augmented, including single-image augmentation and multiple-image augmentation. Single-image data augmentation is processed within a single image and includes random horizontal flips, photometric distortion, random Gaussian blur, and random erasure. Multiple-image augmentation combines two or more images to enhance a single image, including Mixup and Mosaic.

4. Experiments and Results

4.1. Dataset

We conducted experiments on two datasets, PASCAL VOC and MS-COCO. There are four experimental setups in total: (1) 1%/2%/5%/10% randomly selected data from MS-COCO train2017 as labeled data and the rest as unlabeled data; (2) MS-COCO train2017 (~118 k images) as labeled data and MS-COCO additional (~123 k images) as unlabeled data; the above model performance are evaluated on MS-COCO2017 val; (3) VOC07 train (5011 images) as labeled data and VOC12 trainval (11,540 images) as unlabeled data; (4) VOC07 train (5011 images) as labeled data and VOC12 trainval (11,540 images) plus 20 classes of VOC categories from MS-COCO train2017 as unlabeled data, and model performance are evaluated on the VOC07 test.

4.2. Evaluation Metrics

IOU thresholds are set from 0.5 to 0.95, with 0.05 as the interval. They determine whether the object is detected as foreground or background. The mean average precision (mAP) evaluation metric is used to evaluate the experimental results. The average precision (AP) is the area under a two-dimensional curve with recall rate as the horizontal axis and precision as the vertical axis. The mAP is the average AP of all categories, as shown in Equation (7):

$$mAP=\frac{{{\displaystyle \sum }}_{q=1}^{Q}AP\left(q\right)}{Q}$$

(7)

4.3. Experimental Details

In this paper, we follow STAC [1] using a Faster RCNN [5] with a feature pyramid network, using ResNet-50 [45] initialized by ImageNet pre-training as the backbone network. We use confidence threshold τ = 0.7 and IOU threshold ε = 0.6 as the basis for pseudo-label filtering and use AP50:95 (denoted as mAP) as the evaluation metric and inference on the strong head of the teacher model. Batchsize is 32 with 16 labeled and 16 unlabeled images, and we set the learning rate to 0.02 and use SGD as the optimizer. The EMA coefficient is set to α = 0.0001, and 8 NVIDIA TeslaV100 GPUs are used for all experiments.

4.4. Experimental Results on COCO

Table 1. Experimental results on the COCO dataset.

Method	1% COCO	2% COCO	5% COCO	10% COCO	COCO-Full
Baseline	9.05 ± 0.16	12.70 ± 0.15	18.47 ± 0.22	23.86 ± 0.81	37.63
CSD [38]	11.12 ± 0.15	14.15 ± 0.13	18.79 ± 0.13	22.76 ± 0.09	38.52
STAC [1]	13.97 ± 0.35	18.25 ± 0.25	24.38 ± 0.12	28.64 ± 0.21	39.21
ISMT [4]	18.88 ± 0.74	22.43 ± 0.56	26.37 ± 0.24	30.53 ± 0.52	39.64
Instant-teaching [3]	18.05 ± 0.20	22.45 ± 0.15	26.75 ± 0.05	30.40 ± 0.05	40.20
Unbiased Teacher [2]	20.75 ± 0.12	24.30 ± 0.07	28.27 ± 0.15	31.50 ± 0.10	41.30
Ours	18.70 ± 0.32	24.55 ± 0.22	28.10 ± 0.20	32.50 ± 0.35	42.01

shows the mAP results for IOU = 0.5:0.95 evaluated on MS-COCO 2017 val at different data ratios. The different ratios indicate labeled data and the rest indicate unlabeled data, where COCO-full indicates the full amount of MS-COCO train2017 as labeled data and MS-COCO additional as unlabeled data. The offsets in the table are from the results of multiple sets of experiments with different random seeds. As can be seen from the table, at 1%/2%/5%/10% of labeled data, we all improve about 10 mAP after semi-supervised training compared to the supervised baseline. At 2%/10% of labeled data and COCO-full, our experiments have optimal results. In summary, the experimental results on MS-COCO validate that our method is effective and significantly superior when the labeled data accounts for a smaller percentage.

4.5. Experimental Results on VOC

As shown in

Table 2. Experimental results on the VOC dataset.

Method	Labeled	Unlabeled	AP	AP50
Baseline	VOC07	-	42.6	72.6
CSD [38]	VOC07	VOC12	42.7	76.7
STAC [1]			44.6	77.5
ISMT [4]			46.2	77.2
Instant-teaching [3]			48.7	78.3
Unbiased Teacher [2]			49.7	77.4
Ours			52.5	81.0
CSD [38]	VOC07	VOC12+MSCOCO20cls	43.6	77.1
STAC [1]			46.0	79.1
ISMT [4]			49.6	77.8
Instant-teaching [3]			50.8	78.8
Unbiased Teacher [2]			50.3	78.8
Ours			53.5	82.1

, the superiority of our method is more significant in two experimental settings. When the training set is composed of VOC07 train as labeled data and VOC12 trainval as unlabeled data, our method can improve the AP from 49.7 to 52.5 compared to Unbiased Teacher. When the training set is composed of VOC07 train as labeled data and VOC12 trainval plus MS- COCO train2017 in the VOC category of 20 classes as unlabeled data, we improve the AP from 50.3 to 53.5.

4.6. Ablation Experiments

For each of the following studies of ablation experiments, we used 10% COCO as the data configuration.

4.6.1. Multi-Head Ablation Experiment

Our method produces four sets of predictions:

• A strong head of the teacher model.
• A weak head of the teacher model.
• A strong head of the student model.
• A weak head of the student model.

The prediction results of these heads are shown in

Table 3. Ablation experimental results of strong and weak heads for teacher-student models.

Head	mAP	CBAM/mAP
Teacher-Weak	31.9	32.1
Teacher-Strong	32.2	32.5
Student-Weak	29.9	30.1
Student-Strong	30.3	30.5

. From this table, it is clear that the strong and weak head performance of the teacher model is stronger than the strong and weak head performance of the student model, respectively; the strong head performance is better than the weak head performance for both the teacher and student models. This is why we adopt the strong head inference of the teacher model. As can be seen from Table 3, CBAM has a 0.2–0.3 point improvement for all heads.

4.6.2. Data Augmentation Ablation Experiment

We investigate a variety of data augmentations. As shown in

Table 4. Data augmentation ablation experiment results.

Method					AP
Others	√	-	-	-	31.1
CutOut	√	√	-	-	31.3
Mixup	√	√	√	-	32.3
Mosaic	√	√	√	√	32.6

, our method benefits from stronger augmentation. Others refer to random flips and Gaussian blur. Of all the data augmentation methods, Mixup is the most effective, improving performance from 31.3 mAP to 32.3 mAP, and Mosaic further improves by 0.3 mAP.

4.7. Visualization

Figure 5 shows the visualization results. The red boxes represent the prediction, and the blue boxes represent the ground truth. From top to bottom, Figure 5 shows the prediction results of the baseline (row 2, column 5 in Table 1), CSD (row 3, column 5 in Table 1), STAC (row 4, column 5 in Table 1), and our method (row 8, column 5 in Table 1). From the four sets of images, it is evident that our method’s location quality is higher than the other three methods, such as the truck, the bus, and the fire hydrant. Compared with STAC [1] and CSD [38], we can improve the pseudo label localization quality and train these pseudo label boxes appropriately. As seen from the two sets of images in the figure’s third and fourth columns, our method can detect densely distributed objects, such as cars in the fourth column. However, our performance needs to be improved for tiny objects with few pixels, such as the “tiny” person in the fourth column.

5. Conclusions

The most important contribution of this paper is to address two core problems not tackled by other self-training methods [1,2,3,4] in SSOD. Other approaches filter pseudo labels only by classification confidence, so their quality of pseudo label localization is poor. Their strategy to mix pseudo and ground truth labels for training further aggravates the training because of the noise in the pseudo labels. We propose a semi-supervised object detection algorithm using a teacher-student model with strong-weak heads for solving the pseudo label quality measurement and pseudo data training problems. The quality measurement problem of pseudo label localization is solved by the strong and weak heads of the teacher model. The shared part of the student model is used to learn the common features in the pseudo and ground truth data, and the strong and weak heads of the student model are decoupled to reduce the negative impact of noise on classification and regression in pseudo labels. We have conducted sufficient experiments on PASCAL VOC and MS-COCO datasets and achieved superior performance. We reach 52.5 mAP (+1.8) on the VOC dataset and 53.5 mAP (+3.2) with MS-COCO train2017 as additional unlabeled data. Our method also improves about 1.0 mAP on the MS-COCO dataset with the experimental configurations of 10% COCO and COCO-full as labeled data. The visualization results visually illustrate the superiority of the localization quality of our method compared with others. We will focus on how to improve the detection performance of tiny objects in future work.