HDetect-VS: Tiny Human Object Enhancement and Detection Based on Visual Saliency for Maritime Search and Rescue

Abstract

Strong sun glint noise is an inevitable obstruction for tiny human object detection in maritime search and rescue (SAR) tasks, which can significantly deteriorate the performance of local contrast method (LCM)-based algorithms and cause high false alarm rates. For SAR tasks in noisy environments, it is more important to find tiny objects than localize them. Hence, considering background clutter and strong glint noise, in this study, a noise suppression methodology for maritime scenarios (HDetect-VS) is established to achieve tiny human object enhancement and detection based on visual saliency. To this end, the pixel intensity value distributions, color characteristics, and spatial distributions are thoroughly analyzed to separate objects from background and glint noise. Using unmanned aerial vehicles (UAVs), visible images with rich details, rather than infrared images, are applied to detect tiny objects in noisy environments. In this study, a grayscale model mapped from the HSV model (HSV-gray) is used to suppress glint noise based on color characteristic analysis, and large-scale Gaussian Convolution is utilized to obtain the pixel intensity surface and suppress background noise based on pixel intensity value distributions. Moreover, based on a thorough analysis of the spatial distribution of objects and noise, two-step clustering is employed to separate objects from noise in a salient point map. Experiments are conducted on the SeaDronesSee dataset; the results illustrate that HDetect-VS has more robust and effective performance in tiny object detection in noisy environments than other pixel-level algorithms. In particular, the performance of existing deep learning-based object detection algorithms can be significantly improved by taking the results of HDetect-VS as input.

1. Introduction

Currently, unmanned aerial vehicles (UAVs) are employed in object detection for maritime search and rescue (SAR) tasks. Due to the large field of view of UAVs, the observed objects are too small for high-resolution images. In addition, the common phenomenon of sun glint on the sea surface severely affects the visual saliency of tiny objects. In computer vision, the detection of tiny objects is an inherently difficult task for conventional and deep learning-based detectors. Furthermore, glint noise can degrade the quality of captured images and significantly weaken the features of objects when the latter are concealed by sun glint, further complicating tiny object detection. All the above issues make tiny object enhancement and detection based on visual saliency a meaningful task for maritime SAR in noisy environments.

Studies on object detection for maritime SAR mainly focus on radar signals [1], infrared images [2], and visible images [3]. The radar devices employed in maritime environmental perception are shore-based and airborne [4]. The objects detected using these devices based on the methods of signal processing, deep learning [5], and graph convolutional networks [6] mainly include vessels, floating ice, aircraft, and boats. The carriers employed for infrared imaging are UAVs [7] and vessels [8]. The objects detected in infrared images via methods of intelligent algorithms [9] and signal processing [10,11] are mainly low-speed vessels [7,9]. For the detection of tiny human objects easily concealed by noise signals, the use of visible images with rich colors, concrete shapes, and texture details is more effective than that of radar and infrared images.

Sea surface sun glint shows strong reflection characteristics, which can significantly influence UAV imaging quality, making this phenomenon one of the causes of strong noise in tiny human object detection.

A large number of infrared-based methods for maritime object detection based on visual saliency have been proposed. In single-frame detection methods, which have been widely applied because of the need for less prior knowledge compared with sequential detection methods [12], the single frame can be further divided by using background-characteristic-based [13,14], object-characteristic-based [15,16], and background–object-characteristic-combination-based [17,18] methods. Background-characteristic-based methods are based on the estimation of the difference between the original image and the background. Object-characteristic-based methods are based on a saliency map constructed by using the local information of the objects [15,16] and can be used to efficiently detect tiny objects based on infrared images. However, these methods are very sensitive to complex noise and show poor performance in tiny object detection in environments with complex and noisy maritime backgrounds, as high false alarm rates are caused by such background conditions.

In tiny human object detection with strong noise, background noise suppression and object enhancement are crucial for methods based on visual saliency. The pixel intensity value in grayscale and infrared images can reflect the saliency of the object. However, strong noise generally has a similar pixel intensity value to that of tiny objects, which can cause a high false alarm rate. Tiny objects and noise in visible images may have similar saliency to those in grayscale images, but they usually have different color characteristics in the RGB channels, which can be used for tiny object enhancement in maritime environments with strong glint noise.

In deep learning-based algorithms, performance is significantly worse when tiny objects are concealed by sun glint. Since noise can be introduced during image restoration, care should be taken when studying image denoising and reflection removal in images with tiny objects. In addition, strong glint noise can make a tiny object not noticeable when employing deep learning-based methods using feature learning [19,20].

With the development of computer vision in recent years, UAVs’ capability to automatically detect and recognize human attributes has become more relevant in the context of 24/24 h surveillance [21,22,23,24,25,26,27]. SeaDronesSee [27] is a large-scale dataset with high-resolution visible images captured by UAVs in which human targets are generally concealed by sun glint. This dataset has prompted research on tiny target detection for maritime SAR based on UAVs [3,28,29,30,31]. On this basis, we developed a method for sea surface sun glint suppression to achieve the enhancement and detection of tiny targets based on visual saliency.

In summary, the existing methods for maritime human detection based on visual saliency present the following issues:

• When tiny human targets are concealed by sea surface sun glint, saliency-based methods have high false alarm rates, and deep learning-based methods have low recall.
• When solely based on infrared and grayscale images, the saliency values of strong glint noise and targets are too similar to achieve noise suppression and object enhancement.
• Object detection based on visual saliency is generally carried out on single-channel images, while visible images with tiny objects in maritime scenarios are seldom used for denoising and tiny object enhancement. Further, color characteristics are not studied effectively in maritime human target detection using saliency-based methods.

To solve the above-mentioned problems, we propose a robust method to achieve maritime tiny human object detection in sun glint-containing images based on visual saliency (HDetect-VS), as shown in Figure 1. The contributions of this study are summarized as follows:

• Tiny object enhancement and detection under sea surface sun glint conditions is carried out based on visible images rather than single-channel images (infrared and grayscale images). The color characteristic differences between noise and objects are utilized to suppress strong glint noise and enhance object features.
• HSV-gray, a method based on color characteristic difference, is used to suppress strong sea surface sun glint. Large-scale Gaussian Convolution is employed to suppress background noise based on the characteristics of the pixel intensity value. Two-step clustering is employed to achieve tiny object detection and localization based on the spatial distribution characteristics of object and noise saliency.
• A simple and effective method based on visual saliency is used to perform maritime tiny target detection. Our experiment results on SeaDronesSee demonstrate that the proposed method is robust in noisy maritime environments with respect to detection accuracy. By taking the HDetect-VS results as prior knowledge, existing deep learning-based methods can achieve a significant improvement in detection performance.

Figure 1. Tiny human target detection method for maritime SAR based on visual saliency (HDetect-VS).

Figure 1. Tiny human target detection method for maritime SAR based on visual saliency (HDetect-VS).

2. Related Works

2.1. Tiny Object Detection in Maritime Scenarios

The significant difference in pixel intensity values between tiny objects and background is crucial in object detection based on visual saliency. The method based on pixel intensity difference is generally applied to single-channel infrared images and rarely to grayscale visible images. Compared with maritime scenarios, the background in field scenarios is relatively complicated, which leads to a low signal-to-noise ratio around tiny objects, which in turn makes it difficult to distinguish objects from strong noise and to use the saliency-based method to obtain excellent performance. In maritime UAV-captured images, sun glint, waves, and haze can diminish the saliency of objects, while flat background and monochromatic characteristics inherently enhance this characteristic in tiny objects with distinctive colors. Thus, under favorable conditions, maritime tiny human object detection can be performed based on visual saliency and the analysis of background and object characteristics.

Generally, research on object detection is based on the local contrast measurement mechanism (LCM) [32] and its derivative methods [16,33,34,35]. The local differences in pixel intensity values between objects and the background in grayscale imagery are analyzed to highlight the saliency of objects; then, object localization can be performed by setting a threshold. The methods based on visual saliency are generally applied in infrared images with flat backgrounds. However, it is difficult to limit the high probability of false alarms caused by sun glint [36,37]. Multi-scale average absolute gray difference (MS-AAGD) and Laplacian of point spread function (LoPSF) are utilized to suppress false alarm sources [38]. However, when object signals are concealed by strong noise, it is more difficult to detect them based on pixel intensity difference. Therefore, in maritime scenarios, it is much more difficult to localize tiny human targets in noisy infrared images, while target detection can be achieved in visible images with rich details.

Deep learning-based methods are well suited for real-time object detection, with excellent performance in generic scenarios. In images captured by UAVs, high-resolution characteristics and tiny-sized objects complicate object detection. Under limitations of computing and time resources, it is difficult for deep learning-based algorithms to directly train models based on high-resolution images. Generic algorithms scale the input images to a fixed size to reduce computing and time costs, which inevitably causes a reduction in the detail level of features. Moreover, the difference between the theoretical receptive field and the effective receptive field is intensified for tiny objects, further leading to the reduction in positive target samples [39,40].

Genetic models [41] show much better performance on large and medium objects than on tiny objects. To improve performance in tiny object detection, data augmentation [42,43,44,45] and feature fusion [46,47] have been proposed to enhance the learning and presentation of features. However, the details that can be learned in the receptive field for tiny objects are insufficient; therefore, the problem of poor and weak features cannot be effectively overcome. A solution is represented by generative adversarial networks (GANs) [47], which transform the original poor and weak features of tiny objects into highly discriminative ones by introducing fine-grained details from lower-level layers.

In tiny object detection tasks in maritime SAR, the negative effect of sun glint on object texture needs to be addressed. When objects are concealed by sun glint, model performance is significantly poorer due to their weak features. In real-world SAR tasks, it is more important to find objects reliably, rather than accurately localizing them [48]. According to the evident clustering in maritime object distribution, we developed a two-step strategy to detect tiny objects in maritime SAR. The results of HDetect-VS can be used as prior knowledge to improve the detection performance of existing deep learning-based methods.

2.2. Image Processing for Glint Suppression

Because of the variable trajectory and view of UAVs, inevitably, there are glint areas with high pixel intensity values in maritime UAV images, which can significantly influence the visual saliency of objects.

Polarizers, the devices conventionally employed for image sensing, can suppress sea surface sun glint noise but can be significantly influenced by the wind speed, wind direction, and solar zenith angle [49]. Moreover, the application of polarizers is generally based on imagery time-domain polarization characteristics [50] and sensor information [51]. Research on glint suppression based on deep learning originates from image restoration, which is used to refine the poor image quality caused by improper exposure of the primary objects in the image; the drawback of this process is the introduction of noise, which can reduce the signal-to-noise ratio and weaken the features and saliency of objects. Therefore, employing image restoration for denoising is not appropriate for tiny human detection in maritime scenarios.

Despite sun glint and objects in maritime scenarios having high pixel intensity values, based on the difference in their spatial distribution and color characteristics, HDetect-VS can suppress the saliency of sun glint and enhance that of objects.

3. Sea Surface Sun Glint Suppression and Tiny Human Object Detection Based on Visual Saliency

3.1. Characteristic Analysis of Maritime UAV Images

In SAR tasks, compared with field scenarios, in maritime scenarios, the background is relatively flat and generally includes the elements of sea surface, glint, and sky. The visual characteristics of the background and objects in maritime UAV images are as follows:

• For the areas of sea surface and sky, there is almost no large-area shading caused by occlusion. The color distribution of the whole image is uniform, with few color mutations.
• Sun glint on the sea surface, which is due to the variable trajectory and view of UAVs, appears white in RGB images. The visual saliency of tiny objects concealed by sun glint is decreased significantly. However, the color and brightness of maritime objects are different from those of sun glint and background.
• Because of the large field of view of UAVs, maritime objects generally present a clustered spatial distribution.

The characteristics at the pixel level reflecting the above visual features are as follows:

• The pixel intensity values of the sea surface and sky change continuously and evenly on the grayscale, and those of objects such as human targets and boats differ from those of the background, which can form pixel peaks or valleys.
• The pixel intensity values of sun glint on the grayscale and in different RGB channels are close to 255. Objects also have high pixel intensity values on the grayscale, and when they are concealed by sun glint, it is difficult to separate them from glint noise on the grayscale. However, tiny objects have different pixel distributions in the RGB channels compared with sun glint.

As shown in Figure 2, the high pixel intensity values of sun glint and sky on the grayscale significantly interfere with object detection based on pixel saliency. Against backgrounds that appear as pixel surfaces, objects have much higher pixel intensity values and are well distinguishable, but when they are concealed by sun glint, their visual saliency decreases significantly. Therefore, the difficult task of separating the salient points of objects and glint to achieve tiny object detection based on visual saliency needs to be addressed.

3.2. Sea Surface Sun Glint Suppression and Object Saliency Enhancement

3.2.1. Glint Suppression Based on HSV-Gray Method

Because of the variable trajectory of UAVs, sea surface sun glint inevitably appears in visible images. To achieve object saliency enhancement, the high pixel intensity values of glint and sky should be suppressed based on the characteristics of pixel distribution in the RGB channels.

Both glint and objects have high pixel intensity values on the grayscale, which cannot reflect the color characteristics of maritime objects. However, the difference in the pixel distributions of objects and glint noise in the RGB channels is significant. Specifically, in different RGB channels, a colored object has distinct pixel intensity values, while white glint presents no significant differences, with all values being close to 255.

In the HSV (Hue, Saturation, and Value) model, based on human vision, different HSV channels numerically represent the color characteristics of objects. Therefore, the saliency of objects can be enhanced and that of glint noise can be effectively suppressed by using a grayscale mapped from the HSV model (HSV-gray).

As shown in Figure 3, different colors have distinct values in the HSV channels. The values of white are zero in the H and S channels and maximum in the V channel, while the value of black is zero in all channels. Red and green have maximum values in the S and V channels and different values in the H channel because of the different color properties. We map the H, S, and V channels to the R, G, and B channels, respectively. HSV-gray is calculated according to the mapped HSV model as a common grayscale calculated based on the RGB model.

On the grayscale calculated based on the RGB model (RGB-gray) and mapped HSV model (HSV-gray), the pixel intensity value of black is zero. The pixel intensity values of white are 9.69 in HSV-gray compared with 255 in RGB-gray. Red and green have much higher pixel intensity values compared with white in HSV-gray. Therefore, HSV-gray can be applied to enhance the saliency of objects concealed by the glint noise.

The pixel intensity values are calculated based on HSV-gray, and the mapped results are presented as P_HSV-gray. The relationship between P_HSV-gray and the pixel intensity values in different RGB channels is shown in Figure 4. Individual colors have distinct differences in pixel intensity values in different RGB channels. They present much higher P_HSV-gray values than the colors that have similar values in the RGB model. Therefore, glint with similar values in different RGB channels has a much lower P_HSV-gray value than objects with bright colors, which allows the saliency of the glint area to be suppressed.

3.2.2. Maritime Background Suppression Based on Large-Scale Gaussian Convolution

Because of the view and trajectory variation of UAVs, the sea surface and sky, forming the background with high pixel intensity values, significantly influence the saliency of objects. To suppress this effect, the concept of pixel surface based on large-scale Gaussian Convolution was developed.

Gaussian Convolution is the weighted average of the intensity of adjacent positions, with weights decreasing with the spatial distance from the center position (p). The influence that one pixel has on another thus depends only on their distance (r) in the image [52]. As shown in Equation (1), an image I filtered with Gaussian Convolution is given as GC[I]_p, and the weight for pixel q is defined as G_σ(||p − q||) as follows:

$$\begin{array}{l}GC{\left[I\right]}_{\mathrm{p}}={\displaystyle \sum _{\mathrm{q}\in S}{G}_{\sigma }\left(‖\mathrm{p}-q‖\right)I_{\mathrm{q}}}\\ G_{\sigma }\left(r\right)=\frac{1}{2\pi {\sigma }^2}\exp \left(-\frac{r^2}{2{\sigma }^2}\right)\end{array}$$

(1)

where G_σ(r) denotes the 2D Gaussian kernel, σ is a parameter defining the standard deviation, and S represents all the possible image locations.

The weight and the spatial distance can be changed by adjusting the Gaussian kernel size (r_max) and the standard deviation (σ). The edge and saliency of an object are lost with high r_max and σ because averaging is performed over a much larger area. Then, the image is blurred to obtain a smooth pixel surface (P_gauss).

$$\begin{array}{l}{H}_{\mathrm{RGB}}=H\left(I_{\mathrm{RGB}}\right)\\ P_{\mathrm{HSV}-\mathrm{gray}}=Gray\left(H_{\mathrm{RGB}}\right)\end{array}$$

(2)

$$P_{gauss}=GC\left(H_{RGB}\right)$$

(3)

$$P_{saliency}=255\frac{P_{HSV-gray}-P_{gauss}}{\max \left(P_{HSV-gray}-P_{gauss}\right)}$$

(4)

As shown in Equation (2), the HSV mapping function, H(I), is applied to the visible image (I_RGB) to obtain the mapped HSV image (H_RGB). The mapped gray intensity values (P_HSV-gray) are calculated with a common grayscale function. The representations of P_HSV-gray are shown as the first column in Figure 5. Large-scale Gaussian Convolution is applied to the maritime UAV images according to the object size, and the standard deviation (σ) is much larger than the latter. The 4σ Gaussian kernel size is applied to obtain the Gaussian pixel surface (P_gauss) given in Equation (3). The value of σ is about 2 or 3 times that of tiny objects. In this study, we set σ = 40. The pixel surface results are shown in the second column in Figure 5. The difference between P_HSV-gray and P_gauss is object saliency, P_saliency, given in Equation (4). The representations of P_saliency are shown in the third column in Figure 5.

As shown in Figure 5, P_HSV-gray shows that the color characteristics of an object can be effectively highlighted, and P_gauss demonstrates that the high pixel values around objects can be effectively suppressed with large-scale Gaussian Convolution. Finally, P_saliency-3D illustrates that the pixel intensity values of background such as the sea surface and sky can be significantly suppressed and tiny human targets highlighted.

3.3. Maritime Object Detection Based on Two-Step Clustering

As shown in the representation of object saliency in Figure 5, the saliency of the maritime tiny human objects is relatively low compared with that of boats and other objects. In addition, the pixel intensity values of the set of background noise signals are much higher and close to the saliency of the targets. To perform object localization, an adaptive threshold is applied for preliminarily screening of the salient points in the whole image. Namely, localization is initially conducted based on the pixel intensity values greater than the adaptive threshold; then, the salient points of objects and partially strong noise are separated by employing two-step clustering to achieve maritime tiny object detection based on visual saliency.

The salient points identified based on the adaptive threshold mainly comprise object and noise salient points, where the latter can be further divided into glint noise and background noise. Using a lower threshold can generally identify the salient points of tiny human objects, but a set of background noise signals with higher pixel intensity values can also be introduced, as shown in Figure 6b.

The differences in spatial distribution characteristics between object and noise salient points can be used for maritime UAV image denoising. The spatial distribution characteristics of different salient points are as follows: Generally, background noise salient points are represented as outliers, as the amount of background noise is small and sparsely distributed in the whole image. The amount of glint noise is much larger and centrally and uniformly distributed in the area, with high pixel intensity values; however, the density of glint noise in the area is low. By comparing the uniform and sparse distribution characteristics of glint and background noise, respectively, we can conclude that their salient points are in relatively high quantity and their density characteristics are well defined.

DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is an unsupervised clustering method based on minimum density estimation [53]. All the points are clustered by setting two parameters of the algorithm: the distance radius (ε) and the minimum quantity of clustered points (P_min). Points that do not belong to any clusters can be considered outliers.

Background noise, glint noise, and objects have different spatial distribution characteristics. Therefore, two-step DBSCAN clustering is carried out to successively eliminate background and glint noise.

The first step of clustering, based on large ε and small P_min, is to eliminate background noise that is sparsely distributed in the whole image. ε = 150 and P_min = 50 were used in this study for this first step. A large ε value can identify the points distributed in local areas such as object clusters and glint clusters. The P_min for this step is much smaller than those of object and glint clusters but larger than that of the background cluster. Thus, the points of background noise are too far from each other to satisfy the requirement of P_min within the distance of ε for being clustered.

The second step of clustering, based on small ε and large P_min, is to eliminate glint noise, which is centrally and uniformly distributed. ε = 2 and P_min = 20 were used in this study for this second step. A small ε can identify object clusters with high density. For this step, ε is smaller than the distance among glint noise points. The points of glint noise are too far from each other to satisfy the requirement of P_min within the distance of ε for being clustered.

As shown in Figure 6c, the salient points of the objects in the original visible image were effectively separated by using two-step DBSCAN clustering. The area of the objects, especially including tiny human targets, is indicated by the red box. It is essential to appropriately set the parameters ε and P_min in two-step DBSCAN clustering, since objects are generally concealed by sun glint and background noise.

4. Experiments

4.1. Dataset

Experiments were conducted on the dataset of Object Detection V2 of SeaDronesSee [27], which is a large-scale dataset aimed at helping to develop systems for SAR by using UAVs in maritime scenarios. The dataset includes high-resolution visible images captured by a UAV under different field-of-view and flight height conditions, with 8930 images for training and 1547 images for validation. There are five classes in the dataset, i.e., swimmer, boat, jet ski, lifesaving appliances, and buoy, where the swimmer class represents about 64% of the total and most of the objects are tiny objects.

4.2. Experimental Settings

The experiments included two parts: tiny object detection based on visual saliency (HDetect-VS) was conducted on the training and validation sets of Object Detection V2; then, object detection based on YOLOX [54] was conducted based on the results of HDetect-VS. This experiment was carried out on a computer with an NVIDIA A30 (NVIDIA, Santa Clara, CA, USA) and training was performed with PyTorch (v.3.8.1) [55].

Given the wide field of view of UAVs, maritime objects are generally densely distributed in certain areas. The proposed HDetect-VS method can be used to perform area localization, consisting in determining boxes that include maritime multi-scale objects. For the evaluation of the algorithm, we employed the Aerial Detection Ratio (ADR) between the number of objects in the box and all ground-truth objects. As shown in Figure 7, there are three boxes in the results: a mismatched box with no objects, an error-matched box with part of the objects, and a well-matched box with all the objects. Considering the size of an input image in deep learning-based object detection, the sizes of the boxes were set to multiples of 640 × 640.

4.3. Analysis of Experimental Results

• Object detection based on visual saliency (HDetect-VS)

The results of the experiment conducted on Object Detection V2, shown in

Table 1. Results of tiny object detection based on visual saliency (Object Detection V2).

Class	Number of Ground-Truth Objects	Number of Detected Objects	ADR
Swimmer	6206	5973	96.25%
Boat	2214	2091	94.44%
Jet ski	320	319	99.69%
Lifesaving appliances	330	330	100.00%
Buoy	560	507	90.54%
Val. total	9630	9220	95.74%
Swimmer	37,096	35,679	96.18%
Boat	13,022	12,104	92.95%
Jet ski	2330	2306	98.97%
Lifesaving appliances	923	920	99.67%
Buoy	4389	4237	96.54%
Train. total	57,760	55,246	95.65%

, indicate that the ADRs for all classes and for tiny human targets were over 95% and over 96%, respectively, on both the training and validation sets. This indicates that the proposed method based on glint suppression and pixel surface can effectively enhance the saliency of objects. The density characteristics of maritime object distribution can be utilized to perform tiny human target detection.

First, to achieve the suppression of sea surface sun glint, we applied the HSV-gray method to effectively suppress the high pixel intensity values for areas that presented white in the RGB images. However, since the boat objects in SeaDronesSee also have white characteristics, their saliency was also slightly suppressed, resulting in a low ADR for boats. On the contrary, the color characteristics of lifesaving appliances and jet ski objects were much more defined, resulting in relatively high ADRs.

Second, to comprehensively evaluate the effectiveness of HDetect-VS results as prior knowledge, as shown in Table 2, objects that fail to be detected should be considered in the result of the deep learning-based algorithm.

The results of tiny object detection for SAR obtained with different algorithms are compared in Figure 8.

Improved Local Contrast Measure (ILCM) [15], MS-AAGD, and HDetect-VS were used to perform tiny object detection in a normal maritime environment. The performance and edge quality of the objects detected with ILCM were better than the results of MS-AAGD and HDetect-VS. When sun glint appeared in an image, ILCM could not distinguish between glint noise and objects, and when the objects were concealed by sun glint, as in the Glint images in Figure 6 and Figure 8, ILCM failed to detect them, generating false alarms.

Compared with the ILCM algorithm, MS-AAGD slightly suppressed the dense glint in Glint 1 and Glint 2 in Figure 8. However, the effect of MS-AAGD on scattered glint suppression was comparatively small, as illustrated for Glint 3 in Figure 8. It should be noted that MS-AAGD is sensitive to the edge of tiny objects but not to their color characteristics, as illustrated for Glint 3 in Figure 8, in which the buoy was not detected.

Our proposed HDetect-VS algorithm achieved tiny object detection in strong glint, as shown in Figure 6 and Figure 8. This indicates that the performance of HDetect-VS is robust and effective for maritime SAR. The edge quality of the detected objects was slightly poorer compared with ILCM and MS-AAGD. However, in maritime SAR, it is more important to find tiny objects concealed by strong noise signals than accurately localize them.

Figure 8. Comparison of results of tiny object detection algorithms for maritime SAR. Yellow points in Glint Crop of input images represent locations of tiny objects.

Figure 8. Comparison of results of tiny object detection algorithms for maritime SAR. Yellow points in Glint Crop of input images represent locations of tiny objects.

• Benchmark experiment of object detection based on YOLOX (wo-Prior)

This experiment was conducted on Object Detection V2 (wo-Prior), aiming to provide a benchmark for object detection conducted on the results obtained with HDetect-VS.

In Table 2, we report the results obtained with wo-Prior for maritime object detection based on the multi-scale YOLOX model with different input image sizes. The variation in 0.5–0.95 mAP was about 0.5–1 points. By increasing the input image size from 640² to 1024², for different classes, AP increased by 4–8 points and mAP by 4 points. It can be noted that increasing the level of detail of objects is more conducive to performance improvement compared with increasing the size of the feature map based on a multi-scale model. Moreover, the improvement for tiny objects is much higher than that for medium and large objects.

The performance ceiling for tiny objects for a certain input image size is much lower than that for medium and large objects. Therefore, increasing the level of detail of objects is essential for performance improvement in tiny object detection under conditions of a wide field of view.

• Object detection on prior results based on YOLOX (w-Prior)

The experiment was conducted based on the area results obtained with HDetect-VS (w-Prior), aiming to effectively increase the details of tiny objects.

As shown for w-Prior in Table 2, maritime object detection performance based on the multi-scale model with different input image sizes showed significant improvements. The improvement for 640² images was much higher than that for 1024² images. The improvement in 0.5 mAP was relatively high: 6 points for the average and 10.3 points for the maximum. The improvement in 0.5–0.95 mAP was about 4–6 points due to the improvement for tiny objects, such as 13–17 and 2–6 AP points for the lifesaving appliances and swimmer classes, respectively. By contrast, the improvements in AP for the boat and jet ski classes were much lower.

Table 2. Deep learning method results of maritime object detection on prior results obtained with our methodology.

Model			mAP		AP
Experiment	Size	YOLOX	IoU: 0.5	IoU: 0.5–0.95	Swimmer	Boat	Jet Ski	Lifesaving Appliances	Buoy
wo-Prior	6402	s	79.2	45.5	31.62	69.58	52.85	24.89	48.46
		m	77.5	46.5	33.21	71.23	55.93	22.48	49.90
		l	76.9	47.0	32.93	72.56	57.34	21.60	50.59
	10242	s	82.7	50.7	37.10	74.05	56.33	29.27	56.66
		m	81.0	50.9	38.35	74.47	56.94	26.18	58.38
		l	81.3	51.3	37.96	75.41	58.28	27.34	57.33
w-Prior	6402	s	85.3	51.7	36.69	70.94	56.85	38.91	54.88
		m	83.7	50.9	37.70	70.24	57.61	35.34	53.50
		l	87.2	53.2	38.90	72.91	60.07	36.48	57.62
	10242	s	87.1	54.7	39.41	72.52	59.33	42.86	59.55
		m	88.1	54.8	40.00	72.68	58.61	43.78	59.11
		l	89.3	56.2	41.12	74.28	60.46	45.16	60.15
w-Prior Total	6402	s	81.7	49.5	35.31	67.00	56.67	38.91	49.69
		m	80.1	48.7	36.28	66.34	57.43	35.34	48.43
		l	83.5	50.9	37.44	68.86	59.88	36.48	52.17
	10242	s	83.4	52.4	37.93	68.49	59.15	42.86	53.91
		m	84.3	52.5	38.50	68.64	58.42	43.78	53.52
		l	85.5	53.8	39.58	70.15	60.27	45.16	54.46

Table 2. Deep learning method results of maritime object detection on prior results obtained with our methodology.

Model			mAP		AP
Experiment	Size	YOLOX	IoU: 0.5	IoU: 0.5–0.95	Swimmer	Boat	Jet Ski	Lifesaving Appliances	Buoy
wo-Prior	6402	s	79.2	45.5	31.62	69.58	52.85	24.89	48.46
		m	77.5	46.5	33.21	71.23	55.93	22.48	49.90
		l	76.9	47.0	32.93	72.56	57.34	21.60	50.59
	10242	s	82.7	50.7	37.10	74.05	56.33	29.27	56.66
		m	81.0	50.9	38.35	74.47	56.94	26.18	58.38
		l	81.3	51.3	37.96	75.41	58.28	27.34	57.33
w-Prior	6402	s	85.3	51.7	36.69	70.94	56.85	38.91	54.88
		m	83.7	50.9	37.70	70.24	57.61	35.34	53.50
		l	87.2	53.2	38.90	72.91	60.07	36.48	57.62
	10242	s	87.1	54.7	39.41	72.52	59.33	42.86	59.55
		m	88.1	54.8	40.00	72.68	58.61	43.78	59.11
		l	89.3	56.2	41.12	74.28	60.46	45.16	60.15
w-Prior Total	6402	s	81.7	49.5	35.31	67.00	56.67	38.91	49.69
		m	80.1	48.7	36.28	66.34	57.43	35.34	48.43
		l	83.5	50.9	37.44	68.86	59.88	36.48	52.17
	10242	s	83.4	52.4	37.93	68.49	59.15	42.86	53.91
		m	84.3	52.5	38.50	68.64	58.42	43.78	53.52
		l	85.5	53.8	39.58	70.15	60.27	45.16	54.46

To comprehensively evaluate the effect of previously obtained area results on deep learning algorithms, the objects that could not be previously detected were considered, as shown in the results of w-Prior Total in Table 2. The improvement in object detection with 640² images was relatively high. The improvement in 0.5 mAP was about 2.5–6.6 points, and that in 0.5–0.95 mAP was about 2.2–4 points. The lifesaving appliances and swimmer classes were improved by about 13–15 and 3.1–4.5 AP points, respectively. The ADR for the boat class was lower than that for the other classes due to the use of the HSV-gray method, and the reduction for this class was about 3 AP points.

Compared with the field and city environments, the maritime environment is more complex for tiny object detection tasks. In particular, the strong glint noise caused by light reflection significantly affects the imaging quality of and detection performance on tiny objects. To improve tiny human target detection performance under limited computing resources, deploying HDetect-VS in edge equipment on UAVs should be researched in the future. Different lighting conditions, variable flight paths, and flight attitudes might affect the robustness of the algorithm; thus, robustness improvement in real maritime scenarios under the above harsh conditions will be considered in our future work.

5. Conclusions

In this study, we argue that the use of visible images is more effective in tiny human target detection based on visual saliency in noisy maritime environments than that of infrared images. Sea surface sun glint can significantly deteriorate the performance of pixel-level detection algorithms, such as LCM-based algorithms. To this end, HDetect-VS is proposed to suppress strong noise and enhance the saliency of tiny human targets. The color characteristics of glint noise and objects, which are not taken into account in conventional pixel-level algorithms, are introduced into the HSV-gray method to suppress glint noise and retain the saliency of objects. The pixel intensity value distribution is presented as the pixel intensity surface calculated with large-scale Gaussian Convolution. For maritime objects with specific spatial distribution characteristics, two-step clustering can be used to separate tiny objects from sun glint and background noise. Moreover, the results of HDetect-VS can be used as prior knowledge in deep learning-based algorithms to significantly improve object detection performance. Experiments on the SeaDronesSee dataset showed that the proposed HDetect-VS method is robust and effective in maritime tiny human object detection.