Vessel Detection with SDGSAT-1 Nighttime Light Images

Abstract

The Sustainable Development Goals Science Satellite-1 (SDGSAT-1) Glimmer Imager for Urbanization (GIU) data is very sensitive to low radiation and capable of detecting weak light sources from vessels at night while significantly improving the spatial resolution compared to similar products. Most existing methods fail to use the relevant characteristics of vessels effectively, and it is difficult to deal with the complex shape of vessels in high-resolution Nighttime Light (NTL) data, resulting in unsatisfactory detection results. Considering the overall sparse distribution of vessels and the light source diffusion phenomenon, a novel vessel detection method is proposed in this paper, utilizing the high spatial resolution of the SDGSAT-1. More specifically, noise separation is completed based on a local contrast-weighted RPCA. Then, artificial light sources are detected based on a density clustering algorithm, and an inter-cluster merging method is utilized to realize vessel detection further. We selected three research areas, namely, the Bohai Sea, the East China Sea, and the Gulf of Mexico, to establish a vessel dataset and applied the algorithm to the dataset. The results show that the total detection accuracy and the recall rate of the detection algorithm in our dataset are 96.84% and 96.67%, which is significantly better performance than other methods used for comparison in the experiment. The algorithm overcomes the dataset’s complex target shapes and noise conditions and achieves good results, which proves the applicability of the algorithm.

1. Introduction

At present, achieving large-scale and long-term active monitoring of maritime vessels is an urgent task for improving maritime jurisdiction and safety internationally [1,2]. Optical remote sensing technology provides a long-distance and economical dynamic observation method, which has the advantages of a wide observation range and long observation time and can achieve large-scale and multi-element monitoring and analysis at sea [3]. Numerous research results have already been obtained in these areas at present [4,5,6,7]. However, traditional optical remote sensing relies on detecting sunlight reflection and cannot provide adequate nighttime data. As a new direction and cutting-edge hotspot in remote sensing development, the low-light remote sensing technology can detect weak light sources on the surface at night or dusk, compensate for the limitations of traditional optical remote sensing methods, and provide Nighttime Light (NTL) products [8,9]. Low-light remote sensing technology is expected to achieve nighttime vessel detection by capturing the light emitted by vessels at night. Currently, relevant research has demonstrated the potential of NTL images for detecting vessels at night [10,11].

Many researchers have designed methods for nighttime vessel detection according to the relevant characteristics of low-light remote sensing. Unlike traditional optical remote sensing, which mainly uses reflectivity to reflect surface features, low-light remote sensing detects artificial light sources on the surface to form radiation products [12]. This results in vessels displayed as “bright spots” in NTL data, such as pixels or small areas with higher radiance values than the surrounding background.

Therefore, some researchers designed detection methods based on radiance features. The main idea behind these methods is to control noise, suppress background, enhance contrast to highlight vessel lighting information in the preprocessing stage, and design more flexible threshold methods based on local statistical data in the vessel detection stage. For example, Elvidge et al. [13] first enhanced the contrast of features by logarithmically enhancing the radiance values during the preprocessing phase to highlight the vessel and then used a region-adaptive Weiner filter to flatten the noise of the entire strip. The Spike Median Index (SMI) was adopted in the vessel identification stage to further suppress light source diffusion and background noise by subtracting the regional median. Elvidge et al. [14] improved their method by determining the threshold with the average radiance value data corresponding to a set of maximum gradient values in the data, which overcomes the difficulty of threshold selection caused by local condition changes. They applied this method to the ship tracking [15,16] and fishery monitoring [14]. Lebona et al. [17] applied Constant False Alarm Rate (CFAR), an adaptive threshold method commonly used in Synthetic Aperture Radar (SAR) images, to nighttime remote sensing data. Zhong Liang et al. [18] proposed a vessel detection method based on a two-parameter CFAR [19] and conducted it on Luojia1-01 data. Guo Gang et al. [20] improved the feature enhancement of radiation differences between light fishing vessels and background pixels using SMI. Also, they designed threshold segmentation and local peak detection using the maximum entropy method.

Some researchers linked the relevant characteristics of vessels with natural phenomena and used empirical models to acquire more accurate radiance information. Kim Euihyun et al. [21] introduced the lunar phase into the preprocessing stage and calculated the bias parameter used for correction through an empirical model for lunar phase correction. Xue et al. [22] processed noise information through adaptive filtering and quantitatively calculated and compensated for the loss of light sources spreading by establishing atmospheric transmission and diffusion models.

Some researchers have applied deep learning methods to extract the relevant characteristics of the ship target automatically. For example, Shao et al. [23] proposed an improved YOLOv5 [24] algorithm and named it TASFF YOLOv5. This algorithm is designed based on the idea of multi-scale feature fusion to enhance the performance of small object detection. Motomura et al. [25] designed FishNet based on the image segmentation network U-net [26], adding temporal feature input to roughly estimate the number of vessels in the area. Liu et al. [27] proposed an improved Fast R-CNN [28] method, namely Scale Expansion Attention Fusion Fast R-CNN (SEAFF), to detect luminous vessels in complex backgrounds.

There are still many characteristics that could be improved in current research. Due to the lack of other relevant features, such as contour or textures, the current target detection algorithm based on the image features applied in computer vision is unsuitable for vessel recognition tasks in NTL data. The methods developed to utilize the brightness feature have limitations and result poorly in complex scenarios. The threshold selection is usually based on manual experience or local statistical data. The manually selected threshold is not flexible enough to cope with different actual situations of vessels’ artificial light sources; the threshold may be too high or too low in different areas, leading to detection errors. The local environment significantly interferes with the threshold determined by local statistical data. Establishing empirical models based on natural phenomena sounds promising but requires atmospheric models, atmospheric inversion products, and diffusion reference data. Most of these reference data lack nighttime records, significantly impacting model accuracy. Moreover, due to the complexity of the types of vessel light sources and the actual manual setting, the physical model designed based on the ideal situation is difficult to deal with, which may cause unreliable detection results. The idea of deep learning is currently not effectively adjusted based on the actual condition of lacking contour, texture, and other related features, resulting in the inability to obtain convincing results while consuming a large number of computational resources.

In addition, most of the other current methods are based on low-resolution NTL data and cannot be applied to higher-resolution data. For many years, the option for researchers to study low-light remote sensing was the U. S. Air Force Defense Meteorological Satellite Program (DMSP) Operational Linescan System (OLS) [12] and the Visible Infrared Imaging Radiometer Suite (VIIRS) Day/Night Band (DNB) data [29] provided by the SNPP and JPSS-1 satellites. The spatial resolution of OLS and DNB data is 3 km × 3 km [30] and 742 m × 742 m [31], respectively. This makes the vessel display a sub-pixel target, meaning that the vessel takes only one pixel. However, with the continuous development of low-light remote sensing technology, low-light remote sensing satellites with much higher spatial resolution, including Luojia1-01 [32] and Sustainable Development Goals Science Satellite-1 (SDGSAT-1) [33], have been launched and produced remote sensing data. The comparison under different resolution images of the vessel is shown in Figure 1. In high-resolution data, vessels are represented as a collection of 10 to 100-pixel-sized light source targets with irregular shapes, sometimes internally disconnected with multiple peak points. The detection object of the existing detection algorithms is a single pixel rather than a ship target composed of multiple pixels. When applied to high-resolution data, the vessel will be detected repeatedly. Therefore, the existing low-light vessel detection algorithms cannot be applied to the latest low-light remote sensing data due to the spatial resolution difference, and algorithms based on high spatial resolution data are relatively lacking.

In this study, we developed an unsupervised vessel detection method for the Glimmer Imager for Urbanization (GIU) NTL data of the SDGSAT-1. We have established a vessel dataset containing 30 images and 562 vessels, which includes vessel data from different sea areas and spatial resolution to ensure diversity in the dataset. On this basis, we have designed a robust and unsupervised nighttime vessel detection algorithm. We observe that the background changes smoothly and slowly; the brightness of the random noise is low, and the ship targets are sparsely distributed in the NTL data. Therefore, we adopted a background noise separation method based on the Robust Principal Component Analysis (RPCA) [34] and added local contrast weights to realize the complete preservation of the vessel while removing background and noise. Considering that the artificial light source diffuses locally to form a high-light pixel cluster and the ship target is composed of one or more high-light pixel clusters, we use Density-Based Spatial Clustering of Applications with Noise (DBSCAN) [35] to detect vessel light sources and then use relative interconnection index to merge different light sources of the same vessel to obtain accurate target detection results. This method guarantees target detection’s robustness when the target size and brightness vary greatly and the background noise is complex and changeable, performing well in our dataset.

The rest of this article is organized as follows. Section 2 introduces the SDGSAT-1 GIU data and discusses the characteristics of the vessels displayed in high-resolution GIU data; Section 3 provides a detailed description of the vessel detection method designed in this article; Section 4 introduces the experimental results of applying this method to our vessel dataset; Section 5 draws conclusions.

2. Data

2.1. Overview of the SDGSAT-1 GIU Data

Sustainable Development Goals Science Satellite-1 (SDGSAT-1) is the world’s first scientific satellite dedicated to serving the 2030 Agenda [33]. The SDGSAT-1 is designed to carry three payloads with multiple observation modes to realize all-day, multi-load coordinated observation. The Glimmer/Multispectral Imager of the SDGSAT-1 satellite adopts a multi-mode common optical path imaging design, which can realize day/night switching imaging. The Glimmer imager is designed with a panchromatic band, divided into a low gain (Panchromatic Low, PL), high gain (Panchromatic High, PH), and high and low gain fusion (High Dynamic Range, HDR). In addition, there are three color bands (RGB). Technical details about SDGSAT-1 and GIU data are shown in

Table 1. Technical details about SDGSAT-1 and GIU data.

Satellite and Imager	Index Item	Detail
SDGSAT-1	Orbit type	Sun-synchronous orbit
	Orbit height	505 km
	Orbit angle	97.5°
	Revisit cycle	11 days
	Band	4 Glimmer Imager for Urbanization (GIU) bands, 7 Multispectral Imager for Inshore (MII) bands, 3 Thermal Infrared Spectrometer (TIS) bands
Glimmer Imager and GIU data	Imaging width	300 km
	Detection spectrum	P: 444~910 nm (PL/PH) B: 424~526 nm G: 506~612 nm R: 600~894 nm
	Pixel resolution	PL/PH/HDR 10 m, RGB 40 m
	SNR for urban trunk road light	≥50 (P/RGB, 1.0 × 10−2 W/m2/sr)
	SNR for urban residential area	≥10 (P/RGB, 1.6 × 10−3 W/m2/sr)
	SNR for Polar Moonlight	≥10 (P, 3.0 × 10−5 W/m2/sr)
	Dynamic range of single scene	≥60 dB

.

2.2. Characteristics of Vessels in GIU Data

In the GIU data, the vessel presents many unique characteristics. Figure 1b shows the vessel in the SDGSAT-1 GIU data. Vessels in GIU data mainly have the following characteristics:

• The vessels are presented as a local bright spot or bright spot with an irregular shape;
• Compared with the vast and empty background, the vessels are small in size and sparsely distributed, accounting for a low proportion of the overall image;
• Part of the vessels are unconnected targets, consisting of several close but disconnected bright spots;
• The image noise is complex, but the noise brightness value is generally lower than the target peak area of the vessels.

These unique characteristics are closely related to the imaging process of low-light remote sensing imager. The glimmer imager of SDGSAT-1 is designed to detect artificial light sources carried by human beings. It cannot detect detailed information such as the vessel’s geometric shape, contour, and texture. Compared with the maximum spatial resolution of 10 m, the light source carried by the vessel can be regarded as a point light source. The process of point light source transmission from the ground through the atmosphere to the satellite sensor is complex and will be affected by many aspects. It will produce a diffusion phenomenon, manifesting as light source leakage to the surrounding area. This phenomenon is shown in the GIU data, as the point light source occupies the local peak point in the data, and the neighboring pixels are also lightened up, resulting in the vessel appearing as a regional bright spot. This diffusion phenomenon is affected by atmospheric conditions, random noise of satellite sensors, and other factors in the imaging process. It is also closely related to the design and function of the light source and the running direction and state of the vessel itself. The complex influence factors make the diffusion of vessels unpredictable, resulting in irregular shapes and different sizes of diffusion bright spots. Moreover, considering that some large vessels are equipped with multiple light sources, these light sources are far away, and the diffusion of light sources is not enough to connect, which causes these vessels to become unconnected targets. Last but not least, due to the weak artificial light source at night, the large gain of low-light imager leads to poor signal-to-clutter ratio and complex noise. At the same time, there is also a certain probability of cloud and moonlight interference factors.

In summary, although the characteristics of GIU data and the light source of the vessel bring many difficulties to the design of detection methods, these characteristics also bring the basis for distinguishing the vessel from background and noise. We designed the vessel detection algorithm based on these characteristics.

3. Method

In this section, the low-light remote sensing vessel detection method used in this paper is described in detail. The main idea of this method is to separate the target from the background noise in the preprocessing stage to obtain the target map and then use the idea of density clustering for target detection. In the post-processing stage, the inter-cluster clustering method merges the non-connected light sources of the same vessel. The details are presented in the following subsections. The process diagram of our vessel detection method is shown in Figure 2.

3.1. Noise and Background Separation through Weighted RPCA

Generally, the low-light remote sensing data of relevant sea areas containing vessels can be formulated as

$$D=B+T+N,$$

(1)

where D is the original remote sensing data; B is the background image; T is the target image, and N is random noise. Usually, background images are considered to be slowly changing, meaning that regions in the background image are highly correlated, indicating that background image B can be regarded as a low-rank matrix.

$$rank(B)\le r,$$

(2)

where r is a constant used to reflect the complexity of the background image. The larger the value of r, the more complex the background. Obviously, r is much smaller than size of B.

In practical applications, a vessel usually keeps changing all the time. The brightness may vary from dim to bright, and its size may vary from 2 × 2 to more than 10 × 10 (in pixels). However, the size of the vessel is small compared to the background, and the distribution in the remote sensing data accounts for a small proportion. The target image T is a sparse matrix, namely,

$$\parallel T{\parallel }_0\le K,$$

(3)

where $\parallel \cdot {\parallel }_0$ denotes the $l_0$ norm (i.e., the number of non-zero elements in the vector), and K (K ≪ m × n, m × n is the size of the target image) is an integer determined by the number and size of the small target.

According to the actual situation of the remote sensing data we observed, we assume that the random noise is i.i.d. and $\parallel N{\parallel }_F\le \delta$ for some $\delta >0$. Thus, we have

$$\parallel D-B-T{\parallel }_F\le \delta ,$$

(4)

where $\parallel \cdot {\parallel }_F$ is the Frobenius norm, (i.e., $\parallel X{\parallel }_F=\sqrt{{\displaystyle {\sum }_{i,j}{{\displaystyle X}}_{i,j}^2}}$).

Based on the above discussions, B can be considered a low-rank component, and T can be regarded as a sparse component. Using the correlation characteristics of components, the traditional vessel noise and background separation problem can be transformed into an RPCA optimization problem.

$$\underset{B,T}{\min }\parallel B{\parallel }_{\ast }+\lambda \parallel T{\parallel }_1,s.t.\parallel D-B-T{\parallel }_F\le \delta ,$$

(5)

where $\parallel \cdot {\parallel }_1$ denotes the $l_1$ norm (i.e., $\parallel X{\parallel }_1={\displaystyle {\sum }_{i,j}\left|{{\displaystyle X}}_{i,j}\right|}$), and $\parallel \cdot {\parallel }_{\ast }$ denotes the nuclear norm (i.e., the sum of singular values). Given that dealing with the non-convex $l_0$ norm and $rank(B)$ is NP-hard (non-deterministic polynomial), we use $l_1$ norm and nuclear norm to replace them for tractable computation [36,37]. $\lambda$ is a balance parameter.

Random noises existing in GIU images are non-target sparse points. Due to the influence of the vessel diffusion phenomenon, the brightness value of some target edge areas is close to random noise, which makes it difficult to suppress it effectively only through the $l_1$ norm. As a weight coefficient to control the trade-off between background and target in Equation (5), a larger λ can suppress non-target sparse points and background effectively but will cause the target to shrink excessively, that is, only separating the target peak area, damaging the local density information of the vessel. Conversely, smaller λ can preserve the local density information but will introduce non-target noise. Because λ can only control the target image globally, it is easy to neglect one aspect and lose the other. We can control the separation of targets and noise more accurately in different regions by adopting a weighted $l_1$ norm [38] determined by local statistical data.

In the GIU data, the brightness value of non-target sparse points is usually much lower than the central high brightness area of the vessel, and its local variation [39] situation is different from the edge area of the target. We utilize the above characteristics to design a weighted $l_1$ norm based on brightness values and local variation. We first set the local variation calculation

$$v(x)={\displaystyle \sum _{a,b}}\left|x_a-x_b\right|,$$

(6)

where $x_a$ and $x_b$ are 8-neighbor pixels in x.

The local variation is calculated based on local image blocks; we take the maximum total variation of each point in the blocks as a measure of the total degree of change in the local area:

$$m(x)=\underset{\xi \in x}{\max }v(\xi ),$$

(7)

We set ξ as a 5 × 5 block of x.

Based on local contrast and pixel brightness, the reweighted $l_1$ norm is designed and used as follows:

$$\parallel T{\parallel }_{w,1}=\parallel W\odot T{\parallel }_1={\displaystyle \sum _{ij}}{W}_{ij}\left|T_{ij}\right|,$$

(8)

$$W_{ij}=\frac{1}{(\left|T_{ij}\right|+{\epsilon }_T)}\times \frac{1}{m_{ij}},$$

(9)

where $W=\left\{W_{ij}\right\}$ are weights for the elements $T_{ij}$ of $T$; $\left|\cdot \right|$ denotes the absolute value operator; $\odot$ denotes the Hadamard product; ${\epsilon }_T$ is a positive constant.

Then, the target-noise separation model based on the proposed weight $l_1$ norm can be formulated as follows:

$$\underset{B,T}{\min }\parallel B{\parallel }_{*}+\lambda \parallel T{\parallel }_{w,1},s.t.\parallel D-B-T{\parallel }_F\le \delta$$

(10)

We solve the model through the Inexact Augmented Lagrangian multiplier (IALM) method [36]. The augmented Lagrange function of (10) is formulated as

$$L(D,B,T,Y,\mu )=||B{||}_{*}+\lambda ||T{||}_{w,1}+\langle Y,D-B-T\rangle +\frac{\mu }{2}||D-B-T{||}_F^2,$$

(11)

where Y is the Lagrange multiplier matrix; $\mu$ is a penalty scalar; $\frac{\mu }{2}||D-B-T{||}_F^2$ is the regularization term to ensure the convergence of the function; $\langle \cdot ,\cdot \rangle$ denotes the inner product operator.

Next, B and T can be solved alternately as

$$B^{k+1}=\underset{B}{\mathrm{argmin}}||B{||}_{*}+\frac{{\mu }^k}{2}{‖D-B-T^k-\frac{Y^k}{{\mu }^k}‖}_F^2,$$

(12)

$$T^{k+1}=\underset{T}{\mathrm{argmin}}\lambda ||T{||}_{W,1}+\frac{{\mu }^k}{2}{‖D-B^{k+1}-T-\frac{Y^k}{{\mu }^k}‖}_F^2$$

(13)

According to [36], Equations (12) and (13) can be solved as

$$B^{k+1}={\mathcal{D}}_{{\mu }^{-1}}\left(D-T^k+{\mu }_k^{-1}{Y}^k\right)$$

(14)

$$T^{k+1}={\mathcal{S}}_{\lambda {\mu }^{-1}W}\left(D-B^{k+1}+{\mu }_k^{-1}{Y}^k\right),$$

(15)

where ${\mathcal{D}}_{{\mu }^{-1}}$ and ${\mathcal{S}}_{\lambda {\mu }^{-1}W}$ are the soft thresholding operators formulated as

$${\mathcal{D}}_{\tau }[X]=\left\{\begin{array}{llll}\left(1\right)& & & [U,S,V]=svd(X)\\ \left(2\right)& & & S=\mathrm{sgn}(S)\cdot \times \max (\left|S\right|-\tau ,0)\\ \left(3\right)& & & {\mathcal{D}}_{\tau }[X]=U\times S\times V^{\mathrm{T}}\end{array}\right.,$$

(16)

$${\mathcal{S}}_{\tau }[X]=\mathrm{sgn}(x)\cdot \max (\left|x\right|-\tau ,0)$$

(17)

As we solve B and T, matrix Y is updated via

$$Y^{k+1}=Y^k+{\mu }_k\left(D-B^{k+1}-T^{k+1}\right)$$

(18)

The solution of our weighted RPCA method via IALM is shown in Algorithm 1.

Algorithm 1 Solution via Inexact ALM Method

Input: Original image $D$, Weighting parameter $\lambda$ Initialize:  $B^0=T^0=Y^0=0$; ${\mu }_0=5\times {10}^{-4}$, $\rho =1.1$, $k=0$ While not converged do:      Update $B$: $B^{k+1}={\mathcal{D}}_{{\mu }^{-1}}\left(D-T^k+{\mu }_k^{-1}{Y}^k\right)$;      Update $T$: $T^{k+1}={\mathcal{S}}_{\lambda {\mu }^{-1}W}\left(D-B^{k+1}+{\mu }_k^{-1}{Y}^k\right)$;      Update $Y$: $Y^{k+1}=Y^k+{\mu }_k\left(D-B^{k+1}-T^{k+1}\right)$;      Update $\mu$: ${\mu }_{k+1}=\rho {\mu }_k$;      Check the convergence conditions      $\frac{{‖D-B^{k+1}-T^{k+1}‖}_F}{\Vert D{\Vert }_F}<\epsilon$;      Update k: $k=k+1$; End while Output: Background image $B^k$, Target image $T^k$

3.2. Light Source Detection Based on DBSCAN

After we separated the interference of background and noise in GIU data and obtained the target image, we expect that the vessel can be accurately and automatically identified through the target image by a target detection algorithm. As mentioned in Section 2.2, several characteristics limit the application of traditional target detection methods. Firstly, the vessel’s shape, size, and brightness values in SDG data are irregular. As a result, no reliable feature can be utilized, and target detection based on these features may lead to missed detection. Secondly, since many vessels carry different light sources at different positions, many vessels are non-connected targets, which may lead to false detection. We observe that due to the artificial light diffusion phenomenon, the local density of high-brightness pixels in the target area is significantly larger, as shown in Figure 3a. Based on this feature, we can convert the highlighted pixels into objects and then use local object density clustering to complete the object detection, as shown in Figure 3b.

We use density-based spatial clustering of applications with noise (DBSCAN) algorithm [35] for vessel light source detection. DBSCAN is widely used in various disciplines because of its unsupervised characteristics. It has a good clustering effect without the need to specify the number of clusters and cluster centers. The main idea of DBSCAN is that for each object in a cluster, the number of objects in the EPS neighborhood must be greater than or equal to a given threshold. Related concepts are as follows.

Definition 1.

Eps neighborhood: The Eps neighborhood of an object p denoted by $N_{Eps}\left(p\right)$ is defined by

$$N_{Eps}\left(p\right)=\left\{q\in \Omega |\mathrm{dist}\left(p,q\right)\le Eps\right\}$$

(19)

Definition 2.

Core Object: The number of sample objects in the Eps neighborhood of a given object is greater or equal to the threshold MinPts:

$$p\in N_{Eps}\left(p\right) \mathrm{and} \left|N_{Eps}\left(p\right)\right|\ge MinPts$$

(20)

Definition 3.

Boundary Object: A non-core object in the Εps neighborhood of the core object.

Definition 4.

Noise object: Object that is not a core object or boundary object.

Definition 5.

Direct density-reachable: For object p, q in object set $\Omega$, if the object q is in the Εps neighborhood of p, and p is the core object, then the object q is directly density-reachable from object p.

Definition 6.

Density-reachable: For the object set $\Omega$, given a series of sample objects $p_1,p_2\cdots p_n,p=p_1,q=p_n$, if every object $p_i$ is directly density-reachable by $p_{i-1}$, then object p is density-reachable from q.

Definition 7.

Density-connected: For objects p, q, o in object set $\Omega$, if objects o can be density-reachable from both p and q, then p and q are density-connected.

A schematic of the related concepts of DBSCAN is shown in Figure 4. Among them, $p_1,p_2,p_3,p_4$ are core objects; q and o are boundary objects; n is noise object; q is directly density-reachable from $p_1$; $p_1$ is density-reachable from $p_4$; q and o are density-connected.

For the input object set, there can be more than one core object in each object cluster, and these objects are density-reachable with each other. Therefore, the process of the DBSCAN algorithm is mainly to iteratively obtain the maximum density connected sample set derived from the density reachability relation. The detailed process is shown in Algorithm 2.

Algorithm 2 Light source detection via DBSCAN

Input: target image T, Initialize: MinPts = 5, Eps = 1, object set $\Omega$, $\mathrm{k}=1,$ For pixel ai in image T:     If $a_i>3$, $\Omega =\Omega \cup \left\{a_i\right\}$;     End if End for While all objects in $\Omega$ are not traversed     Take an untraversed object p from $\Omega$;     If p is a core object,         Initialize Target cluster set ${\mathrm{C}}_k$         traverse $\Omega$ to find all objects $p_i$ which are Density-reachable from p,             $C_k=C_k\cup \left\{p_i\right\}$         traverse $\Omega$ to find all objects $q_i$, which are Directly Density-reachable             from any object in $C_k$, $C_k=C_k\cup \left\{q_i\right\}$;       $\Omega =\Omega -C_k$;       k = k + 1;     end if end while Output: target cluster result $C_1,C_2,\dots ,C_k$

3.3. Clusters Merging through Relative Interconnection Index

Since many vessels carry different light sources at different positions, a large part of the vessels are non-connected targets. In addition, due to the high brightness of the light sources carried by some vessels, dazzling light and reflection inside the lens are produced in the imaging process. The density clustering in the previous step identifies these targets as multiple clusters. There is an obvious need to merge these adjacent light source clusters belonging to the same target. The interconnectivity index between objects is designed based on the brightness value of objects and the distance between objects. Then, the relative interconnectivity index between clusters is obtained based on the normalization of internal interconnectivity. Based on this index, we merge the adjacent clusters of the same target to complete the vessel detection. The measure of interconnectivity between objects $\mathrm{E}(p,q)$ is as follows:

$$\mathrm{E}(p,q)=\frac{T_p{T}_q}{{r_{pq}}^2+{\epsilon }_r}\epsilon ,$$

(21)

where $T_p,T_q$ represent the brightness value of the target image corresponding to these two-pixel object positions p and q; $r_{pq}=\mathrm{dist}\left(p,q\right)$, meaning the distance between two objects. $\epsilon ,{\epsilon }_r$ is a constant term.

To deal with the difficulty of threshold setting caused by the complexity of the actual vessels, we normalize their absolute interconnection by using internal interconnection to obtain the relative interconnection index [40,41] between clusters:

$$\mathrm{RI}(C_i,C_j)=\frac{2\mathrm{EC}(C_i,C_j)}{\mathrm{EC}(C_i)+\mathrm{EC}(C_j)},$$

(22)

where $C_i,C_j$ is a pair of clusters; $\mathrm{EC}(C_i,C_j)={\displaystyle \sum _{p\in C_i,p\in C_j}}\mathrm{E}(p,q)$; p and q are the pixel objects inside $C_i$ and $C_j$, respectively. We set a threshold based on $\mathrm{RI}(C_i,C_j)$ to conduct the merging process. The algorithm of the cluster merging process through the relative interconnection index is shown in Algorithm 3.

Algorithm 3 Clusters merging through relative interconnection index

Input: target cluster result $C_1,C_2,\dots ,C_k$ Initialize: $th=0.005$, $C=\{C_1,C_2,\dots ,C_k\}$, $V=\left\{\right\}$, $\mathrm{change}=\mathrm{true}$, $\mathrm{number}=0$, $n=1$ While $\mathrm{size}(C)>1$     $\mathrm{change}=\mathrm{true}$     While $\mathrm{change}=\mathrm{true}$        $\mathrm{change}=\mathrm{false}$, $\mathrm{number}=\mathrm{size}(C)$       for $i=2:\mathrm{number}$           if $\mathrm{RI}(C\{1\},C\{i\})>th$                $C\left\{1\right\}=C\left\{1\right\}\cup C\left\{i\right\}$, $C\left\{i\right\}=\left\{\right\}$, $\mathrm{change}=\mathrm{true}$, $\mathrm{number}=\mathrm{number}-1$           end if       end for     end while     $V\left\{n\right\}=V_n=C\left\{1\right\}$, $C\left\{1\right\}=\left\{\right\}$, $n=n+1$ end while $V\left\{n\right\}=V_n=C\left\{1\right\}$, $C\left\{1\right\}=\left\{\right\}$ Output: Vessel detection result $V=\{V_1,V_2,\dots ,V_n\}$

4. Results

This article selects four scenarios of HDR and RGB-level moonless SDGSAT-1 GIU images from three study areas to validate the proposed algorithm: the area around the Bohai Sea; the Gulf of Mexico; and the East China Sea. More details about the data we select are shown in

Table 2. Information about the GIU data we select.

Area	Image	Time and Position	Target Number
Bohai Sea	Image 1(RGB)	36.78°N to 40.61°N, 117.33°E to 121.97°E	314
		2022-03-20T13:10:38
East China Sea	Image 2(HDR)	27.30°N to 30.27°N, 122.84°E to 126.02°E	99
		2022-08-20T12:59:43
Gulf of Mexico	Image 3(HDR)	25.33°N to 28.34°N, 86.71°W to 89.56°W	157
		2022-05-19T03:10:29
	Image 4(RGB)	28.04°N to 31.04°N, 87.32°W to 90.55°W
		2022-05-19T03:11:12

. The vessel’s position is marked by artificial visual interpretation, and the detection results are compared with the manual marking results to verify the performance of the proposed method. The main process of artificial vision interpretation of the vessel is to identify the vessel light sources and mark the closely related light sources as targets. We adopt this method in [13] to distinguish lighting objects such as vessels, oil platforms, and minor islands through VIIRS band M10 and the land–sea mask data. We use the VIIRS band M10 data to conduct combustion source detection and locate the position of the offshore oil platforms. We also used the land–sea mask data to locate the positions of minor islands in relevant areas. Considering the memory capacity limit, we roughly cropped the original remote sensing data to form an experimental dataset. Data containing offshore oil platforms and minor island areas were eliminated from the dataset. The final dataset has 25 images of different sizes, with 570 vessels recorded by artificial visual interpretation. These vessels have different environmental conditions, directions, and sizes, which can effectively test the algorithm’s applicability.

We computed the grayscale brightness from RGB data through Equation (23) [42]

$$\mathrm{Brightness}=0.2989\times \mathrm{Red}+0.5870\times \mathrm{Green}+0.1140\times \mathrm{Blue}$$

(23)

We chose precision, recall, and F₁ score as the performance evaluation metrics for the quantitative analysis of the vessel detection. Precision and recall can be expressed as follows:

$$\mathrm{precision}=\frac{TP}{TP+FP},$$

(24)

$$\mathrm{recall}=\frac{TP}{TP+FN},$$

(25)

where TP represents true positive, that is, the detected vessel is the actual target; FP stands for false positive, indicating that the detected target is not the real vessel; FN stands for false negative and represents a real vessel that is not detected.

To be able to consider both the precision and recall of these two metrics, the F₁ score is also introduced as follows:

$${\mathrm{F}}_1=2\times \frac{\mathrm{precision}\times \mathrm{recall}}{\mathrm{precision}+\mathrm{recall}}$$

(26)

which is the harmonic average of precision and recall rate.

The schematic diagram of the detection results in this study’s sea areas is shown in Figure 5, where the red box represents the location of the vessel determined by the detection algorithm. Figure 5 presents the data for several different cases. Here, Figure 5a–f shows RGB data, and Figure 5g–o shows HDR data. It can be seen from Figure 5 that the shapes and sizes of the vessels in these data have significant differences. Moreover, Figure 5i,l,o,n contains non-connected targets. These complex factors bring more difficulties to the detection algorithm.

The detection results of our method in three study areas are shown in Table 2. It can be seen that the total precision and recall are 96.84% and 96.71%, respectively. Obviously, our method achieves more than 95% in both precision and recall in different regions; it proves that the method we proposed can handle complex data situations well and accurately identify most vessel targets.

We chose the methods proposed by Elvidge et al. [14] and Zhong Liang et al. [18] to conduct experiments on our dataset and compare the detection performance with our method. Elvidge’s method is designed for low-resolution NTL data, which has a wide influence and has produced a large number of ship target detection products based on DNB data. Zhong’s method is the only method currently designed for high-resolution NTL data that can achieve good detection results without requiring massive data as a training set, which is suitable for the actual situation of our experimental environment. Therefore, we choose these two methods for comparison. The detection results of the comparison method are shown in

Table 3. Calculation Results of Performance Assessment Index.

Method	Study Area	Precision	Recall	F1 Score
Our method	Gulf of Mexico	0.9623	0.9745	0.9684
	Bohai sea	0.9743	0.9650	0.9696
	East China Sea	0.9596	0.9596	0.9596
	total	0.9684	0.9667	0.9675
SMI Threshold method	Gulf of Mexico	0.4421	0.6561	0.5282
	Bohai sea	0.4134	0.6306	0.4994
	East China Sea	0.4776	0.6465	0.5494
	total	0.4471	0.6825	0.5403
Two-parameter CFAR	Gulf of Mexico	0.7584	0.7197	0.7386
	Bohai sea	0.7672	0.7452	0.7561
	East China Sea	0.7556	0.6869	0.7196
	total	0.7629	0.7281	0.7451

. Among them, the total precision and recall calculated by the algorithm proposed by Elvidge et al. are 55.33% and 61.05%, respectively, which is lower than the method proposed in this study. The experimental results show that the vessel detection algorithm designed based on DNB data is unsuitable for the higher-resolution data. Figure 6 shows several typical detection error cases of the compared threshold methods. It can be seen from the actual detection result figure that the exact brightness value of different vessels in the actual data varies greatly, and the detection algorithm based on the threshold method cannot deal with this situation, resulting in the missed detection of low brightness vessel and the repeated detection of high brightness vessel. The main reason for this phenomenon is that the detection object of the methods designed for DNB data is a single pixel rather than a vessel composed of multiple pixels. When applied to high-resolution data, the vessel will be detected repeatedly. Our method can avoid this situation by detecting clusters of highlighted pixels. As for the Two-parameter CFAR method proposed by Zhong Liang et al., the total precision and recall are 76.29% and 72.81%, respectively. In general, this method has a certain effect on detecting vessels, but this method needs to adjust the size of the target candidate box manually, and the size of the candidate box is fixed in the same image, so the detection effect of this method is not robust on the data with large differences in the size of different targets. The detection method based on DBSCAN can automatically adapt to ship targets with different shapes and sizes to obtain better detection results.

5. Discussion

It can be seen from the experimental results that our method achieves better detection results in complex detection tasks and successfully detects the vast majority of ship targets compared to the comparison methods. The occurrence of a small number of false positives and false negatives is mainly related to the following two factors: First, a small number of vessel targets are relatively smaller than other targets, accounting for only 10–20 pixels, and their radiance is much lower, too. In the meantime, these targets are in areas with relatively high noise levels. These situations lead to the vessel target being suppressed in the noise removal stage and, subsequently, ignored in the artificial light source detection stage, which eventually results in missed reports. Second, the independent artificial light source carried by some vessels is relatively far away, and its brightness is low. Hence, the relative connectivity index between clusters is lower than the fixed threshold, and the mechanism of merging clusters belonging to the same target fails to work.

There are still potential investigation values and prospects in the aspect of nighttime vessel detection. We expect to further improve our method to enhance the detection of weak and complex targets in future work. We also hope to further investigate the process of point light source transmission from the ground through the atmosphere to the satellite sensor, analyze the characteristics generated by this process, and design relevant methods to achieve better detection results.

6. Conclusions

In this paper, we propose an object detection method based on Robust PCA and density clustering to automatically detect nighttime vessels from SDGSAT-1 GIU data. In the first step, the target image and the background noise image are transformed into a constrained optimization problem based on RPCA. At the same time, a local contrast factor is designed to suppress the sparse noise based on the characteristics of the relevant sea night image, and then, the optimization problem is solved based on IALM. In the second step, based on the artificial light diffusion phenomenon, the highlighted pixels are transformed into objects, and the artificial light is detected by DBSCAN. In the third step, the relative interconnectivity index between clusters is designed to characterize the relation between light sources clusters quantitatively, and the non-connected light sources belonging to the same vessel are merged by comparing with a threshold to complete the vessel detection.

To test the performance of the algorithm, we selected three research areas, the Bohai Sea, the East China Sea, and the Gulf of Mexico, to establish a vessel dataset and applied the algorithm to the dataset. We annotated the target through artificial visual interpretation. We applied the algorithm to this dataset, and the results show that the total detection accuracy and recall rate of the detection algorithm in the three research areas are 96.84% and 96.67%, respectively. The algorithm overcomes the data’s complex vessel shape and noise situation and achieves good results, showing that the algorithm is reliable.

In future work, we plan to further explore the relevant characteristics of the vessel light source, expecting to improve the performance of the detection algorithm. In addition, the light settings of different types of vessels will be investigated, and a vessel recognition algorithm will be designed and implemented on this basis.