DAENet: Deformable Attention Edge Network for Automatic Coastline Extraction from Satellite Imagery

Abstract

Sea–land segmentation (SLS) is a crucial step in coastline extraction. In CNN-based approaches for coastline feature extraction, downsampling is commonly used to reduce computational demands. However, this method may unintentionally discard small-scale features, hindering the capture of essential global contextual information and clear edge information necessary for SLS. To solve this problem, we propose a novel U-Net structure called Deformable Attention Edge Network (DAENet), which integrates edge enhancement algorithms and a deformable self-attention mechanism. First of all, we designed a multi-scale transformation (MST) to enhance edge feature extraction and model convergence through multi-scale transformation and edge detection, enabling the network to capture spatial–spectral changes more effectively. This is crucial because the deformability of the Deformable Attention Transformer (DAT) modules increases training costs for model convergence. Moreover, we introduced DAT, which leverages its powerful global modeling capabilities and deformability to enhance the model’s recognition of irregular coastlines. Finally, we integrated the Local Adaptive Multi-Head Attention-based Edge Detection (LAMBA) module to enhance the spatial differentiation of edge features. We designed each module to address the complexity of SLS. Experiments on benchmark datasets demonstrate the superiority of the proposed DAENet over state-of-the-art methods. Additionally, we conducted ablation experiments to evaluate the effectiveness of each module.

1. Introduction

The coastline holds significant importance in topographic maps and maritime charts and has been officially recognized as one of the 27 terrestrial features by the International Geographic Data Committee [1,2,3]. Since the 20th century, economic hubs in coastal nations globally have progressively migrated toward coastal regions [4,5]. The natural attributes of coastlines have rapidly diminished due to the proliferation of coastal development projects, substantial population growth, rapid economic expansion, and escalating geographical significance. The original productive capacities and ecological functions of coastlines have undergone substantial changes [6,7], leading to severe challenges for the natural ecological environment in coastal areas [8,9]. Coastal regions have become some of the areas with the most frequent and intense human activities [10,11,12]. In light of this, the objective analysis of the temporal and spatial evolution characteristics of coastlines, as well as the quantitative assessment of their impacts and dynamic responses to human interventions, has emerged as a core concern in the academic community.

Coastal-related information is crucial across multiple applications, including coastal management [13], ship detection [14], and water resource management [15,16]. With the introduction of satellites equipped with visible light and infrared sensors [17], remote sensing imagery has become a viable alternative for coastline extraction, replacing laborious techniques such as photogrammetry and GPS field surveys [1,18,19]. Nevertheless, the extraction process faces obstacles primarily due to challenges in effective SLS. These challenges encompass irregular band intensity, complex land textures, and a restricted contrast between sea and land [20]. From an image classification perspective, these methods mainly rely on object-oriented and pixel-oriented classification [21]. Pixel-based segmentation methods [3,22,23,24] exacerbate these issues by dealing with mixed pixels, leading to noisy classification results and undermining the accuracy of coastline extraction. Moreover, the complexity of rules within the object-oriented methodology poses additional challenges. Currently, developing models tailored explicitly for intricate coastal structures remains a significant challenge [25,26,27]. Consequently, there is an urgent need to explore innovative approaches for intelligent analysis and extraction of coastlines in complex environments.

Given that deep learning aims to assign each pixel in the input image to fixed categories in semantic segmentation tasks, its objectives align closely with those of sea–land segmentation. In recent years, researchers have endeavored to integrate deep learning techniques into coastline extraction. Liu et al. [28] utilized convolutional neural networks (CNNs) for SLS, leading to enhanced image segmentation accuracy. Following this advancement, several effective SLS networks based on CNNs have emerged. These include a deep convolutional neural network (DeepUNet) [29], squeeze and excitation rank faster R-CNN [30], fully convolutional DenseNet (FC-DenseNet) [31], a multi-scale sea–land segmentation network (MSRNet) [32], a more comprehensive range of batch sizes network (WRBSNet) [33], and a deep learning model based on the U₂-Net deep learning model [34]. For example, WRBSNet [33] integrates a broader range of batch sizes to enhance performance. MSRNet [32] integrates squeeze and attention modules to bolster features across different scales, thereby reinforcing weak sea–land boundary information. Furthermore, many researchers also take into account the semantic and edge characteristics of sea–land segmentation and have devised networks for coastline extraction, such as dual-branch structures [35], multi-scale transformation [36], and edge-semantic fusion [37]. Recent studies primarily focus on optimizing and innovating based on CNNs. During the feature extraction process, CNN-based models frequently down-sample features to reduce computational requirements, potentially resulting in the loss of small-scale features [38,39,40]. Land objects from various semantic categories may share similar sizes, materials, and spectral characteristics, thereby posing challenges in their differentiation. Furthermore, occlusions within the network model frequently result in semantic ambiguities. Consequently, there is an urgent requirement for more comprehensive global contextual information and refined spatial features to serve as cues for semantic reasoning [41].

Recently, transformers have demonstrated significant benefits in natural language processing and computer vision, exhibiting outstanding performance across numerous tasks. The first introduction of transformers in this domain was by Yang et al. [42]. They first introduced transformers into this domain by experimenting with the pure Transformer architecture SETR [43], achieving performance comparable to existing CNN methods in land-sea segmentation. Another hybrid approach, SegFormer [44], which combines CNNs and transformers, surpassed state-of-the-art CNN methods and exhibited strong robustness. This illustrates the capability of Transformer architectures in the field of SLS. Subsequently, Yang et al. [45] employed a Transformer model to forecast alterations in the coastline of Weitou Bay, China. Zhu et al. [46] perform parallel feature extraction using both the Swin Transformer and ResNet branches concurrently. However, research in the field of SLS has been relatively limited. With the continuous evolution of Transformer structures, numerous attention-based variant structures [47,48,49,50,51,52,53,54] have emerged. Among these approaches, Swin Transformer [50] utilizes window-based local attention to confine attention within local windows, whereas Pyra-mid Vision Transformer (PVT) [51] decreases the resolution of key and value feature maps to reduce computation. Although effective, hand-crafted attention patterns are agnostic to data and may not be optimal. Relevant keys/values may be dropped, while less important ones are retained. Ideally, the candidate key/value set for a given query should be flexible and capable of adapting to each individual input, thereby alleviating issues with hand-crafted sparse attention patterns. In fact, in CNN literature, learning a deformable receptive field for convolution filters has been demonstrated to effectively focus on more informative regions based on data [52]. The coastline frequently displays irregular shapes. Additionally, due to the typically indistinct boundary between the ocean and land, its demarcations often appear blurred in remote sensing images. Consequently, coastline extraction requires both more global contextual information and clear edge information. Currently, there is a scarcity of results from such studies. To address this issue, our research investigates integrating a self-attention mechanism into sea–land segmentation to emphasize more global contextual information. Meanwhile, considering the linear characteristics of coastlines, our study aims to utilize edge enhancement algorithms and integrate the Deformable Attention Transformer mechanism [55], leading to the development of a Deformable Attention Edge Network (DAENet). The core structure of this network comprises an encoder, decoder, bottleneck region, and skip connections. Experimental results demonstrate that our network structure can accurately extract coastline features, closely matching the actual coastline. The main contributions of this study can be summarized as follows:

(1)

Introduction of the multi-scale edge detection module: Upon image input, we introduced the multi-scale transformation (MST) module, applying canny edge detection across multiple spatial scales and stacking them as input channels. This not only enhances the model’s robustness but also facilitates faster convergence.

(2)

Construction of an adaptive edge detection module: During training, we developed a Local Adaptive Multi-Head Attention-based Edge Detection (LAMBA) module to enhance the disparities in edge features in the spatial dimension, thus reducing semantic ambiguities that may arise from similar features among objects across different semantic categories.

(3)

Exploration of the Deformable Attention (DAT) Application: In order to enhance DAENet’s receptive field, we incorporated deformability into the U-shaped structure. This integration serves to alleviate constraints imposed by the fixed convolutional kernel in CNNs and the conventional patch generation in Transformers. Additionally, edge maps are utilized to compute an edge-aware loss, optimizing a novel edge loss function and accelerating model convergence.

2. Methods

2.1. Overall Framework

To tackle the complexities of SLS environments, our proposed DAENet incorporates a series of specialized modules, each designed to address specific features of SLS. The MST in DAENet is strategically designed for use in SLS. MST enhances edge feature extraction and model convergence through multi-scale transformation and edge detection, enabling the network to more effectively capture spatial–spectral changes and ensuring accurate depiction of SLS with varying spatial–spectral characteristics. Specifically, the core principle of multi-scale edge detection involves step-by-step downsampling of a two-dimensional image and applying edge detection to capture edge information at various scales. This is crucial because the deformability of DAT modules leads to significant training costs for model convergence. Our MST module filters the information for validity before it enters the DAT module. After the filtered information enters DAT, the relationships between tags are modeled, sampled, and projected, guided by important regions in feature mapping, obtaining the keys and values after deformation. The exchange of information between edge details at different scales, captured by MST, and the global context, captured by the converter, is enhanced using standard multi-head attention that focuses on sampling keys and aggregating features. This is achieved by calculating attention weights that reflect the importance of each feature in the context of the entire image, enabling the model to make informed predictions based on both local and global features.

Finally, in the output generation phase, we integrated the LAMBA module to improve the spatial differentiation of edge features, consequently decreasing semantic ambiguities resulting from similarities among features from disparate semantic categories. Specifically, we use gradient information and a multi-head self-attention mechanism to generate a projection function. This function calculates the relationship between the current pixel and its surrounding pixels. Weight parameters are introduced to compute comprehensive features, ensuring consideration of local characteristics and rotation invariance. Furthermore, our study utilizes skip connections to combine multi-scale features from the encoder with upsampled features, concatenating shallow and deep features to reduce spatial information loss caused by downsampling. During network training, the edge-aware loss is calculated using the binary Dice Loss function, thereby improving the model’s performance. These design choices are based on our understanding of the spectral, spatial, and edge characteristics of SLS. Together, they enable DAENet to excel in the challenging task of semantic segmentation in SLS.

As shown in Figure 1, the proposed deep learning network model, D DAENet, a hybrid of DAT and UNet, inherits the robust structure of UNet. DAENet employs skip connections between encoders and decoders, forming an encoder–decoder structure with MST, LAMBA, and DAT.

2.2. Multi-Scale Transformation Module

To address the complexity and variability of SLS, capturing both edge and semantic information is crucial for generating informative features. To achieve this goal, we designed the multi-scale transformation (MST). The fundamental principle of multi-scale edge detection involves analyzing the image at different scales to capture edge information of various granularities. Specifically, this approach utilizes the concept of scale-space and is implemented through a Gaussian pyramid. We use a linear Gaussian kernel to construct the image scale to add another dimension to the 2D image without introducing noise. The input image is convolved with a Gaussian filter with an increasing standard deviation. After generating a series of smoothed images, edge detection is performed on each image using the Canny algorithm to capture edge information at different scales. The detailed structure of this module is illustrated in Figure 2.

In the first instance, with regard to a specific image $\mathrm{I}\left(\mathrm{x},\mathrm{y}\right)$, its spatial representation within the Gaussian scale-space is:

$$\mathrm{L}\left(\mathrm{x},\mathrm{y},\sigma \right)=\mathrm{G}\left(\mathrm{x},\mathrm{y},\sigma \right)\ast \mathrm{I}\left(\mathrm{x},\mathrm{y}\right)$$

(1)

Here, L denotes the scale-space representation, G is the Gaussian function with a variance of ${\sigma }^2$, and $\ast$ signifies the convolution operation.

When transitioning across scales, the Difference of Gaussians (DoG) can be expressed as:

$$\mathrm{D}\left(\mathrm{x},\mathrm{y},\sigma \right)=\mathrm{L}\left(\mathrm{x},\mathrm{y},\mathrm{k}\sigma \right)-\mathrm{L}\left(\mathrm{x},\mathrm{y},\sigma \right)$$

(2)

In this equation, k is a constant multiplier, typically greater than 1.

This module builds upon the principles of differential Gaussians, which encompass a range of scales. It utilizes the summation of cross-scale disparities to integrate information across these scales. The governing expression is elaborated as follows:

$$\mathrm{A}\left(\mathrm{x},\mathrm{y}\right)={{\displaystyle \sum }}_{\sigma }\mathrm{D}\left(\mathrm{x},\mathrm{y},\sigma \right)$$

(3)

In this equation, $\mathrm{A}\left(\mathrm{x},\mathrm{y}\right)$ denotes the cumulative response.

2.3. Deformable Attention Transformer

Xia et al. [55] used the Vision Transformer with a Deformable Attention module to create a robust pyramid skeleton network called DAT, suitable for image classification and various dense prediction tasks. Unlike Deformable Convolutional Networks (DCN), which learn different offsets for different pixels across the entire feature map, this module learns a set of offsets independent of the query. These offsets direct the keys and values towards significant areas, as shown in Figure 3a. This design retains linear spatial complexity and introduces a Deformable Attention pattern to the Transformer backbone. Each attention module initially generates reference points as a uniform grid, which remains consistent across the input data. Subsequently, an offset network uses the query features as input to generate corresponding offsets for all the reference points. As a result, the candidate keys and values are shifted towards essential regions, enhancing the flexibility and efficiency of the original self-attention module to capture richer information features. The detailed structure of this module is described as follows:

Firstly, as illustrated in Figure 3a, given an input feature map $x\in {ℝ}^{H\times W\times C}$, a uniform grid composed of points $p\in {ℝ}^{H_G\times W_G\times 2}$ is generated as a reference. Specifically, the grid size is downsampled by a factor of $r$ from the input feature map, so that the grid dimensions are $r$, $H_G=H/r$, $W_G=W/r$. The reference points are linearly spaced two-dimensional coordinates ranging from $\left\{\left(0,0\right),\dots ,(H_G-1,W_G-1)\right\}$. These coordinates are then normalized to lie within the range $\left[-1,+1\right]$, based on the grid shape $H_G\times W_G$, where $\left(-1,-1\right)$ represents the top–left corner and $\left(+1,+1\right)$ signifies the bottom–right corner.

Subsequently, to obtain the offset for each reference point, the feature map is linearly projected to the query tokens as $q=x{W}_q$. Then, the feature map is fed into a lightweight sub-network ${\theta }_{offset}(\cdot )$ generate the offsets $△p={\theta }_{offset}\left(q\right)$. To stabilize the training process, we scale the amplitude of $△p$ by some predefined factor $s$ to prevent the offset from becoming too large, $\mathrm{i}.\mathrm{e}.$, $△p\leftarrow stanh\left(△p\right)$. Next, the features are sampled at the positions corresponding to the deformed points, serving as both keys and values, and subsequently processed using projection matrices:

$$\mathrm{q}=\mathrm{x}{\mathrm{w}}_{\mathrm{q}},\tilde{\mathrm{k}}=\tilde{\mathrm{x}}{\mathrm{W}}_{\mathrm{k}},\tilde{\mathrm{v}}=\tilde{\mathrm{x}}{\mathrm{W}}_{\mathrm{v}}$$

(4)

$$\mathrm{w}\mathrm{i}\mathrm{t}\mathrm{h} △\mathrm{p}={\theta }_{\mathrm{o}\mathrm{f}\mathrm{f}\mathrm{s}\mathrm{e}\mathrm{t}}\left(\mathrm{q}\right),\tilde{\mathrm{x}}=\phi \left(\mathrm{x};\mathrm{p}+△\mathrm{p}\right)$$

(5)

The symbols $\tilde{k}$ and $\tilde{v}$ represent the deformed keys and values embeddings, respectively. Specifically, the sampling function $\mathrm{\varphi }(\cdot ;\cdot )$ made differentiable by translating it into a bilinear interpolation.

$$\mathrm{\varphi }\left(\mathrm{z};\left({\mathrm{p}}_{\mathrm{x}},{\mathrm{p}}_{\mathrm{y}}\right)\right)={\displaystyle \sum }_{\left({\mathrm{r}}_{\mathrm{x}},{\mathrm{r}}_{\mathrm{y}}\right)}\mathrm{g}\left({\mathrm{p}}_{\mathrm{x}},{\mathrm{r}}_{\mathrm{x}}\right)\mathrm{g}\left({\mathrm{p}}_{\mathrm{y}},{\mathrm{r}}_{\mathrm{y}}\right)\mathrm{z}\left[{\mathrm{r}}_{\mathrm{y}},{\mathrm{r}}_{\mathrm{x}},:\right]$$

(6)

The function $g\left(a,b\right)=max(0,1-\left|a-b\right|),$ and $({\mathrm{r}}_x,{\mathrm{r}}_y)$ index all the positions on $z\in {\mathrm{R}}^{{\mathrm{H}}_{\mathrm{G}}\times {\mathrm{W}}_{\mathrm{G}}\times 2}$. Given that $g$ is non-zero only at the four integral points closest to $({\mathrm{p}}_x,{\mathrm{p}}_y)$, it simplifies Equation (5) to the weighted average of these four positions. In line with established methodologies, multi-head attention is employed, along with the adoption of relative position offsets. The resulting attention-head output can be formulated as follows:

$$z^{\left(m\right)}=\sigma (\frac{q^{\left(m\right)}{\tilde{k}}^{{\left(m\right)}^T}}{\sqrt{d}}+\phi \left(\widehat{B};R\right){\widehat{v}}^{\left(m\right)}$$

(7)

Here $\phi \left(\widehat{\mathrm{B}};\mathrm{R}\right)ϵ{\mathrm{R}}^{\mathrm{H}\mathrm{W}\times {\mathrm{H}}_{\mathrm{G}}{\mathrm{W}}_{\mathrm{G}}}$.

In conclusion, the Deformable Multi-Head Attention (DMHA) has a computational cost that is analogous to that of the Pyramid Vision Transformer (PVT) or the Swin Transformer. The sole additional overhead is attributed to the subnetwork designated for offset generation. The computational complexity of the entire module can be encapsulated by the following equation:

$$\mathrm{\Omega }\left(\mathrm{D}\mathrm{M}\mathrm{H}\mathrm{A}\right)=2\mathrm{H}\mathrm{W}{\mathrm{N}}_{\mathrm{S}}\mathrm{C}+2\mathrm{H}\mathrm{W}{\mathrm{C}}^2+2{\mathrm{N}}_{\mathrm{S}}{\mathrm{C}}^2+\left({\mathrm{k}}^2+2\right){\mathrm{N}}_{\mathrm{S}}\mathrm{C}$$

(8)

Here, $N_S=H_G{W}_G=\frac{HW}{r^2}$ denotes the number of sampling points.

2.4. Local Adaptive Multi-Head Attention-Based Edge Detection Module

The fundamental principle of LAMBA is to adapt to different image features by comprehensively considering gradients, textures, and intensity. Unlike most self-attention mechanisms, in the initial stages of the task, we employ local binary pattern (LBP) features and adaptive edge detection to capture local texture and edge information, guiding the multi-head self-attention mechanism to focus on various directional information, thereby enhancing the model’s ability to extract coastlines. In LAMBA, the spatial information of pixels is leveraged to integrate multi-directional feature modules. The following section outlines the detailed steps Figure 4:

Initially, the input image undergoes prediction to generate the model’s output image. Let the prediction result be denoted as $\mathrm{P} \left(\mathrm{x}, \mathrm{y}\right)$, where $\left(\mathrm{x}, \mathrm{y}\right)$ represents pixel coordinates. The Sobel convolution kernel is utilized to compute horizontal and vertical gradients, and the gradient information, along with the multi-head self-attention mechanism, is employed to adjust the projection direction.

$$\begin{array}{cc}& {\nabla }_{\mathrm{x}}\mathrm{I}\left(\mathrm{x},\mathrm{y}\right)={\mathrm{Sobel}}_{\mathrm{x}}\ast \mathrm{P}\left(\mathrm{x},\mathrm{y}\right)\\ & {\nabla }_{\mathrm{y}}\mathrm{I}\left(\mathrm{x},\mathrm{y}\right)={\mathrm{Sobel}}_{\mathrm{y}}\ast \mathrm{P}\left(\mathrm{x},\mathrm{y}\right)\end{array}$$

(9)

$${\mathrm{A}}_{\mathrm{i}\mathrm{j}}=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}(\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{1}\left(\nabla \mathrm{I}\right),\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}2\left(\nabla \mathrm{I}\right),\dots ,\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{K}\left(\nabla \mathrm{I}\right))$$

(10)

where ${\mathrm{Sobel}}_{\mathrm{x}}$ and ${\mathrm{Sobel}}_{\mathrm{y}}$ represent the Sobel convolution kernels, and ${\mathrm{A}}_{\mathrm{i}\mathrm{j}}$ denotes the projection adjustment function.

Next, for each pixel, compute its local binary pattern (LBP):

$$\mathrm{L}\mathrm{B}\mathrm{P}\left(\mathrm{x},\mathrm{y}\right)={\displaystyle \sum }_{\mathrm{p}=0}^{\mathrm{P}-1}\mathrm{s}\left({\mathrm{I}}_{\mathrm{p}}-\mathrm{I}\left(\mathrm{x},\mathrm{y}\right)\right)\cdot 2^{\mathrm{p}}$$

(11)

where $\mathrm{I}\mathrm{p}$ represents the grayscale value of the neighborhood points with the current pixel as the center, $\mathrm{P}$ is the number of neighborhood points, and $\mathrm{s}\left(\mathrm{x}\right)$ is the step function.

Consider dynamically adjusting high and low thresholds based on local image properties and calculating the average intensity value within the local window.

$$\mathrm{W}\left(\mathrm{x},\mathrm{y}\right)=\frac{1}{\mathrm{N}}{\displaystyle \sum }_{\mathrm{i}=\mathrm{x}-{\mathrm{W}}_{\mathrm{x}}}^{\mathrm{x}+{\mathrm{W}}_{\mathrm{x}}}{\displaystyle \sum }_{\mathrm{j}=\mathrm{y}-{\mathrm{W}}_{\mathrm{y}}}^{\mathrm{y}+{\mathrm{W}}_{\mathrm{y}}}\mathrm{P}\left(\mathrm{i},\mathrm{j}\right)$$

(12)

$${\mathrm{T}}_{\mathrm{high}}\left(\mathrm{x},\mathrm{y}\right)={\mathrm{k}}_{\mathrm{high}}\cdot \mathrm{a}\mathrm{v}\mathrm{g}\left(\mathrm{W}\left(\mathrm{x},\mathrm{y}\right)\right)$$

(13)

$${\mathrm{T}}_{\mathrm{low}}\left(\mathrm{x},\mathrm{y}\right)={\mathrm{k}}_{\mathrm{low}}\cdot \mathrm{a}\mathrm{v}\mathrm{g}\left(\mathrm{W}\left(\mathrm{x},\mathrm{y}\right)\right)$$

(14)

Finally, we introduce weight parameters and combine LBP, adaptive edge detection, and self-attention mechanisms to compute comprehensive features.

$$\mathrm{F}\left(\mathrm{x},\mathrm{y}\right)=\delta \cdot \mathrm{L}\mathrm{B}\mathrm{P}\left(\mathrm{x},\mathrm{y}\right)+\gamma \cdot \mathrm{a}\mathrm{v}\mathrm{g}\left(\mathrm{W}\left(\mathrm{x},\mathrm{y}\right)\right)+\left(1-\gamma -\delta \right)\cdot \mathrm{D}\mathrm{A}$$

(15)

$$\mathrm{D}\mathrm{A}\left(\nabla \mathrm{I},{\mathrm{A}}_{\mathrm{i}\mathrm{j}}\right)=\alpha \cdot {\nabla }_{\mathrm{x}}\mathrm{I}+\left(1-\alpha \right)\cdot {\mathrm{A}}_{\mathrm{i}\mathrm{j}},\beta \times {\nabla }_{\mathrm{y}}\mathrm{I}+(1-\beta )\times {\mathrm{A}}_{\mathrm{i}\mathrm{j}}$$

(16)

where $\mathrm{L}\mathrm{B}\mathrm{P}\left(\mathrm{x},\mathrm{y}\right)$ represents the LBP feature for each pixel, $\mathrm{a}\mathrm{v}\mathrm{g}\left(\mathrm{W}\left(\mathrm{x},\mathrm{y}\right)\right)$ is the average intensity value within the local window, DA$\left(\nabla \mathrm{I},{\mathrm{A}}_{\mathrm{i}\mathrm{j}}\right)$ is the function obtained after adjusting the projection direction using gradient information and the multi-head self-attention mechanism, where $\alpha$, $\delta$, $\gamma$ and $\beta$ are weight parameters.

Perform pixel labeling based on the threshold values.

$$\left\{\begin{array}{ll}\mathrm{StrongEdge}\hfill & \mathrm{F}\left(\mathrm{x},\mathrm{y}\right)\ge {\mathrm{T}}_{\mathrm{high}}\left(\mathrm{x},\mathrm{y}\right)\hfill \\ \mathrm{PotentialEdge}\hfill & {\mathrm{T}}_{\mathrm{low}}\left(\mathrm{x},\mathrm{y}\right)\le \mathrm{F}\left(\mathrm{x},\mathrm{y}\right)<{\mathrm{T}}_{\mathrm{high}}\left(\mathrm{x},\mathrm{y}\right)\hfill \\ \mathrm{NonEdge}\hfill & \mathrm{F}\left(\mathrm{x},\mathrm{y}\right)<{\mathrm{T}}_{\mathrm{low}}\left(\mathrm{x},\mathrm{y}\right)\hfill \end{array}\right.$$

(17)

2.5. Auxiliary Loss Function

For adaptive thresholds, the threshold is not static but is determined based on the local attributes of the image. Consequently, we have redesigned the auxiliary function. This involves the incorporation of edge information to further modify the Dice Loss. The underlying principle is that the edges or boundaries of objects within an image are particularly crucial in many segmentation tasks. By incorporating these boundaries into the loss function, the network can be directed toward generating enhanced segmentations, particularly along the edges of objects. The formulation is as follows:

$$\mathrm{L}={\mathrm{L}}_{\mathrm{D}}+\mathrm{k}\times {\mathrm{l}}_{\mathrm{E}}$$

(18)

${\mathrm{L}}_{\mathrm{D}}$ is computed between the prediction and the ground truth.

${\mathrm{l}}_{\mathrm{E}}$ represents the Dice Loss calculated between the predicted edge map (obtained through canny edge detection) and the ground truth.

$\mathrm{k}$ is a hyperparameter employed to determine the degree of emphasis on the edge Dice Loss within the final loss value.

In this paper, numerical stability is maintained by introducing the ‘smooth’ parameter. In instances where both the prediction and the ground truth lack discernible features, the Dice Loss may assume an indeterminate form of $0/0$. By adding a small “smooth” value to both the numerator and the denominator, such scenarios are circumvented, ensuring the stability of the loss value.

In summary, this enhanced Dice Loss leverages both global segmentation information and local edge details to yield superior segmentation outcomes, especially around object boundaries. Incorporating an edge-weighted term in the loss function is anticipated to guide the model to better handle boundaries and provide more accurate segmentation.

3. Experiment

3.1. Study Area

China, located in the southeastern part of the Asian continent, borders the Northwest Pacific Ocean. With nearly 3 million square kilometers of maritime territory and a coastline stretching over 32,000 km, China ranks sixth in the world. This coastline is bifurcated into mainland coastlines and island coastlines, with the former accounting for 18,000 km [56,57,58]. From 1980 to 2015, China’s coastline expanded by 3000 km. However, its natural coastline decreased by approximately 50% during the same period [57,59,60]. This transformation can be attributed to various factors, including monsoonal waves, tectonic uplift and subsidence, and human activities. Notably, the coastal regions, known for their rich resources and transportation convenience, accommodate only 40% of the country’s population but contribute 70% to China’s GDP, showcasing one of the most robust economic activities globally.

This study primarily examines the mainland coastline of China, as illustrated in Figure 5. The study area extends from the Yalu River Estuary in the north, situated on the China–North Korea border, to the Beilun Estuary in the south, which marks the China–Vietnam border. It encompasses four seas: the Bohai, Yellow, East, and South China Seas. It spans across 14 primary provinces, municipalities, and autonomous regions, cutting through three climatic zones: temperate, subtropical, and tropical. Based on geological and geomorphic features, coastlines can primarily be categorized into four types: sandy coasts, bedrock coasts, muddy coasts, and artificial coasts. From this, it is evident that China’s coastline is elongated and intricate, making extraction quite challenging.

Due to substantial topographical differences between northern and southern China, we chose datasets from Fujian Province, Guangxi Province, and Shandong Province to be the test set for evaluating our model’s generalizability comprehensively. The datasets from the other provinces were randomly divided into training and validation sets at a 4:1 ratio. The validation set adjusts model hyperparameters and monitors model performance during training. After each training epoch, the model is evaluated on the validation set to assess its ability to generalize to unseen data. The test set, consisting of data not encountered during training and validation, is used for the model’s final performance evaluation. It assesses the model’s ability to generalize to real-world data and evaluates the overall performance of the model. The experimental data in this study come from 103 multispectral images of the mainland China coastal zone captured by the “Landsat 8” satellite in 2020 [61,62].

After acquiring the images, various preprocessing steps were undertaken. These steps encompass orthorectification, image fusion, mosaicking, and cropping. A standard false-color composite image composed of Band 5, Band 4, and Band 3 was chosen for a more precise capture of coastline features. It was ensured that each selected image exhibited diverse coastline characteristics. Ground truth maps for land–sea segmentation of these images were created through expert visual interpretation. Following cropping of the preprocessed remote sensing images to create samples measuring 256 × 256 pixels, a total of 65,389 sample images were obtained. However, given the extensive scale of the data and the high costs associated with training, this study additionally employed a threshold method and expert visual recognition to filter out images containing solely a single landform feature. Within each remote sensing image, pixel values in the binary label map are set to 0 and 255. The threshold segmentation algorithm removes label maps with uniform gray values of 0 or 255, along with their original remote sensing images. Moreover, black borders in downloaded remote sensing images may misinterpret label images containing 0 and 255 gray values during screening, requiring manual scrutiny of the original image. Only images that concurrently showcased both land and sea features were retained. After this selection process, the final training set consisted of 4637 samples, the validation set contained 989 samples, and the test set had 1922 samples.

3.2. Coastline Dataset

Extracting coastlines through deep learning necessitates the creation of precise remote-sensing image labels, constituting a pivotal step in the process. Ensuring accuracy in label creation is imperative for effectively enabling deep networks to discern terrain features during the training phase. In the past, when extracting coastlines from satellite remote sensing images, the instantaneous water boundary observed during satellite passes has frequently been regarded as the accurate coastline [63,64]. This differs significantly from the definition of the actual coastline as the sea–land boundary at the moment of the average high tide over several years. In order to objectively and accurately extract coastlines that closely resemble the actual coastline, this study utilizes the five coastal-type (

Table 1. Five types of coastline.

Coastline	Sample	Interpretation Signs	Location
Bedrock Coastline		Distinct concave-convex and mountainous texture features	The evident land–water boundary
Silty Coastline		Appears grey or white with a smooth texture	A boundary that is located in an estuary, delta, or low-lying area and has a marked contrast in vegetation density
Sandy Coastline		Clear dividing line between white and other colors	Beach ridges and lack of beach ridges, the beach is directly adjacent to the cliffs of the bedrock shoreline
Biogenic Coastline		Red tones, darker textures, and irregular shapes	It is mainly distributed in Guangdong, Guangxi, Fujian, Taiwan, and parts of Hainan Island
Artificial Coastline		Bright and white structures, smooth textures and narrow stretches, regular layouts, and colors ranging from light beige to tan or even white	A man-made coastline exhibits a multitude of features, which are often intricate and require thorough analysis and consideration

) remote sensing interpretation markers proposed by Sun Weifu to construct a deep learning dataset [65].

3.3. Implementation Details

The experiments were conducted on a computer equipped with an NVIDIA RTX 3060 GPU with 12 GB of VRAM and running the Ubuntu 18 operating system. All models were trained and tested using the PyTorch framework. During the training process, the Adam optimizer was employed to minimize the loss. The initial learning rate was set to 1 × 10⁻³, with the number of iterations set at 100 epochs and a batch size of 8.

3.4. Evaluation Metrics

In this study, we utilize Intersection over Union (IoU), mean Intersection over Union (MioU), Frequency Weighted Intersection-over-Union (FWIoU), and Overall Accuracy (OA) to evaluate the performance of the model. These five evaluation metrics are derived from the confusion matrix, which comprises true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). Below are the computational formulas for each metric.

For each class, IoU is defined as the ratio of the intersection to the union of the predicted and true values. The formula is as follows:

$$\mathrm{I}\mathrm{o}\mathrm{U}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(19)

The MIoU represents the average result of the IoU for each class. Its calculation is given by:

$$\mathrm{M}\mathrm{I}\mathrm{o}\mathrm{U}=\frac{{{\displaystyle \sum }}_{\mathrm{i}}^{\mathrm{n}}\mathrm{I}\mathrm{o}\mathrm{U}}{\mathrm{n}}$$

(20)

FWIoU assigns weights based on the frequency of occurrences for each class. The weight is multiplied by the IoU for each class and then summed up. The formula for this is as follows:

$$\mathrm{F}\mathrm{W}\mathrm{I}\mathrm{o}\mathrm{U}=\frac{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{N}}\times \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(21)

The F₁-Score (F₁) takes into account both precision and recall, aiming for a balance that maximizes both. The formula for the F₁ or each class is as follows:

$${\mathrm{F}}_1=2\times \frac{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}$$

(22)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}$$

(23)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(24)

Accuracy (ACC) indicates the proportion of correctly predicted samples among all samples. This includes both true positive and true negative predictions. The calculation for accuracy is as follows:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{N}}$$

(25)

4. Results

4.1. Performance of Daenet

This paper contrasts the performance of DAENet with several existing methods, including Segmenter [66], Mask2Former [67], and Swin-UNet. All three methods are built upon the Transformer architecture. Specifically, Mask2Former is a hybrid structure based on Mask R-CNN and Transformer, while Swin-UNet constitutes a UNet structure formed purely from Swin Transformer modules. The three methods, PIDNet [68], DDRNet [69], and SegNeXt [70], are state-of-the-art models established based on CNN. This study will analyze the accuracy and adaptability of the DAENet model from two perspectives:

(1) Conducting an analysis using evaluation metrics and results to ascertain and validate its accuracy.

The results for the Shandong Province dataset are presented in

Table 2. Comparison of segmentation accuracy on the Shandong Province dataset.

Method	IoU(%)		F1 (%)		Evaluation Index
	Ocean	Land	Ocean	Land	MioU (%)	OA (%)	FWIoU (%)
Segmenter	90.49	94.09	92.28	92.21	85.61	92.25	85.61
SegNeXt	95.67	93.84	94.71	94.77	90.01	94.74	90.01
PIDNet	91.74	92.06	95.69	95.86	91.90	95.78	91.90
DDRNet	92.18	92.48	95.93	96.09	92.33	96.66	92.33
Swin-UNet	88.59	88.33	93.95	93.80	88.46	93.88	88.46
Mask2Former	88.53	88.44	93.91	93.86	88.48	93.89	88.48
DAENet	93.85	94.04	96.82	96.93	93.94	96.88	93.94

, which showcases the numerical outcomes for each semantic segmentation method. The findings indicate that DAENet outperforms other techniques in metrics such as IoU, MIoU, FWIoU, and overall OA.

For the Mask2Former model, which has achieved SOTA results in semantic, instance, and panoptic segmentation as a unified segmentation structure, Table 2,

Table 3. Comparison of segmentation accuracy on the Fujian Province dataset.

Method	IoU (%)		F1 (%)		Evaluation Index
	Ocean	Land	Ocean	Land	MioU (%)	OA (%)	FWIoU (%)
Segmenter	82.03	84.20	89.90	91.16	83.12	90.82	83.16
SegNeXt	83.71	85.64	91.13	92.26	84.68	91.13	84.72
PIDNet	84.99	86.89	91.88	92.98	85.94	92.47	85.97
DDRNet	85.43	87.30	92.14	93.22	86.37	93.53	86.40
Swin-UNet	83.22	85.29	90.84	92.06	84.25	91.49	83.30
Mask2Former	82.08	84.68	90.16	91.70	83.38	91.00	83.44
DAENet	87.28	88.84	93.20	94.09	88.06	93.68	88.09

and

Table 4. Comparison of segmentation accuracy on the Guangxi Province dataset.

Method	IoU (%)		F1 (%)		Evaluation Index
	Ocean	Land	Ocean	Land	MioU (%)	OA (%)	FWIoU (%)
Segmenter	69.28	77.90	81.85	87.58	73.59	85.25	74.28
SegNeXt	69.70	77.88	82.14	87.56	73.79	85.34	74.45
PIDNet	72.03	81.63	83.74	89.88	77.68	87.53	77.68
DDRNet	72.72	82.02	84.20	90.12	77.37	87.84	78.16
Swin-UNet	74.54	82.88	85.41	90.63	78.71	88.59	79.38
Mask2Former	71.20	81.21	83.18	89.63	76.21	87.17	77.01
DAENet	76.39	84.91	86.61	91.84	80.65	89.86	81.33

reveal its performance on coastline data extraction is not especially commendable. As a comprehensive, large-scale model designed to incorporate a wide array of features, it unavoidably sacrifices precision when isolating specific ones. Consequently, it loses its SOTA edge when extracting the requisite edge information for coastline detection.

In reference to the Swin-UNet, predominantly employing the Swin Transformer module for semantic segmentation, the model, despite its superior global modeling capability, necessitates revision when configured according to the Swin-UNet design for remote sensing imagery [58]. This deficiency becomes apparent in Figure 6, Figure 7 and Figure 8, illustrating fragmented semantic segments in boundary extraction, a common issue stemming from ambiguity in edge information. However, upon integrating the LAMBA and MST modules into our DAENet, Figure 6, Figure 7 and Figure 8 demonstrate more accurate edge extraction and the absence of fragmented semantic segments, thereby reinforcing the model’s robustness in edge detection.

With regard to the straightforward, efficient, and durable semantic segmentation model, Segmenter, our assessments have indicated that its functionality is marginally inadequate. As depicted in Figure 6 (Segmenter. e), although it capably handles simple linear coastlines with minimal discrepancies, its proficiency significantly decreases when confronting complex feature patterns, as exemplified in sections a, f, and g of the same Figure 6, Figure 7 and Figure 8. In comparison, DAENet’s performance remains superior for dense and intricate terrestrial entities, as evident in Figure 6, Figure 7 and Figure 8.

(2) Conducting analysis across diverse regions to assess the adaptability of the model:

First, China’s coastlines can be primarily classified into three types based on geological and geomorphic characteristics: sandy coasts, bedrock coasts, and artificial coasts. As depicted in Figure 6b,c for artificial coasts and Figure 7 for sandy and bedrock coasts, the coastline morphologies extracted by our DAEnet demonstrate superior performance compared to other methods.

Secondly, we also analyzed results from other types of coastlines. During the generation of estuary labels, especially in manual labeling processes, if narrow rivers extend over a considerable distance, our usual approach involves capturing either a segment of the estuary shoreline or rectifying the shoreline directly. Nevertheless, when confronted with images containing incomplete labeling (see Figure 7h) or featuring coastlines marked by aquaculture ponds (see Figure 6a,h), as well as other complex coastal types, we are nonetheless able to accurately discern the correct shoreline.

Lastly, as illustrated in Figure 7, the coastline of Fujian Province is notably intricate, winding its way in an exceptionally complex manner, making it the most complex coastline in mainland China. Despite this challenging topography, our DAENet consistently performs at a high level, with all performance metrics exceeding 84%, establishing it as a leading method in the industry. Additionally, we predicted the coastline of Guangxi Province using the DAENet model, with extracted results demonstrating superior performance compared to other models, confirming the reliability of DAENet. With sufficient data, we will be fully equipped to extract coastlines from across the country and monitor their dynamic changes.

4.2. Ablation Study

To evaluate the performance of the proposed network architecture and its three crucial modules, we utilized UNet as the foundational network for conducting an ablation study on the dataset. Furthermore, we explored the impact of the loss function on the proposed network. The subsequent comparison results indicate that the integration of the proposed MST, LAMBA, and DAT modules yields significant performance enhancements in detection (Figure 9).

(1) Impact of the Deformable Self-Attention Module: As presented in

Table 5. Comparison of ablation results.

Method	IoU (%)		${\mathbf{F}}_1$
	Ocean	Land	Ocean	Land
U-Net	83.21	83.87	90.84	91.23
Swin-UNet	88.59	88.33	93.95	93.80
Swin-UNet + LAMBA	88.93	88.65	94.14	93.98
Swin-UNet + MST	89.16	89.34	94.11	94.01
Swin-UNet + LAMBA + MST	90.21	90.14	94.85	94.81
Dat-UNet	89.66	89.76	94.55	94.60
Dat-UNet + LAMBA	92.33	92.41	96.01	96.06
Dat-UNet + MST	92.89	92.42	95.98	96.04
Dat-UNet + LAMBA + MST	93.85	94.04	96.82	96.93

, introducing the Deformable Attention Transformer (DAT) effectively augments the segmentation performance of the UNet structure. There is an improvement ranging from 5.69% to 6.45% in the IoU accuracy metric relative to the original baseline model. The enhancement in accuracy ranges from 9.42% to 10.28%, providing substantial evidence for the effectiveness of integrating DAT. This enhancement is attributed to the feature maps first undergoing processing via window-based local attention, facilitating local information aggregation. Subsequently, the Deformable Attention block models the global relationships among the locally enhanced tokens. This alternative attention block design, equipped with local and global receptive fields, aids the model’s learning process.

(2) The incorporation of the multi-scale transformation module (MST) into the Swin-UNet + LAMBA and Dat-UNet + LAMBA models yields significant results. As illustrated in Table 5, the integration of the multi-scale deformable edge detection module significantly enhances the segmentation performance of the UNet architecture. Relative to the original model, there is an improvement of 1.28% to 1.93% in the IoU accuracy metric. For the accuracy metric, the enhancement ranges from 1.01% to 1.72%. These outcomes underscore that integrating the MST module facilitates the model by capturing a richer set of feature information.

(3) Impact of the LAMBA Module: The LAMBA module was, respectively, integrated into the Swin-UNet and Dat-UNet models. As depicted in Table 5, the adaptive edge detection module’s introduction bolsters the segmentation performance of the UNet structure. Compared to the original model, there is a boost of 1.03% to 2.67% in the IoU accuracy metric. Regarding the accuracy metric, the uplift spans from 1.07% to 1.45%. These findings highlight that introducing the LAMBA module amplifies the model’s proficiency in edge delineation.

5. Conclusions

In this study, we propose DAENet, a deep learning model that combines semantic segmentation networks with edge detection to address inaccuracies in coastline extraction and localization. To enhance the model’s feature representation, we introduce the multi-scale transformation (MST) module, which incorporates canny edge detection across multiple spatial scales as input channels. By integrating MST into the U-shaped network structure, our model gains improved global modeling capability compared to traditional CNNs and patch-based Transformers. Additionally, our novel LAMBA module focuses on capturing edge features in the spatial dimension to mitigate semantic ambiguity caused by unclear object boundaries. We refine the binary Dice Loss function to expedite convergence and compute an edge-perceptive loss utilizing Canny edge maps to augment performance on edges further. Experimental results demonstrate that DAENet outperforms traditional models like Segmenter, SegNeX, PIDNet, DDRNet, Swin-UNet, and Mask2Former. Compared to the traditional model, Swin-UNet, DAENet shows a 5% improvement in MIoU.

To our knowledge, the proposed DAENet model is the first to apply the DAT block for remote sensing sea–land segmentation. It addresses the limitations of pure CNNs and enhances segmentation accuracy. The proposed network model can be effectively applied to precise positioning tasks for various complex coastal types in different regions, demonstrating its potential for coastal dynamic management and planning. Furthermore, the unique dataset created in this study allows the extracted results to approximate the actual coastline closely.

However, our model has several limitations. (1) DAENet extensively uses the Deformable Attention module, resulting in a larger parameter set and slightly longer training durations than other methods. This may limit DAENet’s use in compact mobile devices, but it still offers valuable insights into the roles of Deformable Attention in remote sensing semantic segmentation. In future research, we will design more accurate geometric prior models and loss functions for SLS segmentation features or generate multi-scale features through style transfer to accelerate model convergence. (2) DAENet still requires improvements in object boundary extraction. The deficiencies mainly appear in the segmentation results, where we aim to explore advanced encoding techniques for boundary features to overcome this limitation. Augmented outcomes deviate from the actual shape of the objects and show slight noise. Additionally, we will prioritize implementing model compression methods to enhance inference efficiency. Overall, the goal is to accelerate model convergence while retaining its deformable characteristics.