A Practical Deep Learning Architecture for Large-Area Solid Wastes Monitoring Based on UAV Imagery

Abstract

The development of global urbanization has brought about a significant amount of solid waste. These untreated wastes may be dumped in any corner, causing serious pollution to the environment. Thus, it is necessary to accurately obtain their distribution locations and detailed edge information. In this study, a practical deep learning network for recognizing solid waste piles over extensive areas using unmanned aerial vehicle (UAV) imagery has been proposed and verified. Firstly, a high-resolution dataset serving to solid waste detection was created based on UAV aerial data. Then, a dual-branch solid waste semantic segmentation model was constructed to address the characteristics of the integration of solid waste distribution with the environment and the irregular edge morphology. The Context feature branch is responsible for extracting high-level semantic features, while the Spatial feature branch is designed to capture fine-grained spatial details. After information fusion, the model obtained more comprehensive feature representation and segmentation ability. The effectiveness of the improvement was verified through ablation experiments and compared with 13 commonly used semantic segmentation models, demonstrating the advantages of the method in solid waste segmentation tasks, with an overall accuracy of over 94%, and a recall rate of 88.6%—much better than the best performing baselines. Finally, a spatial distribution map of solid waste over Jiaxing district, China was generated by the model inference, which assisted the environmental protection department in completing environmental management. The proposed method provides a feasible approach for the accurately monitoring of solid waste, so as to provide policy support for environmental protection.

1. Introduction

With the continuous increase in global population and the accelerated process of urbanization, the amount of urban solid waste is also increasing [1,2]. Managing and handling these solid wastes has become a global challenge [3]. Due to the high cost of waste disposal and the drive for illegal profits, phenomena such as unauthorized dumping and littering of solid waste, as well as the establishment of abandoned disposal sites, are common [4,5]. Untreated solid waste, after being infiltrated by rainwater and undergoing decomposition, results in leachate and harmful chemicals entering the soil through surface runoff, which disrupts soil structure and causes pollution to nearby rivers and groundwater, leading to serious ecological impacts [6,7]. Therefore, it is essential to regulate and govern these illegal solid waste materials, as this work holds significant importance for the construction of urban ecological civilization and sustainable development. To conduct large-area solid-waste management, accurate acquisition of the distribution locations and rugged edge information is highly required, so as to provide policy support for estimating the workload and generating specific strategies for their treatment.

The previous approaches for monitoring solid waste dumps relied on manual inspections and video surveillance, both of which have many drawbacks [8]. These approaches have limited detection range and cannot comprehensively cover monitoring areas, especially in remote areas such as suburbs, mountains, riverbanks, and road edges [9,10]. Inasmuch, many scholars have applied remote sensing technology to relevant research, such as Gill et al. [11], who used thermal infrared remote sensing technology to measure land surface temperature (LST) and outline the most likely dumping areas within landfill sites. However, the sensitivity of temperature difference detection is low, and the LST changes in small garbage heaps are not significant. Other scholars have considered feature extraction approaches, such as Begur et al. [12], who proposed an edge-based intelligent mobile service system for detecting illegal dumping of solid waste. However, the accuracy of such algorithms depends on manually designed features, and although they perform well on some samples, they cannot be applied to large-area automated identification due to the complex and diverse structures and textures of garbage heaps. In addition, satellite imagery as a data source is prone to cloud cover, and the limitations of revisit periods restrict the application of related methods [13]. This paper utilizes unmanned aerial vehicle (UAV) to gather data, which have the characteristics of high resolution, wide monitoring range, and real-time capabilities [14,15]. Moreover, UAV can perform monitoring in hard-to-reach areas, providing a solid data foundation for extracting characteristic information of solid waste and identifying targets in various garbage scenarios [16,17].

In recent years, deep learning technology has gradually matured and is widely used in various complex scenarios due to its powerful feature learning and representation capabilities compared to conventional methods [18,19]. Some scholars have introduced it into the research field of garbage classification [20,21,22,23,24]. Current research mostly regards deep learning as three tasks: target detection [25,26], image classification [27,28,29], and weakly supervised segmentation [30,31]. For example, Yu et al. [32] proposed a street garbage detection algorithm based on remote sensing satellite SIFT feature image registration and fast regional convolutional neural networks. Shi et al. [29] proposed a waste classification method based on a multi-layer hybrid convolutional neural network, changing the number of network modules and channels to improve the model’s performance. Niu et al. [33] proposed a weakly supervised learning method that utilizes multi-scale dilated convolutional neural networks and Swin-Transformer to aggregate local and global features for rough extraction of solid waste boundaries. However, these methods cannot obtain the actual distribution edges of solid waste piles to achieve pixel-level classification, and are generally limited to small-scale areas or single scenarios [22,34,35]. Therefore, they are not suitable for monitoring and controlling large-area illegal solid waste that this study focuses on. Semantic segmentation can obtain detailed edges of segmented objects, but there is a problem of missing high-quality training images [23,24,30], and existing semantic segmentation models are mainly used in street scenes such as Cityscapes [36], CamVid [37], Mapillary Vistas [38], and BDD100K [39]. However, solid waste piles are of different scales, diverse types, varied shapes, and widely distributed, which greatly increase the difficulty of existing models to distinguish objects from backgrounds. It remains questionable whether existing models can be applied to the segmentation of large-area solid waste.

In summary, the semantic segmentation method for large-area solid waste piles is still in the development stage and requires further in-depth research. This paper conducts research in the Nanhu District of Jiaxing City. Compared with existing research, the contribution of this work can be summarized as follows:

(1)

Approximately 450 km² of high-resolution images of the Nanhu District in Jiaxing City and some neighboring areas were collected using drones, and a high-precision solid waste pile segmentation dataset with multi-scene distribution was created by pixel-wise labeling.

(2)

A dual-branch structured semantic segmentation model was proposed, which achieves a more comprehensive feature representation by fusing contextual feature branches and spatial feature branches. In comparison with other semantic segmentation methods, it achieves state-of-the-art performance in the high-resolution solid waste pile segmentation dataset, demonstrating excellent capability in segmenting complex edges of solid waste piles in multiple scenes.

(3)

This work provides new ideas for large-area and high-precision solid waste monitoring, effectively reducing the cost of manual inspection and supervision. It can provide scientific theoretical support for the ecological civilization construction and sustainable development of cities, and play an active role in protecting land resources and the ecological environment.

2. Study Area

Nanhu District, Jiaxing City, is located in the northern part of Hangjiahu Plain in Zhejiang Province. The central coordinates of its geographic location are 120°76′ E and 30°76′ N. It covers a total land area of 368.3 square km and includes 7 streets and 4 towns (Figure 1). The district has a permanent population of 560,000 people. With the development of the economy and its location within the Yangtze River Delta urban agglomeration, Nanhu District, acting as the economic, political, cultural, and commercial center of Jiaxing City, has undergone significant changes. As a result, with the rapid increase in construction and population, the issue of solid waste in the city has become increasingly apparent. There is an urgent need to develop high accuracy monitoring methods for solid waste, providing more intelligent technical means for environmental protection in this region.

3. Methods

3.1. Dataset

3.1.1. Data Acquisition

This study has constructed a high-resolution solid waste semantic segmentation dataset (RSD). The data source consists of aerial images of Nanhu District, Jiaxing City, captured by UAV in June 2021. After geometric correction and stitching, a total of 1210 GB of imagery was obtained, covering an area of approximately 450 km², with a spatial resolution of 0.038 m. The UAV model used in this study is DJI Phantom 4 Pro, and the flight altitude is 90 m.

3.1.2. Data Labeling

The labeling work consists of two steps: preliminary sample area selection and pixel-by-pixel labeling. The former involves creating a mask through visual interpretation (without the need for precise boundaries) and then using the mask to clip the corresponding area of UAV images as the preliminary sample area. The latter involves detailed outlining of the edges of solid waste to create labeled data. All training dataset are from the preliminary sample area. The selection rules for the preliminary sample area can be summarized as follows:

(1)

Area Selection: Priority is given to identifying large, concentrated areas. In the sample area selection, garbage piles with an area of at least 1 square meter, as determined by expert judgment, and consisting of more than 900 pixels based on the spatial resolution, are considered. If there are objects of similar nature near a large garbage pile, they are also considered as garbage.

(2)

Covering Multiple Scene Types: Solid waste piles commonly found in daily life are widely distributed in various environments such as buildings, roads, grasslands, rivers, and forests. In order for the model in this study to possess the ability to recognize solid waste in multiple scene types, the sampling areas need to include various scene types.

(3)

Balancing Garbage Category Proportions: Based on the overview of the entire image, garbage can be classified into three categories: construction waste, household waste, and mixed waste. Construction waste includes materials such as wood, bricks, and concrete generated during construction and demolition processes. Household waste includes garbage such as plastic, kitchen waste, and paper generated in daily life. Mixed waste refers to garbage that is a mixture of different sources and types. This study does not distinguish between different types of garbage, but when creating sampling areas, it is necessary to maintain a balance in the quantities of these three types to meet monitoring requirements.

3.1.3. Training Dataset

The images and labels are divided into synchronized image pairs according to the common deep learning pixel standard of 256 × 256 format. Then, the labels are filtered using histogram pixel value statistics to remove image pairs without solid waste. In this study, a total of 2600 image pairs were retained and named RSD. Figure 2 demonstrates that the distribution details of solid waste can be clearly represented. Additionally, during network training, the RSD dataset is randomly divided into three subsets based on quantity: RSD-Train (2000), RSD-Val (300), and RSD-Test (300), which are used for training, validation, and testing, respectively. Since the RSD dataset has not undergone any data augmentation operations and comes from different regions, the images in the three subsets do not overlap. The random classification ensures that the scales and distribution environment proportions of solid waste in the three subsets are relatively consistent.

3.2. Segmentation Model

This paper proposes a new semantic segmentation algorithm based on UAV imagery to construct a solid waste segmentation model (SWM). The model consists of a backbone, context feature branch, spatial feature branch, feature fusion module, and segmentation head (Figure 3). Firstly, the input image is encoded through multiple layers of operations by the backbone to 1/8 of the original scale, with a tensor shape of [128, 32, 32]. Then it is fed into a dual-branch structure, where the Context feature branch extracts global semantic features, and the features are down-sampled. In addition, the feature map scales of stage 1 and stage 2 are 1/16 and 1/32, respectively. The Spatial feature branch extracts spatial detail information without changing the feature map scale. The dual branches communicate through the fusion block to achieve information exchange and feature fusion. Finally, the segmentation head produces the output, which represents the segmented solid waste result.

3.2.1. Backbone and Segmentation Head

The feature backbone (Figure 4) of the SWM is designed based on the concept of residual network [40]. It extracts features through convolution, downsampling, and residual blocks. The size of the feature map gradually decreases while the number of channels increases. Each level of features is added to the deep features in the decoding stage through skip connections. Finally, the upsampled output generates end-to-end pixel-level prediction results through logarithmic SoftMax, which converts the values into probability distributions.

3.2.2. Context Feature Branch

In high-resolution imagery, the overall visual interpretation characteristics of solid waste include: first, solid waste is prone to be confused with background elements such as buildings and roads; second, solid waste is deeply integrated into the background and often distributed within other land-use types, especially open spaces and lawns. Its identification relies on specific contextual information. To address these two issues, we designed a global semantic module block (GSM Block). This block enables the model to extract global contextual features, which refer to the information across the entire image and are used to understand the relationship between the target object and its surrounding environment. Additionally, it facilitates the extraction of solid waste from the background using an attention mechanism. As a result, the model gains the ability to effectively distinguish solid waste from other elements in the image.

The squeeze block, designed in this paper, is used to perform global feature condensation (Figure 5). The input feature IN $\in {\mathrm{R}}^{H\times W\times C}$ is divided into three parts: $\mathrm{q}=W_q\mathrm{i}\mathrm{n}$, $\mathrm{k}=W_k\mathrm{i}\mathrm{n}$, $\mathrm{v}=W_v\mathrm{i}\mathrm{n}$. Where, $W_q\in {\mathrm{R}}^{C_q\times C}$, $W_k\in {\mathrm{R}}^{C_k\times C}$, and $W_v\in {\mathrm{R}}^{C_v\times C}$ are learnable weights. $x^{\downarrow (h,w,c)}$ represents the transformation of tensor x to dimensions (h,w,c), $w$ denotes a vector with all elements being 1.

Global features are obtained by performing dimension transformation and taking the mean along the horizontal and vertical directions of tensor q. The formula for calculating the mean along the horizontal direction is:

$${\mathrm{q}}_h={\left(q^{\downarrow \left(C_q,H,W\right)}{1}^W\right)}^{\downarrow (H,C_q)}/w$$

(1)

The formula for calculating the mean along the vertical direction is:

$${\mathrm{q}}_w={\left(q^{\downarrow \left(C_q,W,H\right)}{1}^H\right)}^{\downarrow (W,C_q)}/H$$

(2)

Similarly, applying this set of formulas to tensors k and v, we obtain the horizontal mean values ${\mathrm{k}}_h$ and ${\mathrm{v}}_h$, as well as the vertical mean values ${\mathrm{k}}_w$ and ${\mathrm{v}}_w$. Next, the compressed feature maps are multiplied together to obtain a H × W × C feature map. Then, the product is element-wise multiplied with the input feature to weight the correlations between different dimensions.

After completing the global feature condensation, we need to enhance attention by introducing global attention extraction. $P_{i,j}$ represents a position on the feature map:

$$P_{i,j}={\sum }_1^{H}softmax({{\mathrm{q}}_h}^T·{\mathrm{k}}_h){\mathrm{v}}_{\mathrm{h}}+{\sum }_1^{W}softmax({{\mathrm{q}}_v}^T·{\mathrm{k}}_v){\mathrm{v}}_v$$

(3)

The above design amplifies the “global” cross-dimensional interactions, ensuring the ability to reflect the spatial location or feature channel importance of solid waste, allowing its information to be propagated and amplified in subsequent feature maps.

3.2.3. Spatial Feature Branch

In high-resolution images, the edges of solid waste piles do not have regular shapes. Therefore, deformable convolution concept [41] was introduced into the model to design the spatial feature branch, which is aimed at adaptively extracting high-resolution spatial features. Traditional convolutional kernels have a fixed shape and cannot adapt to irregular target shapes or local structural changes. In contrast, deformable convolution allows for adaptive adjustment of sampling positions based on the characteristics of the input data, thereby better capturing spatial information of the target.

The spatial feature branch consists of DCR (Deformable separable Convolution with Regularization) blocks, corresponding to the structure B in Figure 3. Firstly, pointwise convolutions (1 × 1 convolutions) are used to extend the tensor dimension from 128 to 384, then which are further processed by deformable separable convolution + BatchNorm + GeLU (Gaussian Error Linerar Units, activation function), and finally reduced back to 128 using pointwise convolutions. The so-called DCR here is just a minor modification to the existing MobileNet Block equipped by MobileNet V2 [42]. In this study, the depthwise separable convolutions that compose the MobileNet Block in MobileNet V2 were replaced with deformable separable convolution. Since solid waste generally has irregular edges, it is better to use deformable convolution kernels that follow its trajectory. The benefits of pointwise convolutions lie in enhancing features by widening and then consolidating tensor dimensions.

The operational steps of the designed deformable separable convolution in this study are as follows: Firstly, a small convolution operation is performed to predict the spatial offset parameters for the input feature map. The offset parameters $p_{k }^x$ and $p_k^y$ are predicted for the k-th channel at point P. Then, the position adjustment of sampled points is applied to the input feature map, generating new sampled point coordinates $x_{p_k^x+i,p_k^y+j}^k$. Next, a channel convolution operation is performed on the input feature map that has undergone deformable sampling to predict the weights $W_k^c$ for each channel. Finally, based on the predicted channel weights, a spatial convolution operation is performed on the sampled feature map to generate the final feature map. $P_{i,j}^c$ represents the pixel value at the i-th row and j-th column of the c-th channel in the output feature map, and $C_{in}$ represents the number of channels in the input feature map. The deformable separable convolution function can be represented as:

$$P_{i,j}^c={\sum }_1^{C_{in}}(W_k^c·x_{p_k^x+i,p_k^y+j}^k)$$

(4)

For each position in the input feature map, the designed DCR adjusts the sampling position of the convolution kernel based on the offset at that position, introducing positional variability. This allows the model to better adapt to the deformation of solid waste, thereby improving the model’s ability to handle the complex structures and irregular shapes of solid waste.

3.2.4. Feature Fusion Module

Two feature fusion mechanisms are designed to promote information interaction between branches in this study: the first involves downsampling and addition, where spatial features are passed to the contextual feature branch; the second mechanism involves upsampling contextual features, multiplying them with spatial features to obtain semantic features, which are then summed up with the original spatial features and passed to the spatial feature branch. This mainly involves multi-level fusion of DCR boundary information and GSM semantic information. The innovation of this part includes:

(1)

Different preprocessing scales. The GSM block is 1/16 in stage 1, 1/32 in stage 2, and the DCR is 1/8 in both stages. Since the global semantic information covers the entire image, the GSM block can focus on highlighting global semantic features in low-resolution feature maps, while spatial detail information mainly concentrates on local regions. The high-resolution feature maps allow the DCR to focus on highlighting spatial detail information. The separation and extraction mechanism makes feature extraction more efficient.

(2)

Multi-level fusion and complementary information. By conditioning the learning of the GSM block on DCR features, and conditioning the learning of DCR on SWR features, for the contextual branch, downsampling and global semantic information extraction will result in the loss of feature map detail information. However, spatial features from high-resolution can effectively supplement the detail information, addressing the blurring of feature maps caused by information loss. For the spatial feature branch with low-level detailed features, it typically contains a large amount of noise and redundant information. After communication, it can obtain high-level global semantic information at the low level and weight the detail information based on the importance of semantic information.

(3)

Improving the joint training-prediction accuracy of the GSM block and DCR block. Joint training framework refers to a training approach where multiple components are trained together in an integrated manner. Due to the dependency between the two branches, integrating them into a joint training framework can better utilize their complementary information, enhancing the model’s expressive power and prediction accuracy. It also reduces the reliance on the sample size.

4. Experiments and Discussion

4.1. Implementation Details

The networks in this paper are implemented on the Python-based deep learning framework PyTorch 1.11, using NVIDIA A40 with 48 G. To ensure fairness in method comparison, all network model hyperparameters are set uniformly: the number of training epochs is 100, the learning rate is 0.01, the batch size is 16, and the loss function uses cross-entropy loss. The model comparisons include BiSeNetv2 [43], ConvNeXt [44], Danet [45], DDRNet [46], Deeplabv3plus [47], Unet [48], FCHarDNet [49], GCN [50], NestedUnet [51], PSPNet [52], SFNet [53], SegFormer [54], and Swin Transformer [55]. These state-of-the-art models were proved to perform well in traditional semantic segmentation-related downstream tasks.

4.2. Accuracy Metrics

The evaluation metrics for method performance in this study include precision (Prec), recall (Rec), overall accuracy (OA), and F1 score. The F1 score is a comprehensive measure that takes into account both precision and recall, making it the primary metric for assessing the overall performance of the models.

$$F1=\frac{2\times Pre\times Rec}{Prec+Rec}$$

(5)

$$Prec=\frac{TP}{TP+FP}$$

(6)

$$Rec=\frac{TP}{TP+FN}$$

(7)

$$OA=\frac{TP+TN}{TP+TN+FP+FN}$$

(8)

where, TP represents the number of pixels detected as solid waste correctly, TN represents the number of pixels detected as background correctly, FP represents the number of pixels falsely detected as solid waste, and FN represents the number of pixels falsely detected as background.

4.3. Ablation Study

In the SWM network designed in this study, in addition to the conventional backbone and head, the proposed modules include the context feature branch, spatial feature branch, and fusion block. To verify their effectiveness, the combination of backbone and head is used as the baseline network.

According to the comparison of various methods, the results of the ablation study are obtained (

Table 1. Results of Ablation Study.

Method	Backbone + Head	Context	Spatial	Fusion	F1	OA	Rec	Prec
A	√				85.791	92.972	83.915	87.752
B	√		√		87.869	93.956	86.574	89.203
C	√	√			88.142	94.123	86.392	89.964
D	√	√	√		88.564	94.245	88.146	88.986
E	√	√	√	√	88.875	94.390	88.626	89.125

). Compared with the single module class and baseline network, Method A uses the MobileNet block structure for the dual-branch part of the baseline network, Method B adds the Spatial feature branch, and Method C adds the Context feature branch. Compared with Method A, Methods B and C show significant improvement in all four indicators, demonstrating better performance. This indicates that both the spatial feature branch and contextual branch can effectively improve the model’s feature extraction ability. Among them, Method C has a 2.351% increase in F1 score, which is higher than the 2.078% increase of Method B. The two methods are close in recall rate, and the main difference in performance improvement is that Method C has a larger increase in precision, which may be due to the fact that contextual features are more helpful in distinguishing solid waste from the background. Compared with the single module method group and double module method, both Methods B and C only add single modules, while Method D adds double modules. From Table 1, it can be found that Method D performs better than the single module group in comprehensive performance F1 score, with a recall rate increase of 1.57% and 1.75%, respectively. However, not all indicators have increased, and the precision indicator has decreased by 0.22% and 0.98%, respectively. This may be due to the fact that the dual-branch structure allows the model to obtain more information, which helps improve its ability to detect solid waste, but the interaction between the information reduces precision. Compared with the double module method and double module method with fusion block, Method D directly superimposes the information of the two modules in the channel dimension, while Method E designs two feature fusions. The comparison shows that Method E has better overall indicators than Method D. This indicates that a simple feature superimposition method cannot fully utilize the features extracted by the dual branch. By designing a feature fusion mechanism, Method E can make a combined use of the two types of information by interacting and cooperating between the dual branches, providing more comprehensive and accurate data representations and effectively improving the model’s prediction ability and accuracy.

4.4. Method Comparison

4.4.1. Quantitative Analysis

This study compares the SWM with 13 existing models (

Table 2. Accuracy comparison of methods.

Method	F1 (%)	OA	Rec	Prec
PSPNet	81.387	90.985	77.956	85.134
DAnet	83.076	91.632	81.23	85.008
DDRNet	83.246	91.546	83.068	83.424
BiSeNetv2	84.482	92.413	81.683	87.479
SegFormer	85.113	92.719	82.318	88.104
FCHarDNet	85.189	92.713	82.889	87.62
ConvNeXt	85.286	92.738	83.239	87.436
UNet	85.389	92.865	82.457	88.537
Deeplabv3plus	85.843	93.037	83.495	88.328
SFNet	86.094	93.162	83.724	88.602
SwinTransformer	86.173	93.071	85.405	86.956
NestedUnet	86.572	93.505	82.803	90.701
GCN	87.683	93.885	86.087	89.34
SWM	88.875	94.390	88.626	89.125

). Since there are many models involved, the advantages of SWM are discussed in two types of comparisons. #1: Comparison between the SWM and other feature fusion-based models. The compared methods include UNet, NestedUnet, SwinTransformer, DAnet, etc., which achieve feature fusion through skip connections or attention mechanisms. Among them, SwinTransformer and NestedUnet perform better. Compared with the proposed SWM in this study, SwinTransformer lags behind in all four indicators by 2.7%, 1.3%, 3.2%, and 2.2%, with overall performance falling behind; NestedUnet performs 2.3% worse than this study’s method in F1 value, although its precision is slightly higher than this method, its recall rate is 5.8% lower, indicating a large number of missed detections, making it unsuitable for large-area monitoring and control. The SWM in this study allows bidirectional information transmission, enabling the high-dimensional feature space to obtain detailed information during the encoding process to compensate for information loss during feature extraction, while allowing the low-dimensional feature space to gain high-dimensional semantic information to obtain more comprehensive feature representation. #2: Comparison between the SWM and other models based on feature extraction enhancement. Comparison methods include PSPNet, Deeplabv3plus, DDRNet, which adjust the convolution, pooling, and other operation strides to achieve a larger receptive field in the spatial domain, and DANet, BiSeNetv2, SegFormer, GCN, and other models that design some modules to improve their ability to extract contextual features. When comparing the SWM with the best-performing GCN model among these models, the SWM outperforms it by 1.2%, 0.5%, and 2.5% in terms of F1 value, precision, and recall rate, respectively. Although there is a decrease in precision of 0.2%, the SWM significantly reduces the number of missed detections. Overall, compared with the 13 existing methods, the main improvement of this study is a significant increase in recall rate, with the highest overall performance among the 14 methods, indicating that the spatial and contextual branch structures we designed are superior to the compared methods in feature extraction ability, and can better capture the semantic information of image targets, demonstrating good performance in solid-waste recognizing tasks.

4.4.2. Qualitative Analysis

Multi-Scene Comparison

Solid waste piles are widely distributed in various urban land use types, including roads, buildings, grasslands, forests, rivers, etc. Figure 6 shows the segmentation results in the building and road scenes. PSPNet and DAnet models have false detections (marked in red), mistakenly recognizing building and road edges as solid waste. FCHarDNet, ConvNeXt, UNet, SFNet, and SwinTransformer models have a large number of missed detections (marked in yellow), while the SWM proposed in this study has the fewest missed detections, consistent with the quantitative analysis results. Figure 7 shows the segmentation results in the river-forest scene. All models are able to accurately detect it, with a small amount of gray garbage on the bank in the upper left corner. Among them, PSPNet, DAnet, and SegFormer have missed detections on the boundaries. Figure 8 shows the segmentation results in the road-grassland scene. There is a single yellow pattern on the road, which is a pedestrian’s parasol. Deeplabv3plus and SwinTransformer mistakenly identify it as solid waste, while other models generally recognize the garbage on the grassland as background. However, the segmentation result of the SWM in this study is basically consistent with the label boundary. Overall, the diverse scales, textures, shapes, and distribution environments of solid waste make it difficult to accurately segment the targets from the background. The main advantage of our proposed model lies in its higher recall rate, which enables the detection of a larger number of missed instances. As seen from Figure 6, Figure 7 and Figure 8 (showing three demonstration areas), our proposed model tends to extract the solid-waste piles in their complete form, i.e., the segmentation boundaries are almost identical to the labels, whereas the compared models are charactered by ubiquitous missed detections. This indicates that the proposed dual-branch structure can provide more comprehensive and accurate feature representation, which helps improve performance and achieve the best results in the segmentation task of solid waste piles studied.

Extensive Examples

In order to further demonstrate the reliable performance of the SWM in large-area solid waste segmentation tasks, Figure 9 presents segmentation results in diverse scenes. The segmentation results show that three types of garbage can be segmented, indicating the generalization ability of the model proposed in this paper to accurately segment complex boundaries of solid waste in different distribution scenarios. The model is capable of achieving complete segmentation of solid waste occurrences at different scales, with very few missed and false detections, making it suitable for large-area solid waste monitoring.

4.5. Effectiveness of Comprehensive Monitoring and Management

The trained SWM was used to monitor the entire study area using UAV imagery, and a total of 3462 solid waste piles were identified (Figure 10). It can be observed that the garbage is relatively dispersed throughout the entire study area. This study randomly selected 100 solid waste piles, among which the detection results of 89 were accurate. Due to the large-area of the study site, in order to demonstrate the reliability of the SWM’s segmentation over the entire study area in detail, this paper selected the northwest area, which has a high concentration of solid waste, as a representative area to analyze the model’s segmentation results. These waste piles are widely distributed in our living environment (D1–D4). Long-term outdoor waste dumping and improper disposal will lead to the proliferation of rats and mosquitoes, causing the spread of certain diseases and posing a threat to human health. The spatial location map produced in this study can intuitively demonstrate the distribution of solid waste in different areas.

The environmental protection department has implemented governance measures on the solid waste piles, and a portion of the before-and-after comparison results are shown in Figure 11. From the results, it can be seen that the waste identified in this study has disappeared after the governance, indicating targeted management of the waste piles. This also demonstrates the high reliability of the monitoring results presented in this paper. Our work provides data support for ecological environment improvement efforts, effectively reducing the regulatory costs of manual inspection of solid waste in urban management through intelligent monitoring methods. It also promotes waste recycling and reduces environmental pollution.

5. Conclusions

Various semantic segmentation methods have been applied in different research fields by many scholars, but most of them focus on conventional segmentation tasks such as street scenes, farmlands, and buildings. An immediate application to large-area solid waste detection in ultra-high-resolution drone images results in a large number of missed and false detections. In this study, we combined unmanned aerial vehicle remote sensing and deep learning analysis technology to conduct research in 450 km² of ultra-high-resolution UAV images, built a dataset of 2600 images, and solved the problem of the lack of high-resolution semantic segmentation datasets for solid waste.

We then constructed a dual-branch semantic segmentation model that improved the ability to distinguish solid waste from the environment and the ability to segment complex edges. The global attention mechanism was introduced in the contextual feature branch to enhance the discrimination of foreground and background, and the deformable separable convolution was introduced in the spatial feature branch to adapt to complex edges and extract spatial detail information. Finally, we obtained an F1 score of 88.875%, an overall accuracy of 94.390%, a recall rate of 88.626%, and a precision rate of 89.125% on the dataset using our proposed method, which is superior to other 13 state-of-the-art semantic methods. Finally, the trained model was applied to segment the entire regional image, obtaining a distribution map of the entire region, which was successfully applied in large-area solid waste monitoring and governance. In urban solid waste monitoring, this method can save labor costs, facilitate quick treatment, and play a positive role in improving the overall living environment.

This study differs from solid waste recycling scenarios [21,24], as it primarily targets the monitoring of exposed solid waste over extensive regions. In comparison to approaches target detection [25,26], image classification [27,28,29], and weakly supervised segmentation [30,31], this study achieves pixel-level recognition outcomes for solid waste. However, there are certain limitations. With regard to the sample library, the data captured by the visible light band of the unmanned aerial vehicle (UAV) is limited to monitoring exposed solid waste on the ground surface, and is unable to detect waste concealed by vegetation or buried beneath the surface. To address this, there is a potential for introducing thermal infrared data in the future to identify such waste based on temperature changes resulting from the decomposition of garbage. Regarding the model, there is a need to enhance the performance of mixed waste detection, particularly in cases where the mixed waste is dispersed rather than consolidated. Furthermore, the dual-branch structure, while capable of capturing more complementary information, results in an increase in parameter size, thereby escalating the costs associated with training and inference. Therefore, further optimization to improve efficiency is necessary.