Comparing Object-Based and Pixel-Based Methods for Local Climate Zones Mapping with Multi-Source Data

Abstract

The local climate zones (LCZs) system, a standard framework characterizing urban form and environment, effectively promotes urban remote sensing research, especially urban heat island (UHI) research. However, whether mapping with objects is more advantageous than with pixels in LCZ mapping remains uncertain. This study aims to compare object-based and pixel-based LCZ mapping with multi-source data in detail. By comparing the object-based method with the pixel-based method in 50 and 100 m, respectively, we found that the object-based method performed better with overall accuracy (OA) higher at approximately 2% and 5%, respectively. In per-class analysis, the object-based method showed a clear advantage in the land cover types and competitive performance in built types while LCZ2, LCZ5, and LCZ6 performed better with the pixel-based method in 50 m. We further employed correlation-based feature selection (CFS) to evaluate feature importance in the object-based paradigm, finding that building height (BH), sky view factor (SVF), building surface fraction (BSF), permeable surface fraction (PSF), and land use exhibited high selection frequency while image bands were scarcely selected. In summary, we concluded that the object-based method is capable of LCZ mapping and performs better than the pixel-based method under the same training condition unless in under-segmentation cases.

1. Introduction

Urbanization, along with population growth, aging and international migration, is one of four “demographic mega-trends” described by the United Nations. In 2018, more than 55 percent of the population around the world lived in urban areas, and this proportion will increase to two thirds by 2050 [1]. The urbanization process has changed the urban surface structure and environment, resulting in artificial heat elevation, especially in megacities [2,3]. The phenomenon that urban heat is higher than surrounding areas, urban heat island (UHI), has caused climate issues such as heat waves and air pollution all over the world and has aroused people’s attention [4,5]. Traditional studies on UHI only analyze the difference between “rural” and “urban” areas, lacking a standard urban classification framework to evaluate temperature differences in urban regions [6,7,8].

To fill the gap in the urban classification framework, Stewart and Oke proposed the local climate zones (LCZs) scheme in 2012 [9], which divided the areas of interest into 17 standard LCZ types including 10 built types and 7 land cover types, describing diverse urban landscapes ranging from hundreds of meters to several kilometers. The detailed information on each LCZ type is shown in Figure 1. Considering building height, spatial distribution and covering material of land surface structure, the system provides a standard evaluation framework for UHI studies. With the popularity of LCZ research in related fields, the demands for high-quality LCZ maps are also gradually increasing. However, until now, a variety of mapping quality assessments indicate that there is still a lot of room for improvement in LCZ mapping, which can only be improved when considering all built classes together or using weights defined by the morphological and climatic similarity of certain classes [10].

Geographic information system (GIS) based mapping shows high potential in producing high-quality LCZ maps [12], but large-scale GIS-based mapping is limited by data and complexity of operations, making RS-based mapping the mainstream. Most RS-based mapping products use pixels as mapping units, among which the Level 0 product mapping process proposed by the WUDAPT (The World Urban Database and Access Portal Tools) project is the most popular [13]. The WUDAPT method uses free Landsat image and SAGA GIS software to resample remote sensing images to 100 m and realize LCZ mapping through random forest. Until now, LCZ maps for more than 100 cities around the world have been completed and shared. However, WUDAPT-based LCZ maps clearly show a disadvantage in accuracy. Overall accuracy (OA) of the 90 LCZs uploaded on the WUDAPT portal is 74.5%, and the average OA of the built LCZ types of the 90 LCZs is only 59.3%, leaving much room for improvement [14]. To obtain high-quality LCZ maps, many researchers have made efforts in mapping methods. Verdonck used derived spectral information to help classification [15], while Liu and Shi proposed new network architecture for LCZ classification [16]. However, pixel-based classification still has two problems: (1) it cannot provide good boundaries for LCZ types; (2) most LCZ products are made with low resolution, resulting in mixed pixels.

Object-based classification has made great success in the field of high-resolution image analysis. In recent years, some scholars have begun to explore the application of object-based LCZ mapping. Collins and Dronova generated an object-based LCZ map for the urban areas of Salt Lake City, United States of America [17]. They argued that the object-based LCZ classification paradigm could better describe the boundaries of LCZ types. In addition, Ma et al. proved the effectiveness of object-based LCZ mapping through land surface temperature analysis of three cities in China [18]. Although the exploration of OBIA in the field of LCZ mapping has been gradually carried out, their research did not show the advantages of object-based LCZ mapping. If the object-based LCZ mapping is more advantageous, there is still no definite conclusion in existing research. In addition, many studies use auxiliary data to help LCZ mapping at present, which can be divided into two categories: (1) Derivation of related features. This method uses spectral index, mean, maximum, and minimum to aid classification [14]. (2) Addition of datasets. This approach uses multiple datasets simultaneously, such as Landsat, Sentinel, Global Urban Footprint, and Open Street Map [19,20]. Although multi-source data were used in their studies, the mapping units are all pixels, limiting the effectiveness of multi-source data.

The comparison between the object-based method and pixel-based method has been carried out in various fields. Shi and Liu compared two methods for mapping quasi-circular vegetation patches and recommended the object-based SVM approach to map the QVPs [21]. Nachappa proved that the object-based geon approach creates meaningful regional units and performs better than the pixel-based approach in landslide susceptibility mapping [22]. For land use/land cover (LULC) mapping and change detection, similar research was conducted, and the object-based method showed greater potential compared with the pixel-based method [23,24,25]. However, little research has systematically compared object-based and pixel-based methods in the field of LCZ mapping, especially with multi-source data. Since most LCZ products are made through the pixel-based paradigm, it is important to figure out the advantages and disadvantages of an object-based paradigm and whether image objects can maximize the benefits of multi-source data.

This study aims to systematically compare the object-based and pixel-based data with multi-source data. Since few studies have been carried out on object-based LCZ mapping, we further propose an improved object-based LCZ mapping workflow. The objects of this research are to: (1) complete the object-based LCZ mapping process with multi-source data and object-based sampling strategy; (2) compare the LCZ maps generated from two different types of paradigms; (3) assess multi-source features in the object-based LCZ mapping; (4) discuss the future direction for improving the object-based LCZ mapping.

2. Study Area and Multi-Source Data for LCZ Mapping

2.1. Study Area

This study is conducted in Changzhou, a city in Jiangsu Province, China, with longitude 119°08′–120°12′ east and latitude 31°09′–32°04′ north, which is shown in Figure 2. Changzhou is located on the southern bank of the lower reaches of the Yangtze River, the Taihu Lake Basin water network plain, in the southern part of Jiangsu Province. The terrain of the territory is slightly higher in the southwest and lower in the northeast, with a height difference of about 2 m. The climate of Changzhou is a subtropical monsoon climate, characterized by warm winters and cool summers. Changzhou has distinct seasonal characteristics and scarcely suffers from large-area and long-term cloud cover or snowfall, becoming an appropriate city for LCZ mapping research. The area investigated in this study covers approximately 800 km², which comprises the main urban area and nearby rural area. As the number of LCZ classes varies from city to city because of the different urban form, 15 classes (nine built types and six land cover types) were taken into consideration in our study area.

2.2. Multi-Source Data Acquirement

Three types of data were used for mapping in this study: satellite images in raster format, building data and land use data in vector format. Sentinel-2 (S2) images of four seasons were used for data composition, which is introduced in Section 3.1. Six multispectral bands of S2 images were used in this study: B2, B3, B4 and B8 with 10 m Ground Sampling Distance (GSD) and B11 and B12 with 20 m GSD. The use of multi-season remote-sensing data has been shown to improve the accuracy of LCZ classification [19]. So seasonal S2 images are used and the related information is displayed in

Table 1. Remote sensing image data for data composition.

Satellite Image	Season	Date
Sentinel-2 L2A	Spring	2020-03-01 to 2020-05-31
	Summer	2019-06-01 to 2019-08-31
	Autumn	2020-09-01 to 2020-11-30
	Winter	2020-12-01 to 2021-02-28

. Building data and land use data are used in this study to provide urban morphological information, common in GIS-based mapping [12]. Land use data include eight categories: water, low vegetation, woodland, bare soil, road, industrial area, residential area, and other facilities. These two types of data were obtained from Baidu Map and the local department, respectively, representing the study area in 2019.

3. Methods

For LCZ mapping, four main steps were conducted in this study (Figure 3):

• Data preprocessing. To obtain high-quality data for LCZ mapping. Seasonal composite satellite images were obtained as basic mapping data, while building data and land use data were converted to raster form. All data are resampled into 10 m GSD.
• Feature derivation for LCZ mapping. For ricing spectral information and better depicting the urban form, diverse features were selected and derived, including spectral reflectance, spectral indices, zonal information obtained through filtering, urban morphological parameters (UMPs) that depict urban morphology.
• Feature extraction based on image objects and patches. After obtaining segmentation results by multi-resolution segmentation (MRS) and determining the resampling size, we extracted zonal mean, standard deviation and texture from objects and pixels based on a 10 m data block respectively.
• LCZ classification. LCZ samples based on objects or pixels were input to the random forest classifier for training and testing. Finally, each object or pixel was predicted by the random forest classifier for producing the LCZ map.

3.1. Data Preprocessing

Synthetic seasonal cloudless S2 images, rasterized building data and land use data were obtained to integrate RS and GIS information. For S2 images, after geometric correction and atmospheric correction from Google Earth Engine, a series of image preprocessing operations were carried out to make sure to obtain images without cloud. The QA60 band, which is a bitmask band with cloud mask information, was used to remove the pixels with cloud. Subsequently, image sets of each season were created by a date filter for the study areas and were then used to generate synthetic images based on the median of individual pixels. Finally, synthetic multi-season S2 images including B2, B3, B4, B8, B11 and B12 with 10 m were created for LCZ mapping with bilinear resampling method. GIS data were converted and resampled into 10 m. Building layer was assigned based on height while land use layer was assigned based on categories with values 0–7.

3.2. Derived Raster Features for LCZ Mapping

In this study, diverse features were extracted and stacked as a multi-channel data block with a resolution of 10 m. This data block includes spectral reflectance layers, spectral indices layers, zonal information layers in four seasons and UMPs including BH, SVF, BSF, PSF and land use layers. More information is shown in

Table 2. Overview of features in the data block.

Feature	Content	Seasonal	Number
Spectral reflectance	Blue, Green, Red, Nir	√	16
Spectral index	NDVI, MNDWI, NDBI, NBAI, BSI, BRBA	√	16
Zonal information	Convolutional layers	√	24
UMP	BH, SVF, BSF, PSF, Land Use		5

. The whole data block serves as the input for object-based and pixel-based methods to ensure that they are completely consistent for comparison. The input features in this data block are:

Spectral reflectance: blue (B2), green (B3), red (B4) and near-infrared (B8) were selected to provide spectral information. Considering that seasonal images were used in our study, 16 layers were finally obtained.

Spectral index: NDVI (Normalized Difference Vegetation Index), NDBI (Normalized Difference Built Index), NBAI (Normalized built-in Area Index), BRBA (Band Ratio for built-up Area), BSI (bare-soil Index) and MNDWI (Modified Normalized Difference Water Index) were used for rich spectral information. These spectral indices were demonstrated to be able to characterize vegetation, buildings, water and bare soil [26,27], aiding the classifier to mine underlying spectral differences and connections among diverse LCZ types. Notably, as the classification for built LCZ types is harder, three building-related spectral indices were selected for improving the performance of the classifier. Considering four seasons, a total of 24 spectral index layers were obtained.

Zonal information: Verdonck proved that using zonal information can effectively improve LCZ mapping quality on a fine scale [15]. In this study, we used a mean filter to obtain zonal information based on seasonal images. In detail, the filter is defined with size 11*11 and moving with step = 1. When the filter is at the edge of the image, the extent of the processing is outside the image; thus, zero-padding was applied. Finally, we obtained 16 layers containing zonal information.

Urban morphological parameter: Stewart and Oke defined LCZs with measurable and stable physical properties including SVF, Aspect Ratio (AR), BSF, Impervious Surface Fraction (ISF), PSF, Height of Roughness Elements and Terrain Roughness Class [9]. In this study, BH, SVF, BSF, and PSF were calculated, while the land use layer was obtained by assigning the value of every pixel according to the category, because they are available from our data and are widely used for LCZ mapping [12,28,29]. SVF is defined as the visible proportion of the sky hemisphere from a ground point [30]. Figure 4 displays the visible sky (S_sky) and the invisible sky (S_obstacle), which are determined by search radius (R) and building distribution. In this study, the SVF value for each pixel was obtained by SAGA GIS [31]. Building height data with 10 m GSD were input, and R was determined as 100 m. The relevant calculation formulas are displayed in

Table 3. Urban morphological parameters. In this study, the final UMPs for mapping were extracted from mapping units.

UMP	Definition	Formula
Building Height	$n$ is the number of buildings in a classification unit (a resampled pixel or an object); $B{S}_i$ is the area of one building; $B{H}_i$ is the height of one building; $S_{site}$ is the area of a classification unit.	$\mathrm{BH}=\frac{{{\displaystyle \sum }}_{i=1}^{n}B{S}_i\times B{H}_i}{S\_site}$
Sky View Factor	$n$ is the number of pixels in a classification unit; $SV{F}_i$ is the $SVF$ in pixel $i$; $S\_sk{y}_i$ and $S\_obstacl{e}_i$ are the area of visible sky and invisible sky for pixel $i$, respectively;	$SV{F}_i=\frac{S\_sk{y}_i}{S\_sk{y}_i+S\_obstacl{e}_i}$ $\mathrm{SVF}=\frac{{{\displaystyle \sum }}_{i=1}^{n}SV{F}_i}{n}$
Building Surface Fraction	$n$ is the number of buildings in a classification unit.	$\mathrm{BSF}=\frac{{{\displaystyle \sum }}_{i=1}^{n}B{S}_i}{S\_site}$
Permeable Surface Fraction	$\sum S\_per$ is the area of the permeable surface in a classification unit (average NDVI for four seasons > 0.2)	$\mathrm{PSF}=\frac{\sum S\_per}{S\_site}$

.

3.3. Classification Schemes for LCZ Mapping

3.3.1. Object-Based Classification

MRS in Trimble eCognition software was used in this study, which adopts a bottom-up method to gradually merge homogeneous pixels into one object and determines the threshold of an object by parameters such as scale, color/shape, smoothness/tightness [32,33]. In addition to traditional LULC classification tasks, MRS has been successfully adopted in the field of urban functional zones mapping [34,35], which demonstrated that MRS has the potential for classifying diverse landscapes. Thus, MRS was selected for image segmentation in this study.

Regarding parameter settings, segmentation weights of spectral layers (16 layers) were set as 1 while others were set as 0. Segmentation parameters of color/shape and smoothness/compactness were considered as 0.9/0.1 and 0.5/0.5. A total of nine scales ranging from 30 to 150 were selected to evaluate mapping results in different scale scenarios.

For LCZ mapping, it has become well known that values extracted from an adjacent region (mean, standard deviation, maximum, minimum, et.) help classification [20]. However, most researchers used a moving window to calculate statistical values as a new feature map. In the object-based method, we extracted features from image objects. Three types of features were obtained through Trimble eCongnition for our classifier: mean, standard deviation and texture. The total number of features is 87. The number of features for each type is shown in

Table 4. Summary of feature types for object/pixel-based classification.

Feature	Number
Mean	63
Standard deviation	16
Texture	8

.

The total number of mean features is 63. Sixty-one of them were calculated one by one from the layers we input. The remaining two features were MAX.DIFF and brightness. The total number of standard deviation features is 16, which were calculated from the composite S2 images. Gray Level Co-occurrence Matrix (GLGM) was used to extract texture features from the composite S2 images, which describes the distribution of co-occurring values of an image in a given area and provides a statistical view of the texture based on the image histogram. We selected eight features, including contrast, dissimilarity, homogeneity, angular second moment, entropy, mean, standard deviation and correlation with all angles.

3.3.2. Pixel-Based Classification

In pixel-based classification, two schemes with 50 and 100 m were designed for LCZ mapping. Although the most appropriate mapping scale for LCZ classification has not reached a consensus yet, Bechtel et al. argued that a “valid” LCZ may vary depending on the resolution, and mapping with 100–150 m is a good compromise while 10–30 m is too high [13]. Furthermore, the WUDAPT project, which strongly promotes the development of LCZ research, produced LCZ maps in a resolution of 100 m, leading most scholars to achieve LCZ classification in a resolution of 100 m [10,14,16,36]. Thus, 100 m was selected in this study as the first choice. In addition, since multi-source data can provide detailed urban information, making LCZ mapping possible at a fine scale. We selected 50 m as the second choice, which is almost the highest resolution in the LCZ mapping research [37]. Thus, after the optimal scale was selected for the object-based method, we compared the object-based method with the pixel-based method in 50 and 100 m.

In order to make the pixel-based method and the object-based method comparable, zonal mean, standard deviation and texture were extracted from image patches for LCZ classification instead of traditional resampled values. Similar to the object-based method, a total of 87 features were finally adopted: 63 of them were zonal mean calculated from the data block, 16 were standard deviation features, and 8 were texture features calculated from the composite S2 images.

3.4. Random Forest Classifier

Random forest has been widely adopted in the remote sensing field, showing great performance in classification and regression tasks [38,39,40]. By producing independent trees with randomly selected subsets through bootstrapping from training samples and input variables, the classifier obtains the final result through votes of majority trees. In addition, due to the popularity of the WUDAPT project, which achieves LCZ classification through random forest, most of the current LCZ maps were produced by random forest. We selected the same classifier to better exhibit the difference between object-based and pixel-based methods.

Since random forest is composed of independent decision trees, it is a machine learning algorithm that is sensitive to the different input features, which is convenient for us to analyze feature importance in the LCZ mapping field. Finally, the classifier we used in this study consists of 476 trees. In addition, 60% of the total sample data was selected for training the classifier, and all samples were used for testing, as we performed an area-based accuracy assessment for the object-based method.

3.5. Sampling Strategy

Three steps were conducted for sampling in our study. First, the sample polygons were digitalized on Google Earth software. Second, we stacked the samples with the segmentation/resampled layer and finally extracted reference objects/pixels. Considering the actual condition of Changzhou, we adopted 15 LCZ types including nine built types (excluding LCZ7) and six land cover types (excluding LCZC). Figure 5 displays the difference between object-based workflow and pixel-based workflow in sampling. Notably, when digitalizing on Google Earth, sampling criteria from Zhu et al. were conducted to identify diverse LCZ types [41]. In particular, for built LCZ types, after determining the target area uniform, a large area was distinguished as LCZ8 or LCZ10 while others were distinguished by height, compactness and material according to Figure 1.

Since the polygon samples obtained at the digitalization stage cannot completely coincide with the segmented image objects, it is hard to label objects correctly. Radoux set different overlay ratio thresholds for labeling different land cover categories, fully considering the differences among ground objects [42]. In addition, LCZ classification is gradually regarded as a scene classification task, and many scholars attempted to utilize surrounding information of samples to improve the robustness of the classifier [14,16,35]. For a similar purpose, diverse overlay ratios were set for each LCZ type. Taking 0.5 as a basic threshold (too high is meaningless while too low may introduce wrong samples), we adjusted the overlay ratio according to the complexity of an LCZ. Namely, simple scenarios require a higher threshold while complex ones need a lower threshold.

Table 5. Overlay ratio settings for LCZs.

LCZ Type	Overlay Ratio	Explanation
LCZ1	0.55	The composition of LCZ1 is complex, and the threshold should not be too high.
LCZ2	0.65	The distribution of LCZ2 is relatively concentrated, and a higher threshold can be set.
LCZ3	0.60	Same as LCZ2.
LCZ4	0.55	LCZ4 is easier to distinguish, but because it contains a certain degree of vegetation, the threshold should not be too high.
LCZ5	0.55	Same as LCZ4.
LCZ6	0.45	LCZ6 is similar to LCZ5, but constructions are usually sparser, making a low threshold necessary.
LCZ8	0.60	Most of them belong to industrial areas with concentrated distribution; thus, a higher threshold can be set.
LCZ9	0.45	The natural coverage is greater than that of artificial buildings; thus, a low threshold is set.
LCZ10	0.65	Same as LCZ8.
LCZA	0.75	LCZA is widely distributed and easy to identify, setting a high threshold.
LCZB	0.50	LCZB includes low vegetation and a few trees; thus, a low threshold is needed for labeling.
LCZD	0.60	Mostly cultivated land and grassland, covering a large area, including diverse vegetation, the threshold should not be too high or too low.
LCZE	0.65	LCZE is easy to identify but is small in area. A higher threshold is necessary.
LCZF	0.60	Same as LCZE.
LCZG	0.75	LCZG is widely distributed and easy to identify, setting a high threshold.

displays the settings we used. Finally, the stratified random sampling strategy was used to select training and test samples.

3.6. Accuracy Assessment

Overall accuracy ($\mathrm{OA}$), kappa coefficient ($\mathrm{Kappa}$), user’s accuracy, producer’s accuracy and F1-Score ($\mathrm{F}1$) were used for accuracy assessment. Several accuracy metrics proposed for LCZ mapping including ${\mathrm{OA}}_{BU}$, ${\mathrm{OA}}_N$ and $\mathrm{WA}$ were also taken into consideration [43]. The related formulas are shown in

Table 6. Description of accuracy metrics.$N_i^c$ is the area/number of class $i$ correctly classified, $w$ is weight matrix introduced by Betchel [31].

Metric	Formula
Overall Accuracy ($\mathrm{OA}$)	$\frac{1}{N}{\displaystyle \sum _{i=1}^n}{N}_i^c$
Built-up OA (${\mathrm{OA}}_{BU}$)	$\frac{1}{N_{Built}}{\displaystyle \sum _{i=1}^n}{N}_{Built,i}^c$
Natural OA (${\mathrm{OA}}_N$)	$\frac{1}{N_{Natural}}{\displaystyle \sum _{i=1}^{n_{Builit}}}{N}_{Natural,i}^c$
Weighted Accuracy ($\mathrm{WA}$)	$\frac{1}{N}{\displaystyle \sum _{i=1}^{n_{Nature}}}w\times N_i^c$
F1-Score (F1)	$2\left(\frac{Precision \times Recall}{Precision + Recall}\right)$

. For the object-based method, an area-based accuracy assessment was performed counting for the correct proportions of classified areas in segmented objects [44]. For the pixel-based method, all metrics were calculated by counting the correct number of pixels. In the accuracy metrics we designed, user’s accuracy and producer’s accuracy, which reflect commission error and omission error, were performed with a normalized confusion matrix. F1-score can be regarded as a harmonic average of model accuracy for better assessing per-class performance. ${\mathrm{OA}}_{BU}$ is the $\mathrm{OA}$ on only the built types, and ${\mathrm{OA}}_N$ is the $\mathrm{OA}$ of the built versus natural LCZ classes only. $\mathrm{WA}$ is a metric that accounts for similarity and dissimilarity between classes.

4. Results

4.1. Classifications of Object-Based Method in Multi-Scale Scenarios

4.1.1. Visual Interpretation Analysis of LCZ maps at Various Scales

To the best of our knowledge, no detailed assessment for object-based LCZ mapping has been made in state-of-the-art research. To assess the optimal scale of the object-based method, first, we conducted a visual interpretation analysis to evaluate LCZ maps in multi-scale scenarios.

Figure 6 shows LCZ mapping results as scale range from 30 to 150. In general, all maps show no obvious misclassifications because segmentation merges discrete pixels before classification. Analyzing each map individually, the maps at scales 30 and 45 were finer, and major roads were successfully classified into LCZEs. At scales 60–105, the fragmentation of the classification was reduced but still maintained a high agreement with the LCZ maps at scales 30 and 45. When the scale was greater than 105, the classification of urban areas changed greatly. From observation, the maps with scales 30–105 performed better, as they were relatively stable. At scales 120–150, the area of LCZ1-LCZ6 types increased, and the area of LCZ8 decreased. Although Bechtel et al. pointed out that LCZ types may vary with scale [13], our maps should not show much difference because they were trained with the same samples. In order to further analyze the results of each scale, we carried out precision analysis.

4.1.2. Accuracy Analysis in Multi-Scale Scenarios

Table 7. Accuracy statistics from scale 30 to 150 with the average values from 50 runs.

Scale	$OA \mathbf{\%}$	$Kappa \mathbf{\%}$	$O{\mathbf{A}}_{\mathit{B}\mathit{U}} \mathbf{\%}$	$O{\mathbf{A}}_{\mathit{N}} \mathbf{\%}$	$WA \mathbf{\%}$
30	96.59	96.05	94.81	99.57	99.22
45	96.38	95.78	94.34	99.64	99.22
60	96.05	95.35	93.83	99.61	99.19
75	95.67	94.84	92.80	99.71	99.18
90	95.72	94.88	92.43	99.76	99.24
105	95.29	94.38	91.34	99.57	99.09
120	93.84	92.66	89.52	99.09	98.68
135	93.45	92.18	88.46	99.10	98.63
150	91.64	90.15	86.10	98.93	98.13

presents accuracy metrics for the object-based method with a scale ranging from 30 to 150. All results in Table 7 represent the average value of 50 runs. In general, all object-based classifications achieved over 85% accuracy for all accuracy metrics, especially $\mathrm{WA}$ and ${\mathrm{OA}}_N$, which surpassed 98% at all scales. This could be explained that in the object-based method, the classifier benefits from segmentation and avoids huge mistakes while the resampling process leads to mixed pixels (e.g., The classifier cannot distinguish objects between nature type and built type). As the segmentation scale became larger, all metrics decreased except for $\mathrm{WA}$ and ${\mathrm{OA}}_N$, which both peaked at scale 90, indicating the optimal scale required for natural and built types differs. The phenomenon of accuracy decrease was mainly in $\mathrm{OA}$, $\mathrm{Kappa}$ and ${\mathrm{OA}}_{BU}$. As the scale surpassed 105, accuracy loss was accelerated in $\mathrm{OA}$, $\mathrm{Kappa}$ and ${\mathrm{OA}}_{BU}$, which indicates that large scale is not suitable for built-type classification.

In particular, Figure 7 shows the variation of $\mathrm{OA}$ and ${\mathrm{OA}}_{BU}$ to scale. $\mathrm{OA}$ was relatively stable at scale 30–90, and the mean curve formed a platform. When the scale is greater than 105, $\mathrm{OA}$ declines significantly, mainly at scales 105–120 and 135–150, where the slope of the curve is much higher, indicating that a large scale is not suitable for LCZ mapping. Considering that the object size increases as the scale increases, the decrease in accuracy may be caused by the fact that important classification features, such as BH and SVF, are blurred by the excessively large object. Although the variation curve of ${\mathrm{OA}}_{BU}$ is similar to $\mathrm{OA}$, the decline is more obvious than $\mathrm{OA}$, indicating that the accuracy decrease is mainly in the built types. Both figures show a trend of variance increasing with the increasing scale, which can be attributed to two reasons: one is the feature uncertainty caused by large objects, and the other is the insufficient generalization ability due to the reduction of samples. In general, the mapping results are relatively stable at the scale of 30–90, indicating that the object-based method can be capable of LCZ mapping despite under-segmentation cases. In order to minimize the computational demands of mapping, we chose scale 90 as the optimal scale.

4.2. Comparison of Object-Based and Pixel-Based Methods

4.2.1. Overall Accuracy and Statistical Analysis Comparison

According to the results above, we chose scale 90 as our optimal parameter for the object-based method and compared the object-based method with the pixel-based method of 50 and 100 m.

Table 8. Training and test samples of each scheme. The values in the table are the number of objects/pixels.

	Units to Be Classified	Training Samples	Test Samples
Object-based	28,071	560	949
Pixel-based 50 m	316,522	11,529	19,224
Pixel-based 100 m	79,476	3754	6267

displays related information of the three classification schemes, showing huge computational costs in the pixel-based methods.

Figure 8 displays the LCZ maps in three schemes. We can see that the object-based method provides better boundaries for LCZs, improving the visualization of the LCZ map.

Table 9. Accuracy assessment results for three schemes with the average values from 50 runs for each scheme.

	$OA \left({\mathsf{\sigma }}^2\right) \%$	$Kappa \left({\mathsf{\sigma }}^2\right) \%$	$\mathit{O}{A}_{\mathit{B}\mathit{U}} \left({\mathsf{\sigma }}^2\right) \%$	$O{A}_{\mathit{N}} \left({\mathsf{\sigma }}^2\right) \%$	$WA \left({\mathsf{\sigma }}^2\right) \%$
Object-based	95.72 (5.04 × 10−5)	94.88 (7.2 × 10−5)	92.43 (1.7 × 10−4)	99.76 (2.31 × 10−5)	99.24 (1.86 × 10−6)
Pixel-based 50 m	93.71 (2.78 × 10−6)	92.95 (3.5 × 10−6)	91.60 (5.17 × 10−6)	98.60 (6.57 × 10−7)	98.23 (3.34 × 10−7)
Pixel-based 100 m	90.28 (7.57 × 10−6)	89.25 (9.28 × 10−6)	88.11 (1.68 × 10−5)	97.47 (3.49 × 10−6)	97.01 (1.39 × 10−6)

displays the accuracy metrics of three classification schemes. All results were obtained from 50 classification repeats. In general, all metrics revealed that the object-based method performed better than the two pixel-based methods. In three classification schemes, all $\mathrm{WA}$ and ${\mathrm{OA}}_N$ surpassed 97%, much higher than other accuracy metrics. With multi-source data, the pixel-based method performed better in 50 m instead of 100 m, which indicates that features in this study are able to characterize LCZs well at a fine scale. $\mathrm{OA}$ and kappa of the object-based method were both about 2% higher than those in the 50 m pixel-based method, while ${\mathrm{OA}}_{BU}$ was only about 1% higher. Variances are displayed in Table 9. Two pixel-based methods exhibit smaller accuracy variances compared with the object-based method. We think that it can be attributed to more samples in the pixel-based methods.

To prove that there is a significant difference in accuracies between the object-based method and the pixel-based method, Student’s t-test was conducted to compare the means of OA of three schemes. In our study, independent samples t-test was conducted. Furthermore, to ensure that our test was statistically effective, Shapiro–Wilk test was conducted to test whether our data obey a normal distribution and all three sets of data passed the test with p value > 0.05. Levene’s test is conducted to test the equality of variances. All tests are finished with α = 0.05. Every set of data has 50 samples.

Table 10. Results of statistical tests. The test results are the comparison results with the object-based method.

	Levene’s Test		Independent Samples t-Test
	p Value	Significant Difference	p Value	Significant Difference
OBIA vs. 100 m pixel	<0.001	Yes	<7 × 10−50	Yes
OBIA vs. 50 m pixel	<0.001	Yes	<4 × 10−26	Yes

displays the results of statistical tests. Levene’s tests both rejected null hypotheses, which means the data from the object-based method have different variances with two pairs of data from the pixel-based method under the condition α = 0.05. Under the premise of uneven variances, t-tests were conducted, and the results showed significant differences. Through statistical tests, we proved that the object-based method performed better than two pixel-based methods.

4.2.2. Per-Class Accuracy Comparison between Schemes

User’s accuracy and producer’s accuracy were used to assess errors of commission and omission. Each confusion matrix in Figure 9 was an accumulation of normalized confusion matrices (50-runs classification) where the original elements have been divided by the row total, ensuring that we can evaluate the object-based and the pixel-based method equally.

In two pixel-based methods, there was a significant similarity between the two matrices, but the results from 50 m performed better. By adding various features, the classifier can reach high accuracy in the high-resolution scenario. By comparing the object-based method with the pixel-based method in 50 m, the results indicated some differences between the two methods. The distribution of errors of commission and omission in the object-based method was more concentrated, while more non-zero values were scattered in the matrix in the 50 m scenario. In addition, the object-based method performed better in typical LCZ types such as LCZ4, LCZF, which always appear with distinct visual features. However, in some types such as LCZ2, LCZ5 and LCZ6, the object-based method performed worse. This failure indicates that complex scenes are still challenging for the object-based method. However, object-based results were trained with fewer samples, which limited the potential of the method.

It is not objective to evaluate the performance of a certain LCZ type with user’s accuracy or producer’s accuracy only. For instance, in Figure 8, all methods achieved a high producer’s accuracy with LCZD but with a lower user’s accuracy. Thus, for F1-Score, a comprehensive metric was selected to describe the performance of each LCZ type.

Figure 10 displays the performance of each LCZ type in the three schemes. Comparing two pixel-based schemes, the accuracy of each LCZ type increases, which indicates that adding multi-source features can help the classifier learn better in high-resolution cases. The results in the object-based method performed quite differently. Compared with the pixel-based method in 50 m, the variances of most LCZ types of the object-based method are larger. This is because the object-based method was trained with fewer samples. However, the object-based method performed much better on nature types and competitive results in built types despite LCZ2, LCZ5 and LCZ6. Considering that there is a much smaller number of samples in the object-based method, the object-based method showed great potential in LCZ mapping. In addition, according to our analysis above, the accuracy of built types can be improved at a fine scale.

4.3. Feature Importance Analysis Based on the Object-Based LCZ Classification

Random forest has been proven to be a classifier less affected by sample imbalance [45] and has more obvious advantages in improving mapping results with input features compared with deep learning methods [46]. Therefore, feature analysis based on random forest is meaningful. CFS is a relatively stable method in feature selection and it can maintain the effect of a classifier to a large extent [44]. In this study, CFS was used for feature selection, and the frequency of feature selection was used as a metric to assess feature importance.

Figure 11 shows the results of feature selection from scales 30 to 150 with 50 classification repeats at each scale. It can be seen from Figure 10 that the features selected by the object-based method mainly focus on four aspects: first, the urban morphological parameters related to LCZ definition: SVF, BSF, PSF, land Use, BH; second, the spectral indices that can represent characteristics of buildings, vegetation or water: NDBI, NBAI, MNDWI, BRBA; third, the spectral reflectance (including variation) standard deviation layers and convolution layers; finally, texture features.

Urban morphological parameters have high selectivity at each scale, which proves the importance of urban morphological parameters for LCZ classification. The spectral indices selected by CFS were mainly the bands containing short-wave infrared information, such as NDBI, NBAI and BRBA, indicating that short-wave infrared information is also important for LCZ mapping. NDVI did not stand out from many features, which is inconsistent with the conclusion of Qiu [19]. The reason is that PSF is similar to NDVI while PSF can better depict regional permeable surface information. From the selection of spectral features, we can see the uniqueness of LCZ classification. Feature selection focused on convolutional features (including zonal information) and standard deviation features, indicating that LCZ classification pays more attention to the spectral information of a local area rather than to a single pixel. As the scale increases, the selection of convolutional features gradually disperses, indicating that with the increase in mixed objects, the spectral features of objects become more chaotic and meaningless. Meanwhile, the selection of texture features gradually highlights as the scale increases, indicating that as the object grows, the texture features inside the object become more obvious and no longer appear redundant. In addition, features of each season were selected, indicating that time series features help classification.

5. Discussion

Currently, LCZ mapping is mostly divided into two streams: RS based and GIS based. LCZ mapping based on the integration of the two methods is considered to have great potential, and a few studies have begun to focus on the integration of the two [35]. The methods used in this study can be regarded as an integration of the two mapping methods, which further proves the great potential of the integrated framework.

From the perspective of the mapping accuracy, mapping results of small-scale/high-resolution were still higher than all other cases, which is inconsistent with some studies showing that mapping with moderate scale/resolution is better [13,16]. This is mainly because all features adopted in this study have regional information to help mapping reflect the mean or standard deviation value of image objects. Verdonck obtained a similar conclusion to ours and obtained the optimal mapping result by utilizing neighborhood information features [15]. However, an LCZ represents a region spanning at least hundreds of meters, which means that mapping at a too small scale will cause the result to deviate from the application purpose. Thus, it is not encouraged to map at a too small scale. In addition, the amount of computation brought by the small scale is huge, which is more obvious in the pixel-based method.

According to accuracy results and statistical analysis, object-based methods show advantages and are stable at multiple scales, which is because the boundaries offered the by object-based method reduce confusion among LCZs. In the pixel-based method, it is difficult for regular grids to correctly divide LCZ boundaries, making classification difficult. Object-based methods are not beneficial to all LCZ types. For example, segmentation makes labeling and classification of LCZ5 types more difficult. Segmentation makes various features separated, which is an urgent problem to be solved when applying the object-based method to LCZ mapping. Lehner and Blaschke proposed the use of object-based methods for urban structure type mapping [47]. If it is applied to LCZ mapping, the potential of object-based LCZ mapping may be further explored.

In feature analysis, the maximum difference, convolution features, NDBI, BRBA, BH, BSF, PSF, and land use were frequently selected, indicating that the above features are extremely important in LCZ mapping. Among the features of remote sensing data, convolution features that contain neighborhood information and the two spectral indices that contain short-wave infrared information performed well. These features have also been adopted in previous studies and are proven to be effective [20]. Mapping based on GIS data has always been considered as a method of high-precision LCZ mapping [12]; we also demonstrate the validity of GIS data.

The performance of the machine learning classifier is closely related to the number of training samples [44]. Considering that the number of training samples will decrease as the scale parameter increases, the mapping accuracies of multiple scales were similar to the object-based methods. In the case of similar mapping accuracy, the choice of scale should be determined by the application conditions. Misclassification is acceptable in some cases because similar LCZ types represent similar temperature conditions. However, if the 17 LCZ types cannot meet application requirements and if the mapping scale is small, an LCZ subclass can be considered to be added. Kotharkar solved the confusing LCZ types problem by employing 21 LCZ types [48]. Perera also pointed out that using LCZ subclasses is necessary for further analysis of urban development status [49].

6. Conclusions

This study describes an attempt to compare the object-based method and the pixel-based method for LCZ mapping with multi-source data, aiming to determine a more advantageous mapping paradigm. In addition, four types of features were derived for ensuring mapping quality: (1) composite seasonal satellite images; (2) diverse spectral indices; (3) zonal information; (4) urban morphological information.

In multiple scale cases, the object-based method performed stable unless in under-segmentation cases. When the scale was less than 90, the $\mathrm{OA}$ and ${\mathrm{OA}}_{BU}$ of all maps exceeded 95.5% and 92%, respectively, and no significant difference behaved in visual performance. When the scale was larger than 105, the object-based method showed an obvious performance decline, especially in built types, which indicated that multi-source data cannot provide sufficient urban information with under-segmentation.

Comparing the object-based method with a pixel-based method in 50 and 100 m, the object-based method showed the best performance and statistical difference from pixel-based methods (p < 0.05). $\mathrm{OA}$, ${\mathrm{OA}}_{BU}$, ${\mathrm{OA}}_N$ and $\mathrm{WA}$ of the object-based method increased by about 2%, 1%, 1% and 1%, respectively, compared with the 50 m case (the better one) while 10 times the amount of data was processed in the pixel-based method. In per-class analysis, the object-based method showed a significant advantage in the natural types and competitive performance in built types, while partial LCZ types such as LCZ2, LCZ5, and LCZ6 performed worse than the pixel-based method in 50 m. However, we found that built type accuracy can be improved with finer segmentation.

In feature analysis, CFS was selected to evaluate the importance of features, and frequency characterized feature importance. For urban morphological information, BH, SVF, BSF, PSF and land use exhibited an extremely high frequency of selection at all scales. Among other features, convolutional layers, BRBA, NBDI, NBAI, MNDWI of certain seasons performed well, while original image bands were hardly selected, indicating that derived spectral features can better characterize differences among LCZs.

In our study, the object-based method can achieve equivalent mapping results with less computation cost compared with the pixel-based method, making it more suitable for large-scale mapping. In the future, the object-based method will become more advantageous when the segmentation results of urban regions can be improved.