Unsupervised Image Segmentation Parameters Evaluation for Urban Land Use/Land Cover Applications

Abstract

Image segmentation plays an important role in object-based classification. An optimal image segmentation should result in objects being internally homogeneous and, at the same time, distinct from one another. Strategies that assess the quality of image segmentation through intra- and inter-segment homogeneity metrics cannot always predict possible under- and over-segmentations of the image. Although the segmentation scale parameter determines the size of the image segments, it cannot synchronously guarantee that the produced image segments are internally homogeneous and spatially distinct from their neighbors. The majority of image segmentation assessment methods largely rely on a spatial autocorrelation measure that makes the global objective function fluctuate irregularly, resulting in the image variance increasing drastically toward the end of the segmentation. This paper relied on a series of image segmentations to test a more stable image variance measure based on the standard deviation model as well as a more robust hybrid spatial autocorrelation measure based on the current Moran’s index and the spatial autocorrelation coefficient models. The results show that there is a positive and inversely proportional correlation between the inter-segment heterogeneity and the intra-segment homogeneity since the global heterogeneity measure increases with a decrease in the image variance measure. It was also found that medium-scale parameters produced better quality image segments when used with small color weights, while large-scale parameters produced good quality segments when used with large color factor weights. Moreover, with optimal segmentation parameters, the image autocorrelation measure stabilizes and follows a near horizontal fluctuation while the image variance drops to values very close to zero, preventing the heterogeneity function from fluctuating irregularly towards the end of the image segmentation process.

1. Introduction

In the past two decades, object-based image analysis has gained momentum in digital image mapping for geographical information systems and remote sensing applications [1]. Object-based image analysis offers the advantage of extracting objects of interest as distinct land use and land cover features in contrast to pixel-based image analysis, resulting in more accurate thematic mapping. Today, high-resolution aerial photographs and satellite imagery offer some accuracy advantages in urban mapping in the sense that it has now become possible to extract individual buildings as objects of interest using object-based image classification. However, due to their high variability in building sizes, shapes, and roof colors, urban areas remain some of the most difficult environments to map using pixel-based image classification algorithms [2]. In object-based image analysis, attributes such as shape, size, and color can play a vital role in improving the accuracy of image classification outcomes. In contrast to pixel-based image classification, object-based image classification offers the advantage of mapping groups of internally homogeneous pixels into distinct classes [3]. A key element in object-based image analysis is image segmentation, which partitions the digital image into distinct objects that are made of groups of homogeneous pixels. However, achieving image objects that are internally homogeneous remains a challenge in image segmentation due to the fact that the quality of produced image segments largely depends on a fair balance of some user-defined parameter thresholds that drive the image segmentation process [4,5]. These user-defined thresholds are generally associated with shape compactness, shape smoothness, color factor, and scale parameters [6]. Finding a proper balance between these segmentation parameters has been the subject of great research interest in object-based image analysis for the past decades. Not achieving a good balance between these segmentation parameters can result in small segments being submerged by larger ones and larger objects being over-segmented.

Due to the difficulty of finding optimal balances between segmentation parameter thresholds, the supervised trial-and-error strategy has been widely used. Ref. [7], for instance, performed the segmentation of an aerial image of an agricultural field by assigning weights of 0.9 and 0.1 to the color factor and shape compactness parameter, respectively. Meanwhile, [8] cautioned that the subjective selection of large segmentation-scale parameters with the expectation of achieving good quality segments can lead to serious segmentation errors, and the authors suggested an equal weight of 0.5 be assigned to the color factor and shape compactness parameter, respectively, in order to achieve acceptable image segmentation results.

In the last decade, the scale parameter and shape compactness have received high attention in research on the optimal selection of segmentation parameters [9,10]. The multi-resolution image segmentation algorithm in eCognition has been widely used for urban mapping applications. A successful image segmentation process is expected to produce image segments of a shape similar to their real-world boundaries [11]. Unsupervised attempts to identify optimal segmentation parameter thresholds have been proposed in the literature [12,13,14,15,16]. Metrics such as the weighted variance and Moran’s index have been widely used to study intra-segment homogeneity and inter-segment heterogeneity, respectively. Early studies to determine optimal balances between segmentation parameters relied on a global objective function that combines a local image variance measure and the spatial autocorrelation Moran’s index [17]. However, the main limitations of these early approaches include the fact that they only perform well for large homogeneous areas and were designed to search for optimal segmentation parameters at a single segmentation level. These approaches were later improved by [18,19] to include more than one segmentation level. The improved approaches identified optimal segmentation parameters as those defining peak points of the curve of a global objective function that is a function of a spatial autocorrelation model and an image variance measure. Ref. [20] modified the global objective function by restricting it to generate strictly three peak points of the curve, with each peak point corresponding to one segmentation level. This restriction may not be suitable for urban scenes where more than three types of objects may be dominant. This restriction was also reported not to be suitable for high-resolution imagery when the purpose of image classification is to extract features of small sizes, such as individual building units, swimming pools, roads, parking lots, or urban trees [9].

It has been suggested in [21] that methods of selection of optimal segmentation parameters based on the weighted variance and Moran’s index in their most commonly used formulations are very limited when it comes to objective searches for optimal segmentation parameters. Ref. [22] also reported that methods that use Moran’s index in its commonly used formulation are not suitable for heterogeneous areas such as urban environments, and the authors suggested the use of local variance measures instead. This is because Moran’s index in its present formulation makes the global objective function fluctuate irregularly, and this results in the image variance increasing drastically towards the end of the segmentation process, forcing larger objects to merge even though they belong to distinct classes [23]. Ref. [24] also cautioned that when modeling a good local variance curve in order to identify optimal segmentation parameters, it is vital to ensure that the models identify optimal values of parameters such as shape compactness or color factor weights that increase or are equal between successive segmentation levels. Ref. [25] argued that standard deviation measures should replace the current image variance measure since the latter fails to capture all the irregular intra-segment variances throughout the image.

Most segmentation parameter evaluation approaches have mainly focused on the scale parameter and the shape compactness and have not given great attention to the influence of color factor weights on the performance of scale parameters in object-based image analysis [26,27]. This paper proposes one nonlinear global heterogeneity objective function based on a non-weighted standard deviation image variance and a hybrid weighted spatial autocorrelation measure based partly on the current formulation of Moran’s index and a spatial autocorrelation coefficient in order to evaluate optimal segmentation parameters that can minimize under and over-segmentation of urban environments. The rationale behind the use of a non-weighted standard deviation image variance is that the current formulation of the image variance measure is dependent on segment area measures. This inclusion of individual segment area measures can introduce numerical instabilities into the global objective function or the global heterogeneity function in occurrences of under- or over-segmentations since some of the segment area measures do not accurately match their real-world measures, thus undermining the accuracy of the computed image variance. The rationale behind the proposal of the new spatial autocorrelation measure is that Moran’s index in its current formulation is expressed as a ratio in which the presence in the numerator of the sum of the spatial distance weights scaled by the total number of image segments leads to an increase in the measure while the merging process carries on. This will result in objects with low inter-segment heterogeneity not being merged. In addition, the four-nearest neighbor distance matrix used by the current formulation does not always hold true for all the segments within the image since it is possible for a large number of image segments to be surrounded by only one distinct neighbor, and this can result in a large number of zeros in the distance matrix, which could undermine the accuracy of Moran’s index.

2. Related Work

Several land use and land cover mapping studies have used image segmentation for object-based image classification. Many of these studies based the choice of segmentation parameters on trial-and-error procedures [28,29]. One inconvenience of such an approach is that it is tedious and does not always guarantee the best results since the final decision relies on visual interpretations of segmentation results, which can be very subjective when dealing with very complex areas such as urban areas. Ref. [17] proposed a technique to evaluate segmentation parameters in a forest area. The approach relied on the assumption that successful segmentations should produce objects that are internally homogeneous and, at the same time, distinct from their neighbors. Object internal homogeneity is measured through a weighted variance expressed as follows:

$$W_{\mathrm{var}}=\frac{{\displaystyle \sum _{i=1}^n{a}_i{v}_i}}{{\displaystyle \sum _{i=1}^n{a}_i}}$$

(1)

with $a_i$ as the area of a segment of interest $i$ and $v_i$ its variance. The measure returns low values for internally homogeneous segments and produces high values for heterogeneous segments [30]. The inter-segment heterogeneity is, on the other hand, measured through Moran’s index, which is a spatial autocorrelation measure formulated as follows:

$$MI=\frac{n{\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^n{w}_{ij}\left(y_i-\overline{y}\right)\left(y_j-\overline{y}\right)}}{{\displaystyle \sum _{i=1}^n{\left(y_i-\overline{y}\right)}^2\left({\displaystyle \sum _{i\ne j}}{\displaystyle \sum w_{ij}}\right)}}$$

(2)

where $n$ is the total number of objects, $y_i$ is the spectral brightness of a segment of interest $i$, and $\overline{y_i}$ is the mean spectral brightness on the image at a given segmentation level. The parameter $w_{ij}$ is a measure of the spatial proximity between two segments $i$ and $j$. The assignment of a value of one to the parameter $w_{ij}$ indicates that segments $i$ and $j$ are adjacent regions that share at least one boundary, and a value of zero indicates they do not. Moran index returns low values to describe segmentations with high inter-segment heterogeneity, and high values of the index describe segmentations that produce segments with low heterogeneity. Global objective functions are often used to characterize the overall quality of a segmentation outcome, and these functions usually combine the image variance measure in (1) and the spatial autocorrelation measure in (2). The most commonly used formulation of the global score function or global objective function is given as follows:

$$G_s=wVa{r}_{norm}+M{I}_{norm}$$

(3)

with $wVa{r}_{norm}$ and $M{I}_{norm}$ the normalized weighted variance and the normalized Moran’s Index, respectively. The normalizations of the measures in (1) and (2) are generally determined through the following equation:

$${\gamma }_{Normalized}=\frac{\gamma -{\gamma }_{Min}}{{\gamma }_{Max}-{\gamma }_{Min}}$$

(4)

where $\gamma$ is either the image-weighted variance measure or the spatial autocorrelation measure, while ${\gamma }_{\min }$ and ${\gamma }_{\max }$ represent their respective minimum and maximum values. The normalization equation rescales the values of these measures within the range of 0 to 1. Ref. [19] argued that the global objective function proposed in (3) by [17] does not guarantee optimal segmentation results, and the authors came to this assumption after visually observing segmentation results performed with optimal scale parameters determined by the global score function. To address this limitation, a heterogeneity function was proposed and combined the recalculated weighted variance and Moran’s index for under and over-segmented objects. The proposed index also requires normalized image variance and spatial autocorrelation measures and was formulated as follows:

$$H=\frac{wVa{r}_{norm}-M{I}_{norm}}{wVa{r}_{norm}+M{I}_{norm}}$$

(5)

where $wVa{r}_{norm}$ and $M{I}_{norm}$ are the normalized image variance and Moran’s index measures, respectively. The measure enables the identification of optimal segmentation scales that can re-segment the under- and over-segmented objects, which are then merged back to the previous segmentation performed under the condition in Equation (3). Furthermore, ref. [31] tested a hybrid approach combining the assumptions in [17,19]. However, their technique did not rely on the global objective function measure in (3) to identify the optimal scale parameters but rather used the different values of the heterogeneity function in (5) to identify scale parameters that produce high inter-segment heterogeneity and high intra-segment homogeneity measures, as illustrated in Figure 1 below.

The heterogeneity function, as illustrated in Figure 1, shows some high values of the measure, which are potential optimal scale parameters for the segmentation. Ref. [32] also argued that the global objective function in (3) is not sensitive to under- and over-segmentation and suggested an alternative measure, which has an additional weight compared to the traditional measure and was formulated as follows:

$$Gs=\left(1+a^2\right)\left(\frac{M{I}_{norm}\times wVa{r}_{norm}}{a^2\times M{I}_{norm}+wVa{r}_{norm}}\right)$$

(6)

where $a$ is a weight that controls the relative weight of $wVa{r}_{norm}$ and $M{I}_{norm}$. The model returns lower values for a segmentation with more homogeneous objects and high values for a segmentation with more heterogeneous objects. Ref. [21] pointed out that the functions describing the weighted image variance and Moran index in (1) and (2) still impose trustworthy spatial and spectral constraints on the segmentation outcome, in order to identify optimal segmentation parameters; however, the authors were more concerned about the normalization equation proposed in (4). According to their investigation, the Equation in (4) can produce inconsistencies in the sense that any error in the choice of minimum or maximum values of the weighted variance and Moran’s index could compromise the optimal selection of segmentation scale parameters due to errors that might originate from inaccurate segments area measures in occurrences of over- and under-segmentation scenarios. The authors proposed that both the weighted variance and Moran’s index should be rescaled separately with different measures. This means that the variance and Moran’s index should be normalized as follows:

$$V_{norm}=\frac{v}{\overline{v}}\mathrm{and}M{I}_{norm}=\frac{MI+1}{2}$$

(7)

with $v$ the variance of the object and $\overline{v}$ the mean variance of the image at a given segmentation level. It can be noticed that the proposed normalization formulae do not involve a segment area measure. Substituting the expressions in (7) into (3) produces the following global score or objective function:

$$Gs=\frac{2v+\overline{v}\left(MI+1\right)}{2\overline{v}}$$

(8)

The authors argued that the new global score function is more independent from any under- and over-segmentation errors that may occur. Indeed, the authors’ concerns were reasonable since, currently, it is not possible to obtain a segmentation with 100% correctly segmented objects; there are always some segments subjected to under- or over-segmentation even though the main aim of ongoing studies in the selection of optimal segmentation parameters is to minimize the number of segments subjected to under- and over-segmentation errors.

3. Materials and Methods

3.1. Satellite Imagery and Study Area

GeoEye imagery has been widely used for urban mapping applications [33,34]. The multispectral imagery possesses four spectral bands, including the three bands of the visible light region of the electromagnetic spectrum, comprising the blue band with a spectral resolution expanding from 450 nm to 510 nm, the green band, which expands from 510 nm to 580 nm, and the red band, which expands from 655 nm to 690 nm. The fourth band covers the near-infrared region of the electromagnetic spectrum, ranging from 780 nm to 920 nm. The spatial resolution of 1.8 meters classifies the multispectral imagery as high resolution [35,36]. Atmospheric, radiometric, and geometric corrections were already performed on the imagery. The choice of the City Bowl area of Cape Town metropolitan area as our study area was driven by the diversity and complexity of this urban environment that contains grassland fields, urban trees, open space, green recreational parks, sport fields, roads, residential and non-residential buildings of various sizes, parking lots, and water bodies. Five urban scenes within the study area were selected to extract spatial and spectral attributes of urban features. These urban scenes include the residential areas of Zoonnebloem, Vredelhoek, Oranjezicht, the Company’s Garden, Woodstock, and district six suburbs.

3.2. Feature Selection and Extraction

The satellite image covering the study area was segmented at six distinct scale parameters using the multi-resolution segmentation algorithms in eCognition software version 10.3, with varying color factor weights. The choice of the software eCognition was mainly based on the unavailability of any other object-based image analysis software packages in the laboratory made available to conduct our experiments. The multi-resolution segmentation algorithm in eCognition merges adjacent pixels of similar spectral brightness to form larger, internally homogeneous segments that are clearly distinguishable from their respective neighbors. The process stops when the set homogeneity criteria through the scale parameter are achieved [37]. It has been reported in the work of [38] that beyond the scale parameter of 40 up to 100, the number of object segments becomes more stable when image segmentations are conducted with optimal parameters. Following this, we selected the first segmentation parameter to be 50 and extended the range by incrementing by 20 until the scale parameter of 150, resulting in six segmentation scale parameters, namely 50, 70, 90, 110, 130, and 150, respectively. Each scale parameter was used to perform 8 segmentations with various color factor weights ranging from 0.2 to 0.9, making a total of 48 segmentations so that the various scale parameters could be analyzed at different color factor weights. During these preliminary segmentations, a weight of one was given to each spectral band of the imagery in order to equally retain the various spectral information offered by each individual band. From the resulting segments, unique spectral brightness measures describing the various objects were collected at each segmentation level. In addition to the spectral brightness measures, area measures were also collected. Forty-eight (48) image segments were carefully selected per segmentation level, totaling 2160 samples across all the segmentation levels in this study.

3.3. Image Variance and Spatial Autocorrelation Modeling

To evaluate the intra-segment homogeneity, we first calculated the image variance using a measure we denoted “standard deviation” image variance due to the closeness of its formulation to that of the traditional standard deviation measure. The model produces higher values for segments with low internal homogeneity, while low values of the measure would indicate segments with high internal homogeneity. The latter hypothesis fits what is generally accepted as a good image segmentation outcome. The proposed image variance model was formulated as follows:

$$\delta V_{ar}=\frac{{\displaystyle {\sum }_{i=1}^n\sqrt{{\left(a_i-\overline{a}\right)}^2}}}{{\displaystyle \sum _{i=1}^n{c}_i}}$$

(9)

where the quantity ${\displaystyle \sum _{i=1}^n{c}_i}$ is the sum of the area measures of segments at a given segmentation level. The variable $a_i$ is the brightness value of a segment of interest $i$, while the quantity $\overline{a}$ is the mean brightness value of the image at a given segmentation level. The measure $\sqrt{{\left(a_i-\overline{a}\right)}^2}$ in the expression minimizes any numerical instability that may be introduced by segments of very low brightness values. It can be observed from (9) that while smaller homogeneous objects are being merged, there is an increase in the quantity in the denominator of the model, leading to a decrease in intra-segment image variance. The model ensures a positive, numerically stable intra-segment homogeneity measure. Instead of using the Moran’s index formulation in Equation (2), we propose the following spatial autocorrelation coefficient index that we denoted $\partial MI$, which is formulated as follows:

$$\partial MI=\frac{{\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^n}\left({\displaystyle \sum \left(x_i-\overline{x_i}\right)}\times {\displaystyle \sum \left(x_j-\overline{x_j}\right)}-{\displaystyle \sum w_{ij}\left(x_i-\overline{x_i}\right)\left(x_j-\overline{x_j}\right)}\right)}{{\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^n}\left({\left({\displaystyle \sum w_{ij}}\right)}^2-{\displaystyle \sum w_{ij}}\right)}$$

(10)

where $x_i$ represents the spectral brightness of a segment of interest $i$, and $x_j$ is the nearest neighbor segment to the segment of interest $i$. The quantities $\overline{x_i}$ and $\overline{x_j}$ represent the mean brightness measures of all the segments of interest and those of all the segments spatially connected to the segments of interest, respectively. The parameter $n$ represents the total number of segments at a given segmentation level. The parameter $w_{ij}$ represents the spatial distance weight between segments of interest and their respective nearest neighbors, and the weight is assigned a value of one when the segment of interest is spatially connected to another segment, while an assignment of a value of zero indicates that there is no spatial connection between the segments. High values of the spatial autocorrelation coefficient index $\partial MI$ indicate a high inter-object heterogeneity in contrast to the traditional Moran’s Index, while low values of the index would indicate a low inter-segment heterogeneity between adjacent segments.

A high value of the spatial autocorrelation coefficient index and a low value of the image variance are required to achieve segments with low internal variance and high heterogeneity. One advantage of the above-proposed model is that it enforces two spatial autocorrelation constraints. The first constraint is enforced through the parameter of the distance matrix $w_{ij}$ present in the denominator of the expression. Since each pixel within the image is assigned a spatial location, the difference between the product terms in the numerator enforces a second spatial autocorrelation constraint between two adjacent segments $i$ and $j$. The second advantage of the adopted measure is that when more objects are merged, the distance difference in the denominator decreases in contrast to the traditional Moran’s index, resulting in an increase in the index. This association of constraints imposed by models in (9) and (10) prevents larger heterogeneous objects from being merged towards the end of the segmentation process; in other words, only larger homogeneous segments are being merged towards the end of the segmentation process.

To facilitate the graphical projection of image variance and spatial autocorrelation measures, we adopted the normalization function proposed in Equation (4). The motivation for this choice was driven by the fact that the model proposed in [21] gives dominance to the image variance against the Moran index measure since the normalization of the variance would lead to larger values due to the small numerical magnitude of the denominator. In addition, the model proposed to normalize Moran’s index prevents the index from reaching certain numerical values, and there are no clear justifications for the increment of Moran’s index value by a unit measure or why the obtained value is divided by two. From the functions in (9) and (10), we computed a heterogeneity index by modifying the numerator of the expression in (5) since the aim of the parameter selection strategy is to maintain a high value of the spatial autocorrelation measure and a low image variance measure, which cannot be achieved through the original formulation. The motivation to not use the model proposed by [32] is that there are no clear numerical rules for the assignment of numerical values to the weighting parameter $\alpha$ when performing more than two segmentation levels, and this can lead to inaccurate identification of optimal segmentation parameters. For a four-level segmentation, the authors, for instance, assigned weights of 4, 2, 0.5, and 0.25 to the parameter at the first, second, third, and fourth segmentation levels, respectively, while for a three-level segmentation, weights of 3, 1, and 0.33 were assigned to the parameter at the third, second, and third segmentation levels, respectively. The other limitation of their global score model is that even segments with high image variance can undermine the robustness of the function since a high value of the function would still be achieved. For this study, the following modified heterogeneity model was used:

$$H=\frac{\partial M{I}_{norm}-\delta Va{r}_{norm}}{\partial M{I}_{norm}+\delta Va{r}_{norm}}$$

(11)

The proposed heterogeneity function returns values close to 1 for internally homogeneous objects that are distinct from their surrounding neighbors, while values of the index nearing zero would indicate segments internally heterogeneous.

Table 1. A subset of randomly recorded normalized estimates of image variance and spatial autocorrelation measures across the 48 segmentation sublevels.

Normalized Image Variance	Normalized-Spatial Autocorrelation	Normalized Image Variance	Normalized-Spatial Autocorrelation
0.761112	0.001389	0.002861	0.027271
0.279703	0.389601	0.548032	0.894092
0.293666	0.311528	0.270968	0.182229
0.989752	0.455224	0.394835	0.198282
0.810025	0.201453	0.654612	0.384949
0.150011	0.136200	0.186838	0.632118
0.816667	0.551694	0.156677	0.093594
0.001906	0.420993	0.698041	0.076852
0.308909	0.006773	0.446272	0.874585
0.645545	0.085821	0.291909	0.108591
0.498977	0.089321	0.349643	0.025632
0.003216	0.986762	0.356380	0.230527
0.789301	0.326693	0.789180	0.521224
0.681493	0.021069	0.085201	0.746331
0.383669	0.001462	0.858952	0.787381

presents a subset of estimated normalized image variance and spatial autocorrelation measures. These measures were estimated across the six segmentation scales with various color factor weights ranging from 0.2 to 0.9.

3.4. Image Variance Optimization

In order to refine the large number of records of normalized image variance and spatial autocorrelation measures, we computed the average image variance at each individual segmentation level with color factor weights ranging from 0.2 to 0.7. These estimated mean values were useful in the optimization of the selection of segmentation scale parameters.

Table 2. Optimized image variance measures per segmentation levels at six segmentation scale parameters.

Color Factor Weight	Normalized-Image-Variance at Scale Parameter 50	Normalized-Image-Variance at Scale Parameter 70	Normalized-Image-Variance at Scale Parameter 90
0.2	0.783333	0.677419	0.0027
0.3	0.216667	0.241935	0.529032
0.4	0.266667	0.003216	0.270968
0.5	0.975200	0.320297	0.354839
0.6	0.700000	0.789300	0.451613
0.7	0.150000	0.516129	0.154839
0.8	0.816667	0.612903	0.109677
0.9	0.001900	0.000112	0.724100
Color Factor Weight	Normalized-Image-Variance at Scale Parameter 110	Normalized-Image-Variance at Scale Parameter 130	Normalized-Image-Variance at Scale Parameter 150
0.2	0.209091	0.370968	0.248120
0.3	0.372727	0.479839	0.067669
0.4	0.363636	0.657258	0.218045
0.5	0.554545	0.366935	0.961840
0.6	0.190909	0.110956	0.090226
0.7	0.875200	0.821900	0.300752
0.8	0.010200	0.306452	0.002159
0.9	0.281818	0.173387	0.045113

presents the optimized image variance per segmentation scale as a function of color factor weights.

Image variance measures give a first indication of the overall internal homogeneity of image segments at a given segmentation level. This first indication can be further analyzed in order to determine the overall internal homogeneity percentage carried by individual segments at a given segmentation level. The overall internal homogeneity percentage carried by each image object increases inversely proportional to the image variance measure. To quantify such a measure an inverse ratio of the image variance could be used; however, due to the normalization of the later measure, this would not yield realistic results. Instead, we proposed a homogeneity proportion evaluation model that returns high values when the intra-segment image variance decreases at a given segmentation level. The formulation of the proposed model is as follows:

$$\mathrm{Proportion}\mathrm{of}\mathrm{image}\mathrm{segments}’\mathrm{internal}\mathrm{homogeneity}=\frac{\left({\displaystyle \sum _{i=1}^n{c}_i}\cap {\displaystyle {\sum }_{i=1}^n\sqrt{{\left(a_i-\overline{a}\right)}^2}}\right)}{{\displaystyle \sum _{i=1}^n{c}_i}}$$

(12)

where $c_i$ is the sum of area measures of all segments at a given segmentation level, $a_i$ is the brightness of a segment of interest $i$, $\overline{a}$ is the mean brightness of the image, and $n$ is the total number of image segments at a given segmentation level. The estimated metrics were further normalized using the model adopted in (4), and

Table 3. Global homogeneity levels of segments per segmentation level.

Color Factor Weight	Proportion of Intra-Segment Homogeneity at Scale Parameter 50	Proportion of Intra-Segment Homogeneity at Scale Parameter 70	Proportion of Intra-Segment Homogeneity at Scale Parameter 90
0.2	22%	32%	99%
0.3	78%	76%	47%
0.4	73%	98%	72%
0.5	03%	68%	64%
0.6	30%	21%	54%
0.7	85%	48%	84%
0.8	18%	39%	89%
0.9	97%	98%	27%
COLOR Factor Weight	Proportion of Intra-Segment Homogeneity at Scale Parameter 110	Proportion of Intra-Segment Homogeneity at Scale Parameter 130	Proportion of Intra-Segment Homogeneity at Scale Parameter 150
0.2	79%	62%	75%
0.3	62%	52%	93%
0.4	63%	34%	78%
0.5	44%	63%	03%
0.6	80%	88%	90%
0.7	12%	17%	69%
0.8	98%	69%	98%
0.9	71%	82%	95%

presents image segments’ homogeneity levels per segmentation scale parameter as a function of color factor weights.

Figure 2 shows the variation of image variance measures per segmentation scale parameter as a function of color factor weights. Figure 2A suggests that the scale parameter of 50 did not produce meaningful image segments if associated with color factor weights of 0.2, 0.4, and 0.7. This failure of the image variance curve to produce a global minimum point at each of these segmentation levels could be due to the fact that image segments produced at these segmentation levels did not exhibit strong internal homogeneity, with the measures estimated at 22%, 78%, and 85%, respectively. In contrast, the global minimum characterizing the image variance curve at the color factor weight of 0.9 is an indication of very good segment quality. Image objects produced at this segmentation level hold an internal homogeneity level of about 97%. Associating the scale parameter of 50 with the color factor weight of 0.5 resulted in the poorest image segments’ quality, as the segments at this segmentation level exhibit the lowest internal homogeneity level at 3%. Associating the color factor weights of 0.4 and 0.9 with the scale parameter 70 produced very good quality segments, as illustrated by the global minimum points achieved by the image variance curve at these two segmentation levels in Figure 2B. These global minimum points are characteristic of internal homogeneity levels reaching values close to 98%, respectively. In contrast, the association of the scale parameter 70 with the color factor weight of 0.6 resulted in the poorest segment quality, as illustrated by the global maximum achieved by the image variance curve at this segmentation level. This global maximum point originates from the fact that image segments produced at this segmentation level are only characterized by 2% internal homogeneity. The association of the scale parameter of 90 with the color factor weight of 0.9 resulted in the poorest image segment quality. This poor segmentation quality is illustrated by the global maximum point achieved by the variance curve, as revealed in Figure 2C. At this global maximum point of the curve, image segments only achieved an internal homogeneity level of 27%. The association of the scale parameter of 90 with the color factor weight of 0.2 produced very good image segment quality, as illustrated by the global minimum of the curve achieved at this segmentation level. This global minimum is a description of the high internal homogeneity level of segments produced at this segmentation level, which reached a value of 98%. Associating a color factor weight of 0.7 with the scale parameter of 110 produced the poorest segment quality, as illustrated by the global maximum point of the variance curve in Figure 2D. At this segmentation level, segments only reached an internal homogeneity level of 12% at the end of the segmentation process. However, an adjustment of the color factor to 0.8 enabled us to increase the internal homogeneity of image segments to a level close to 98%. As a consequence of this result, the image variance curve decreased to its global minimum value. In contrast, any color factor weights associated with the scale parameter of 130 failed to produce meaningful image segments. The greatest internal homogeneity level achieved by image segments only occurred with a color factor weight of 0.6, although the achieved homogeneity level of 89% remains low for accurate urban mapping applications. The poorest segment quality was achieved when associating this scale parameter with the color weight of 0.7, and this resulted in the majority of segments only reaching a homogeneity level of about 12%, as illustrated by Figure 2E. A similar result was found when associating the color factor weight of 0.5 with the scale parameter of 150. The global maximum point of the curve achieved at this segmentation level was a consequence of image segments only achieving 3% internal homogeneity. However, adjusting the color factor weight to 0.8 improved segment quality, as the majority of image segments achieved a homogeneity level of about 98% at the end of the segmentation process.

3.5. Image Spatial Autocorrelation Optimization

Spatial autocorrelation measures were computed using Equation (10), and mean values were estimated per segmentation level. The computed data were organized into 48 groups, corresponding to individual segmentation levels.

Table 4. Estimates of mean spatial autocorrelation measures.

Color Factor Weight	Normalized-Spatial Autocorrelation-at-Scale Parameter 50	Normalized-Spatial Autocorrelation-at-Scale Parameter 70	Normalized-Spatial Autocorrelation-at-Scale Parameter 90
0.2	0.001343	0.089300	0.023700
0.3	0.390010	0.025632	0.940927
0.4	0.311528	0.986700	0.182229
0.5	0.455224	0.230520	0.198283
0.6	0.201453	0.326693	0.384950
0.7	0.136200	0.521220	0.632119
0.8	0.551694	0.021000	0.093594
0.9	0.420993	0.746331	0.076852
Color Factor Weight	Normalized-Spatial Autocorrelation-at-Scale Parameter 110	Normalized-Spatial Autocorrelation-at-Scale Parameter 130	Normalized-Spatial Autocorrelation-at-Scale Parameter 150
0.2	0.006773	0.025130	0.215200
0.3	0.874585	0.045347	0.803675
0.4	0.001462	0.042762	0.816430
0.5	0.085822	0.142021	0.783985
0.6	0.108592	0.989610	0.850612
0.7	0.787381	0.841753	0.820445
0.8	0.988785	0.943974	0.998540
0.9	0.003337	0.400006	0.944788

presents 48 mean spatial autocorrelation measures computed at each segmentation scale parameter per color factor weight.

Figure 3 presents the fluctuation of the spatial autocorrelation measure at various scale parameters as functions of color factor weights. The association of the scale parameter of 50 with any color factor weights was revealed to not produce a good spatial separation between segments, as the highest dissimilarity level achieved by image segments only reached 0.552, which corresponds to about 55% dissimilarity. However, the poorest inter-segment dissimilarity measure was achieved when associating the scale parameter of 50 with the color factor weight of 0.2. At this segmentation level, image segments only reached a dissimilarity level of 0.13%, as illustrated by the global minimum point achieved by the spatial autocorrelation curve. This is an indication that at this segmentation level, there is still a large number of image segments sharing similar spectral properties that can be merged at higher segmentation levels. An adjustment of the color factor weight to 0.4 and its association with the scale parameter of 70 produced image segments that are highly distinct from one another, as illustrated by the global maximum point achieved by the spatial autocorrelation curve in Figure 3B. At this segmentation level, image segments achieved a dissimilarity level of about 98%. In contrast, the association of this scale parameter with the color factor weight of 0.8 resulted in image segments reaching their lowest dissimilarity level at 2.10%. Similar poor segmentation performance was achieved when the scale parameter of 90 was associated with the color factor weights of 0.2 and 0.9. At these segmentation levels, the spatial autocorrelation curve reached its two lowest points. Image segmentation at these levels produced segments that were spatially distinct from one another only at 2.37% and 7.68%, respectively. However, setting the color factor weight to 0.3 enabled us to achieve segments that are highly distinct from one another, as illustrated by the global maximum achieved by the spatial autocorrelation curve in Figure 3C. At this segmentation level, the segments produced achieved a dissimilarity level of about 94%. The association of the scale parameter 110 with the color factor weights of 0.2, 0.4, and 0.9 did not produce image segments that are distinct enough from one another since the spatial autocorrelation curve achieved its lowest values as illustrated in Figure 3D. Image segments produced at these segmentation levels only achieved dissimilarity levels of 0.67%, 0.15%, and 0.33%, respectively. In contrast, an adjustment of the color factor weight to 0.8 enabled us to achieve image segments that are highly distinct from one another with a dissimilarity level close to 98%. The scale parameter of 130 failed to produce very good quality segments when associated with color factor weights of 0.2 and 0.4, as the spatial autocorrelation curve in Figure 3E achieved its lowest values at these segmentation levels. Image segments produced at these segmentation levels only achieved dissimilarity levels of about 2.51% and 4.28%, respectively. In contrast, an adjustment of the color factor weight to 0.6 enabled the improvement of the quality of image segments, and segments produced at this segmentation level achieved a dissimilarity measure of about 98%. Figure 3E shows that setting the color factor weights to 0.3, 0.4, and 0.6 only enabled us to achieve the local maximum points of the curve. At these segmentation levels, image segments were found to be distinct from their nearest neighbors at 80%, 82%, and 85%, respectively. However, an adjustment of the color factor weight to 0.8 enabled us to achieve image segments that were about 98% distinct from their nearest neighbors.

3.6. Segmentation Scale Parameter Optimization

From the segmentation results analyses, it was revealed that the scale parameter of 50, when associated with the color factor weight of 0.9, produced highly homogeneous image segments. However, the inter-segment dissimilarity level of these segments was revealed to be very low at a level of 42%. The association of the scale parameter of 70 with color factor weights of 0.4 and 0.9 produced image segments with very high internal homogeneity levels at about 98% and dissimilar to their nearest neighbors at 98% and 74%, respectively. From these results, it appears that the scale parameter of 70 could be suitable for the optimal segmentation of a certain type of urban feature. Furthermore, it was revealed that the scale parameter of 90 associated with the color factor weight of 0.2 produced image segments that were 99% internally homogeneous, but these segments were revealed to be dissimilar to their nearest neighbors only at 2%, indicating that they still share 98% of their spectral attributes with their surroundings. It can be suggested that this scale parameter is not suitable for the optimal segmentation of urban scenes within the study area. The scale parameter of 110 associated with the color factor weight of 0.8 produced image segments that were internally homogeneous at 98% and dissimilar to their nearest neighbors at 98%. These results suggest that the scale parameter of 110 can drive an optimal segmentation process that will accurately delineate a certain type of urban feature within the study area. The scale parameter of 130, in association with the color factor weight of 0.6, produced image segments that were distinct from their nearest neighbors at 98% but internally homogeneous only at 89%, and this can suggest that the scale parameter of 130 may not produce good segment delineation if used in our study area. Finally, the scale parameter of 150 associated with the color factor of 0.8 was found to produce image segments that were internally homogeneous at 98% and distinct from their nearest surrounding neighbors at about 99%. From these later results, it can be suggested that the scale parameter of 150 could optimally segment a certain type of urban object within the study area.

Consolidating these results in

Table 5. Computed optimal heterogeneity function measures per scale parameter.

Scale Parameter	Lowest Normalized Image Variance Measures	Largest Normalized Spatial Autocorrelation Measures	Heterogeneous Function
50	0.150000	0.551694	0.572463
70	0.003216	0.986700	0.993503
90	0.109677	0.940927	0.791211
110	0.010200	0.988785	0.979579
130	0.110956	0.989610	0.798366
150	0.002159	0.998540	0.995685

enables us to compute the segmentation heterogeneity measures through Equation (11). The estimated heterogeneity function enables us to confirm the authenticity of the potential ideal segmentation scales pre-identified above. High values of the function confirm that produced segments are internally homogeneous and highly heterogeneous from one another, while low values of the index indicate the presence of segments that are not internally homogeneous, thus not highly heterogeneous from one another.

Figure 4 presents the heterogeneity function curve, showing low and high points. The identified peak points indicate the optimal segmentation scale parameters suitable to perform segmentations of the satellite image and achieve good-quality segments. It can be observed that the scale parameters of 70, 110, and 150 would produce the best image segment quality as they are associated with peaks of the heterogeneity curve, and this consolidates our earlier assumptions on the suitability of these segmentation scale parameters to successfully segment urban features within the study area. In contrast, the scale parameters of 50, 90, and 130 did not qualify as optimal segmentation scale parameters as they failed to produce peaks of the heterogeneity curve.

4. Results and Discussion

Figure 5 shows that the image variance curve follows a very regular fluctuation towards the end of the segmentation, following a sinusoidal-like shape, revealing the robustness of the new proposed image variance model. This result is in line with the statement of [22], who are of the view that a good image variance measure should produce a curve that follows a regular fluctuation shape towards the end of the segmentation. These results are also in line with [23], who argued that a good image variance model should not produce an increase in the image variance towards the end of the segmentation.

The fact that the curve of spatial autocorrelation measures follows a nearly linear trend towards the end of the segmentation shows that segments have reached some status of internal stability, which means that large distinct objects are not being forced to merge towards the end of the segmentation process, in contrast to current segmentation parameter selection methods that rely on the weighted image variance and Moran’s index. These observations indicate that the proposed image variance and spatial autocorrelation formulations are suitable for urban areas, which are very heterogeneous and complex areas, and this is in line with the suggestions made by [21]. Figure 6 illustrates the segmentation of a built-up scene at a scale parameter of 50 (A) and a scale parameter of 70 (B). It can be observed that the segmentation at scale 50 has over-segmented most of the built-up features, including regular-sized buildings as well as large buildings and some trees near buildings. In contrast, the segmentation at a scale parameter of 70 has improved the delineation errors, and all buildings have achieved outlines that are very close to their real-world measurements. This result is coherent with earlier findings by [39], who also found the scale parameter of 70 to be suitable for the segmentation of standard sized and slightly large buildings.

Figure 7 shows the segmentation of a large parking lot at scale parameter 90 (A) and the segmentation of the same area at scale parameter 110 (B). It can be observed that the segmentation at scale parameter 90 produced internally homogeneous objects; however, many of these objects seem to still share spectral similarities and thus are not highly heterogeneous to one another, and this is in line with the earlier results found in Figure 5, which show low heterogeneity between segments at this specific scale parameter. However, the segmentation at a scale parameter of 110 shows an improvement in inter-segment heterogeneity as smaller segments that share similar spectral attributes were successfully merged to produce the full extent of the parking lot. It can also be observed along the diagonal from the top right corner and near the bottom of the image that the scale parameter 110 achieved a good segmentation of elongated urban features, particularly roads that reached near real-world boundaries. However, it is revealed that the majority of standard-sized buildings were under-segmented at this scale parameter. This result confirms an earlier observation by [39], who suggested that scale parameters larger than 100 are not generally suitable for standard and small-sized buildings.

Figure 8 shows the segmentation results of a large green sports field at scale parameters 130 (A) and 150 (B). Unlike the scale parameter of 90, the scale parameter 130 shows some high heterogeneity between the resulting small segments, as most of the segments at this specific scale parameter seem slightly different in terms of spectral brightness. However, a visual examination of individual segments reveals that many of these segments are not internally homogeneous as they seem to carry more than one color tone and this visual result confirms the earlier observation in Figure 5, where distinct segments produced at scale parameter 130 were associated with a high internal segment variance.

Figure 9 shows the segmentation of a large swimming pool structure at scale parameter 130 (A) and at scale parameter 150 (B). As pointed out earlier, segments produced at scale parameter 130 are not yet internally stable in terms of brightness, and this can be extended to the tree cover and some green areas around the swimming pool. However, a segmentation at scale parameter 150 can stabilize the internal variance of the image by further merging smaller segments with the larger ones, and the entire outline of the swimming pool area, the blue roofing structure on the side of the swimming pool as well as some nearby tree patches, were successfully reconstructed.

The segmentation parameters evaluation strategy proposed in this study was further tested on a 0.5 m resolution color aerial photograph covering the same study area as the satellite GeoEye imagery. Radiometric, atmospheric, and geometric corrections were already performed on the imagery acquired from the department of rural development, agriculture and land reform in Pretoria. Three urban scenes were selected from the imagery and subjected to several segmentation tasks at varied scale parameters. Figure 10 illustrates subsets of segmentation results achieved with the scale parameters 40 (B), 50 (C), and 70 (D). The segmentation results in Figure 10B and 10C show poor delineation of building units that were subjected to over- and under-segmentation errors. However, the segmentation result in Figure 10D shows successful delineations of these building units.

Figure 11 presents subsets of segmentation results obtained from the partitioning of the aerial photograph at scale parameters 90 (B), 100 (C), and 110 (D). The result in (D) shows a successful delineation of roads, while the results in (B) and (C) reveal over-segmentation occurrences on the linear urban objects.

Figure 12 shows subsets of segmentation results for urban vegetation cover at scale parameters 120 (B), 130 (C), and 150 (D). The results reveal that scale parameters 120 and 130 could not produce meaningful image segments of urban trees. However, a segmentation at the scale parameter of 150 successfully merged small tree patches into larger homogeneous tree segments that are distinct from their nearest neighbors.

5. Accuracy Assessment

The segmentation accuracy assessment was performed through five quantitative metrics, namely the quality rate ($QR$), the area fit index ($AFI$), the over-segmentation $\left(OS\right)$, the under-segmentation $\left(US\right)$, and the root mean square error $\left(RMS\right)$. The quality rate error quantifies how large the under-segmentation error is [40]. Small values of the measure are achieved in an optimal image segmentation where the outlines of segments are very close to their real-world references. The Area Fit Index quantifies the over-segmentation error, and negative values of the index indicate a one hundred percent overlap between the reference polygon and its corresponding segment image and an overall under-segmentation of the image. Values of the AFI greater than zero generally describe an overlap of less than 100 percent between the reference polygon and its corresponding segment. The under-segmentation error ($US$) describes the amount of over-segmentation in the image, while the over-segmentation error $\left(OS\right)$ characterizes the amount of under-segmentation to which individual segments were subjected. To compute these five assessment measures, we manually digitized some buildings, roads, parking, and swimming pool polygon samples using ArcGIS version 10.8. Area measures of the digitized polygons were converted into pixels. These measures were used with the associated segment area measures to compute the quality rate, area fit index, over and under-segmentation errors, as well as the root mean square error. A total of 390 image segments were carefully selected within the study area, with 65 segment polygons considered per urban scene. Table 6 shows computed values of quality rate, area fit, under and over-segmentation error indices, as well as the root mean square error of six state-of-the-art segmentation evaluation approaches against our proposed strategy.

From the second column of Table 6, it can be observed that the segmentation parameter evaluation strategies proposed by [41] and that proposed by [16] produced the largest quality rate errors, followed by the strategies proposed by [5,24,42,43]. However, our proposed segmentation evaluation strategy achieved the lowest quality rate error, and this implies that the overall under-segmentation to which the various image segments were subjected is very negligible [40]. When it comes to the Area Fit Index, the segmentation parameter evaluation method proposed by [41] produced a positive measure of the error, which indicates that most image segments were subjected to over-segmentation. The evaluation strategies proposed by [42,43] and our proposed method all achieved negative index values. This indicates that the majority of image segments were subjected to under-segmentation errors. However, in terms of the magnitude of the error, our proposed strategy achieved the best results at 0.006. The largest under-segmentation errors were achieved through the segmentation parameter selection methods proposed by [5,24,42,43,44], respectively. The parameters evaluation strategy proposed by [41] achieved the lowest under-segmentation error at 0.005, followed by our proposed strategy at 0.008. This means that some image segments in both evaluation methods were subjected to over-segmentation. A look at the Area Fit Index indicated that the amount of over-segmentation is more prominent in the strategy proposed by [41] than that achieved through our proposed strategy [45]. An observation of the fifth column of Table 6 indicates that the newly proposed segmentation parameter evaluation produced the lowest over-segmentation error, followed by the strategies proposed in [41,42]. This means that the amount of under-segmentation error under which image segments were subjected in our strategy was very negligible, at about 0.782% of the size of the reference polygons, which is a very good and acceptable error. In terms of overall performance, the last column confirms that the newly proposed segmentation parameter evaluation strategy produced the lowest over and under-segmentation errors.

Table 6. Average estimates of five image segmentation accuracy assessment indices from six state-of-the-art methods.

Segmentation Assessment Metrics	QR	AFI	US	OS	RMSE
Yang et al., (2014) [24]	0.257700	-------	0.220500	0.127550	0.180000
El-Naggar, (2018) [42]	0.386800	−1.062000	0.329000	0.058000	0.273000
Vamsee et al., (2018) [43]	0.306700	−0.145000	0.311700	0.295000	0.368000
Wang et al., (2019) [5]	0.314500	-------	0.148200	0.191500	0.171000
Norman et al., (2020) [41]	0.483200	0.004580	0.005000	0.009730	0.014000
Dao et al., (2021) [16]	0.490000	-------	0.080000	0.970000	0.688000
He et al., (2024) [44]	0.321100	-------	0.238000	0.246900	0.342900
The proposed method	0.003910	−0.005740	0.007780	0.007820	0.005510

Table 6. Average estimates of five image segmentation accuracy assessment indices from six state-of-the-art methods.

Segmentation Assessment Metrics	QR	AFI	US	OS	RMSE
Yang et al., (2014) [24]	0.257700	-------	0.220500	0.127550	0.180000
El-Naggar, (2018) [42]	0.386800	−1.062000	0.329000	0.058000	0.273000
Vamsee et al., (2018) [43]	0.306700	−0.145000	0.311700	0.295000	0.368000
Wang et al., (2019) [5]	0.314500	-------	0.148200	0.191500	0.171000
Norman et al., (2020) [41]	0.483200	0.004580	0.005000	0.009730	0.014000
Dao et al., (2021) [16]	0.490000	-------	0.080000	0.970000	0.688000
He et al., (2024) [44]	0.321100	-------	0.238000	0.246900	0.342900
The proposed method	0.003910	−0.005740	0.007780	0.007820	0.005510

6. Conclusions

Image segmentation remains a very important step in object-based image analysis. However, the development of strategies for selecting optimal segmentation-scale parameters for multi-resolution image segmentation is still ongoing and remains a great challenge. This study proposed an unsupervised strategy for the selection of optimal segmentation-scale parameters and color factor weights. The new strategy presents alternative image segmentation evaluation paradigms in addition to those presented in the works of [5,16,25,41,42,43,44]. The proposed robust, repeatable strategy for optimal selection of segmentation parameters (OSSP) consists of three modules. The first module, based on the concept of standard deviation, computes non-weighted image variance to keep the measure independent from area-size errors that may originate from over- and under-segmentation of the imagery. The second module partially relies on the current formulations of Moran’s index and the correlation coefficient to evaluate the spatial autocorrelation of the image in order to reinforce the separation of internally distinct objects during the segmentation process. When the spectral distance between objects increases, the measure levels off in opposition to the traditional trend of this measure. For our measure, the heterogeneity between objects would always increase when the spectral distances between objects increase, which is best for good segmentations. The third module consists of a heterogeneity index that differs from current formulations in terms of the numerator. This module can find multiple optimal segmentation scale parameters as well as associated color factor weights that would guarantee a good delineation of the spatial outlines of the resulting image segments. The effectiveness of the optimal selection of segmentation scale parameters strategy in extracting distinct segments of land use and land cover from multispectral imagery was validated with a GeoEye multispectral image and a color aerial photograph. The optimal segmentation scale parameters obtained from the proposed OSSP strategy could successfully outline land use and land cover of various sizes, including small and standard-sized buildings, large buildings, parking areas, roads, water bodies, and urban trees. Although the proposed approach achieved successful segmentations for most urban land use and land cover features, some objects were slightly over and under-segmented, but these errors were quantified better than some of the state-of-the-art existing methods. Minor under-segmentation in urban mapping using object-based image analysis is preferable, especially when dealing with very complex environments such as urban environments. These minor under-segmentation errors can generally be corrected through manual re-segmentation and merging. Associating the obtained individual optimal segmentation scale parameters to urban features of interest within the imagery was performed through visual observations. Our evaluation strategy produced image segments with outlines very close to their real-world characteristics. Standard-sized building objects were better delineated using low color factor weights, which can be translated to a higher shape compactness measure. This was observed at a scale parameter of 70 with a color weight of 0.4. However, when increasing the segmentation scale parameter measures, larger color weights seem to play a more significant role than shape compactness in producing image segments of good quality, as observed at scale parameters 110 and 150. The proposed segmentation parameter selection strategy successfully brought out three inherent optimal scale parameters suitable to segment the high-resolution multispectral GeoEye image and the aerial photograph, and the tool has the potential to efficiently improve knowledge-based image analysis of complex urban areas.

It was found in this study that the internal homogeneity of image segments does not necessarily increase with an increase in segmentation-scale parameter values or color factor weights. The study found that small values of the color factor weight can optimize the performance of large-scale parameters in opposition to larger weights of the parameter and vice versa. Large color factor weights were also found to optimize the performance of large segmentation scale parameters, and it can be argued that medium-range scale parameters would perform very well with large shape compactness/small color factor weights for the segmentation of urban scenes. In addition, large segmentation-scale parameters would perform very well with larger color factor weights or smaller shape compactness values. It was also found that the number of spectral bands carried by the imagery also affects the segmentation outcomes, and this finding came from the visual comparison of segmentation results from both the satellite image and the aerial photograph, as some minor over- and under-segmentation occurrences were observed on the aerial image segmentation results while they were inexistent on the satellite image segmentation results. Further investigation of this study could involve an urban environment that contains informal settlements and a large amount of vegetation cover and the consideration of a multispectral image with more spectral bands to assess the performance of the newly proposed modules in handling larger spectral variability. Other segmentation parameters, such as shape compactness and an extension of the range of scale parameters, could also be considered in further investigations. Combining the developed strategy with the concepts of deep and machine learning could also be investigated for future works. To improve the speed of selection of optimal segmentation parameters, an automated heterogeneity graph interpretation algorithm could also be developed. This study did not directly focus on image classification assessment; however, there is a direct implication between improvements in image segmentation quality and image classification results. It has been advocated that improvements in image segmentation quality would result in improved image classification outcomes [46].