People Detection Using Artificial Intelligence with Panchromatic Satellite Images

Abstract

The detection of people in urban environments from satellite imagery can be employed in a variety of applications, such as urban planning, business management, crisis management, military operations, and security. A WorldView-3 satellite image of Prague was processed. Several variants of feature-extracting networks, referred to as backbone networks, were tested alongside the Faster R–CNN model. This model combines region proposal networks with object detection, offering a balance between speed and accuracy that is well suited for dense and varied urban environments. Data augmentation was used to increase the robustness of the models, which contributed to the improvement of classification results. Achieving a high level of accuracy is an ongoing challenge due to the low spatial resolution of available imagery. An F1 score of 54% was achieved using data augmentation, a 15 cm buffer, and a maximum distance limit of 60 cm.

1. Introduction

Currently, the detection of people and the quantification of human flows are mainly based on processing street camera imagery. However, due to their limited range of view, street cameras are not sufficient for monitoring large areas. Furthermore, the detection of people from satellite images proves to be more difficult than the detection of other target objects, such as planes or ships. Despite this challenge, satellite images promise many advantages, such as providing an overview of large areas within one time frame and monitoring closed or restricted areas as well as internal parts of residential or industrial complexes. People detection is of importance to security and defense, especially when tracking the movement and gathering of people in potentially dangerous areas. Economic and sociological studies evaluating the appearance of people in public spaces over a longer period of time can also be significant. Other uses include humanitarian monitoring, where the detection and counting of people in refugee camps or affected areas can improve the efficacy of humanitarian aid, border monitoring, and control of illegal immigration [1,2,3,4,5,6,7].

The advent of new satellite missions and enhanced sensing capabilities enables exploration into this new area of applications. The spatial resolution of satellite imagery can reach as much as 30 cm, and new missions, especially satellite swarming and constellation, are becoming clear trends in Earth observation applications [8]. Single satellites simply cannot achieve the necessary frequency of images with the spatial or temporal resolution needed for analysis or provide the necessary data for decision making in near real time [9].

The proliferation of high-resolution images and the development of machine learning (ML) algorithms have greatly enhanced the analysis of these data. Artificial intelligence has unlocked a new perspective on solving various problems, such as the detection of target objects, like people [10,11]. Our study employs a convolutional neural network (CNN), which is a deep learning model capable of directly extracting features from satellite imagery data. It is effective at exploiting realistic imagery data characteristics and has been shown to be highly successful [7,11]. Existing applications in remote satellite sensing are focused on the detection of large persistent objects, such as buildings or swimming pools [12]; applications for the detection of smaller objects, such as people, are lacking. When detecting people, various limitations exist, such as the resolution of satellite images, the costs of the images themselves, and the orientations of the images (bird’s eye view).

Our objective is to assess the current possibilities for people detection in an urban environment from satellite imagery with the highest available spatial resolution, and then to shed light on the potential future exploitation of satellite imagery for such applications. The biggest challenge to this end is the low spatial resolution of current satellite imagery, in which the overhead view of a person is reflected in only a few pixels (typically four pixels in our case). We propose using a combination of several techniques, such as Faster R-CNN, the extension of data augmentation by buffer use, and a specific validation process, to achieve the most accurate results.

The paper is organized as follows: an introduction, a background that provides an overview of state-of-the-art CNN method technology, a description of the field of study and data sources, an explanation of CNN processing, the results and calculated quality parameters describing the success of the human detection methods, and, finally, a discussion of the results and limitations of these methods.

2. Materials and Methods

2.1. Background

Among the various potential ML methods one could use, rapid progress in selected convolutional neural network (CNN) methods has been attracting the most attention recently. CNN methods are able to efficiently learn distinctive high-level features of object detection in remote sensing [7,13]. CNN algorithms can be divided into two categories: so-called two-stage detectors (RCNN) and single-stage detectors (SSD, YOLO, and RetinaNet) [14,15].

Faster R-CNN is used for generic object detection and has been successfully adapted from its two predecessors, R-CNN and Fast R-CNN [16,17], to solve many recognition problems. Faster R-CNN consists of two modules: a region proposal network (RPN) and a Fast R-CNN detector [14,16,17]. Faster R-CNN improves object detection architecture by replacing the selection search algorithm in Fast R-CNN with a region proposal network (RPN) [14,18]. An RPN is a fully convolutional network for proposal generation [14]. The rest of the model architecture remains the same as in Fast R-CNN. The system overview of Faster R-CNN is given in Figure 1.

Duporge and Isupova [20] applied a CNN model to automatically detect and count African elephants in a woodland savanna ecosystem in South Africa. They used WorldView-3 and -4 satellite data. Dumitrescu, Boiangiu, and Voncilă [21] focused on creating a fast and reliable object detection algorithm that is trained on scenes depicting people in an indoor environment. This method combines YOLOv4 and Faster R-CNN. Additionally, Li and Ma as well as Fu and Xu [22,23] used Faster R-CNN to detect people in a sequence of video images. On the other hand, Wang et al. [4] used Mask R-CNN, a representative extension of Faster R-CNN, which produces segmentation masks of objects. Ren, Zhu, and Xiao [24] used a modified Faster R-CNN to detect ships and planes in optical remote sensing images (NWPU VHR-10).

When compared to other popular object detection methods, like YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), or RetinaNet, Faster R-CNN offers distinct advantages that make it particularly effective for this specific application; Faster R-CNN’s RPN generates high-quality region proposals that are more accurate and reliable compared to the single-stage approaches used in YOLO, SSD, and RetinaNet. The RPN is designed to propose regions that are likely to contain objects of interest, which is crucial for detecting small target objects like people in urban satellite imagery. This two-stage approach, where the RPN first proposes regions and then a classifier fine-tunes the detection, ensures higher precision and recall, which are critical for applications where accurate human detection is paramount [14,16,17,18,19,25]. Although YOLO and SSD are known for their speed, they often sacrifice accuracy, particularly in complex and cluttered scenes. YOLO’s single-stage detector processes the entire image concurrently, providing a faster detection time, but at the cost of reduced accuracy for small objects and increased false positives. SSD faces similar issues. RetinaNet introduces focal loss to handle the class imbalance problem (i.e., many more background examples than objects) in object detection [15,26]. In contrast, Faster R-CNN’s two-stage process allows for the more precise localization and classification of people, which is essential when dealing with the varied and detailed backgrounds of urban environments [25,27].

Deep ML methods require a high volume of data, namely for their training [28,29]. The number of training samples for object detection using CNN algorithms can vary widely depending on several factors, including the complexity of the task, the diversity of the objects, the sizes of the objects in the images, and the architecture of the CNN being used. CNN algorithms enable the extraction of image features, which can be effectively processed to reduce the dimensionality of the task [30].

Additional data manipulation methods (data augmentations) are applied to achieve improved results. These techniques help models generalize better by exposing them to a wider variety of image conditions. The following data augmentation techniques for object detection from satellite images are frequently employed: rotating, flipping, shifting, translating, and clipping [11,31,32].

2.2. Study Areas and Data

Prague was selected as the area of interest for analysis of the occurrence of people on streets. The study area in Prague consists of three subsets: Prague Castle, Charles Bridge, and the Old Town Square, including its surroundings (Figure 2). Two reasons for these selections apply: the areas contain various surface types (to verify the ability to detect people in various urban conditions) and the frequent occurrence of people in these locations at the given time (Figure 3). One full WorldView-3 scene from the morning of 23 July 2019, covering 25 km², was obtained. The spatial resolution of the WorldView-3 panchromatic and multispectral images is 0.3 m and 1.6 m, respectively.

In these study areas, ground truth data were obtained by visual vectorization. Ground truth data (

Table 1. Ground truth data in Prague.

Location	Number of People
1—Old Town Square—surface 1	214
2—Old Town Square—surface 2	15
3—Old Town Square—surface 3	19
4—Old Town Square—surface 4	9
5—Charles Bridge	479
6—Prague Castle—surface 1	63
7—Prague Castle—surface 2	116
8—Prague Castle—surface 3	35
Total	950

) are needed for training and accuracy assessment. After vectorization, an exploratory data analysis was performed in which it was found that people in the satellite images were shown as 4 pixels that shared a neighborhood (Figure 3). The digital numbers (DNs) of these pixels were significantly higher than the DNs of surface pixels.

The surface types were as follows:

• Location 1—interlocking pavement.
• Location 2—small square light cobbles, forming large square areas, bordered by small square dark cobbles.
• Location 3—small square light cobbles, forming small square areas, bordered by small square dark cobbles.
• Location 4—small square light cobbles, forming large rectangular areas, bordered by small square dark cobbles.
• Location 5—interlocking pavement.
• Location 6—square light cobbles, forming large square areas, bordered by square dark cobbles.
• Location 7—interlocking pavement.
• Location 8—marble pavement.

Vectored digital data for buildings and greenery were obtained from digital technical maps of IPR Prague (The Prague Institute of Planning and Development) as appropriate auxiliary data. These data enabled the creation of masks that improve the quality of people detection by hiding or blocking certain parts of the input imagery, allowing the neural network to focus on a specific area of interest. By excluding irrelevant areas using the mask, computation can be reduced, leading to a potential increase in detection speed.

2.3. Methodology

Upon review, Faster R-CNN and YOLO were selected for data processing using implementation in ArcGIS Pro version 3.3.1. The selection of these most promising models was confirmed by the “AutoDL” function in ArcGIS. The Python programming language and ArcGIS Python API library were used to automate various subtasks (available at https://github.com/PavelKVSB/people_detection_AI_Satellite_Images, accessed on 2 September 2024), and the hardware parameters include Intel Core i7-8700 CPU 3.20 GHz; NVIDIA GEFORCE GTX 1070; and 16 GB RAM.

To begin, an image that contained three RGB bands was created from a panchromatic image because neural network algorithms perform better with this image type [33,34]. Next, it was necessary to create training data from the ground truth data for the Faster R-CNN model. To generate the training data, the following parameters were set: tile size (size of the image chips) at 128, stride size (distance to move in X and Y when creating the next image chip) at 64, and the metadata format of training data in PASCAL Visual Object Classes in .xml format. This part of the processing was carried out in ArcGIS Pro. After the training data were prepared, the model was trained using Jupyter Notebooks (version 7.2.2) with the GPU enabled in ArcGIS Pro SW. The training data were prepared, and the sizes of the validation portion (20%) as well as the batch size were set. Typical mini-batch sizes were 16, 32, 64, 128, 256, and 512 [35], depending on CPU/GPU availability. The Faster R-CNN algorithm was imported and the backbone model was configured. In addition, the “learning rate find” function was utilized to select the most suitable learning rate for model training. After these settings were fixed (

Table 2. Settings and results of Faster R-CNN model training.

	Input Parameters				Optimization	Quality Assessment
Variant	Buffer (cm)	Backbone	Batch Size	Epochs	Learning Rate	Average Precision Score (%)
1.	0	Resnet101	8	50	0.000479	18.82
2.	0	Resnet101	2	50	0.000575	14.15
3.	0	Resnet101	32	50	0.000132	15.09
4.	0	Resnet101	32	150	0.000158	24.67
5.	0	Resnet101	64	300	0.000229	22.96
6.	0	Resnet101	16	150	0.000158	21.24
7.	0	Resnet152	8	100	0.000331	27.10
8.	0	Resnet152	32	150	0.000191	23.57
9.	0	Resnet152	16	150	0.000158	31.45
10.	0	Resnet152	24	150	0.000229	24.58
11.	0	Resnet152	20	150	0.000275	23.15
12.	0	Resnet50	16	150	0.000275	28.32
13.	15	Resnet152	16	150	0.000191	41.14
14.	30	Resnet152	16	150	0.000132	41.51

), the given model was trained. The 13th and 14th methods achieved the highest values of the average precision score (AP) (almost 42%, Table 2). The AP is the precision average across all recall values between 0 and 1 [36]. The average precision is high when both precision and recall are high, and low when either of them is low across a range of confidence threshold values [37]. It can also be seen from Table 2 that when the batch size is smaller, the average precision score is lower. However, even a large increase in the batch size does not necessarily lead to better results; it is necessary to find the best batch size value, which in our case was 16. After testing various backbones available for Faster R-CNN, Resnet152 was found to provide the best results. Variants 13 and 14 have a higher AP value than all other variants. This is because, in these cases, training data were used with a buffer of 15 cm (13th) and 30 cm (14th). All testing was processed in the ArcGIS SW environment.

By utilizing the buffer parameter, the models received a significant boost in their APs, as indicated in Table 2. Furthermore, this clearly demonstrates that the most optimal outcomes were obtained when the backbone was configured as Resnet152, the batch size was set to 16, and the number of epochs was set to 150. Therefore, these settings were used for further optimization with data augmentation. The following data augmentation functions were used:

• Horizontal flip—The model learns to recognize objects regardless of whether they are oriented to the left or right. This is especially beneficial in tasks where the orientation of objects is not fixed, such as object detection. By enhancing the diversity of the input data, this technique reduces the risk of overfitting and improves the model’s ability to generalize to new, unseen data.
• Vertical flip—This transformation can introduce variability that helps the model generalize better to different orientations.
• Rotation—Random rotation of the image by a maximum of 45° in either direction (clockwise or counterclockwise). This transformation allows the model to become more robust to slight variations in the orientation of objects.
• Zoom—Images were randomly zoomed up to 50%. Zooming helps create a variety of scales within the dataset.
• Warp—Random generation of different deformation of images (including skewing, stretching, and shifting), within the specified 0.3 range. Warping simulates natural distortions and irregularities that might be present in real images.

The use of the “get transforms” function is essential for data augmentation. This function applies a sequence of transformation operations to images during both the training and validation processes. These transformations are random and are applied every time the images pass through the model, meaning that each image may appear differently in each batch. This variability is key for increased data variability, which leads to improved model generalization and a reduced overfitting risk. As a result, the “get_transforms” function enhances the effectiveness of the training process, thereby improving its robustness and performance when encountering new, unseen data.

Variants 15 and 16 represent outputs of training with data augmentation. Variant 15 uses the training data without a buffer, and the AP reached 42% (

Table 3. Settings for training of the Faster R-CNN model using data augmentation.

	Input Parameters				Optimization	Quality Assessment
Variant	Buffer (cm)	Backbone	Batch Size	Epochs	Learning Rate	Average Precision Score (%)
15	0	Resnet152	16	150	0.000092	42.15
16	15	Resnet152	16	150	0.000109	48.65
17	15	VGG16	16	150	0.000275	37.61
18	15	VGG19	16	150	0.000132	37.24
19	15	InceptionV3	16	150	0.000091	31.55
20	15	InceptionV4	16	150	0.000229	0.29
21	15	DenseNet121	16	150	0.000331	0.91
22	15	DenseNet161	16	150	0.000229	1.24
23	15	DenseNet169	16	150	0.000191	0.47
24	15	DenseNet201	16	150	0.000190	0.97
25	15	MobileNet V3	16	150	0.000132	42.65

). When training data are applied with a buffer of 15 cm, the AP value increased to 49%. Based on the results obtained using the Resnet152 backbone, data augmentation was applied to other backbones as well. Subsequently, various additional backbone variants were tested, as shown in Table 3. None reached better results than ResNet152 with a 15 cm buffer. The variant with MobileNet V3 reached almost 43%, VGG16 and VGG19 37%, Inception V3 31.55%, and V4 only 0.29%. The DenseNet backbones (121, 161, 169, and 201) do not provide useful results (mAP about 1%) The best results among the tested backbones were achieved by the MobileNet V3 backbone, with an AP of 42.65%. Slightly lower results were recorded for the VGG16 and VGG19 backbones, where the AP was around 37%. The worst results were obtained using DenseNet, with an AP of approximately 1%.

The training process and validation are depicted by the plotting of validation and training losses after fitting the model (Figure 3). In both cases, the function on the training data is decreasing, i.e., the model is learning, and its predictions are improving. When the loss function on the validation data decreases, it means that the model not only predicts well on the training data, but also generalizes to new data. This is important because it is necessary to have a model that will perform well on various data. The difference between training and validation loss is also important. If the difference between the two losses is large, it may indicate that the model is overtrained (overfitting), which means that it has learned the specifics of the training data too well and cannot generalize to new data. If the curves are close together, it is a good sign that the model generalizes well [38]. The functions for variant 9 show larger differences (Figure 4a) than the functions for variant 15 (Figure 4b), where data augmentation was applied.

The trained model then needed to be tested on a sample of data that was not used during training. ArcGIS Pro with its “Detect Objects Using Deep Learning” function was employed to test the model. Four models were tested: variants 9 and 13 without data augmentation, and variants 15 and 16 with data augmentation. All four models were tested on an image without a mask and one with a mask (Figure 5). The predictions that had a confidence score higher than 50% were included in the result as predicted people.

Simultaneously, the YOLOv3 model was trained with different settings (

Table 4. Settings for the training of the YOLOv3 model using data augmentation.

	Input Parameters				Optimization	Quality Assessment
Variant	Buffer (cm)	Backbone	Batch Size	Epochs	Learning Rate	Average Precision Score (%)
26.	15	Darknet53	8	300	0.002512	2.26
27.	15	Darknet53	16	300	0.002089	4.04
28.	15	Darknet53	32	300	0.003631	36.32
29.	15	Darknet53	64	300	0.000832	4.27

). According to the results of the Faster R-CNN testing, this testing was focused on models with data augmentation and a buffer of 15 cm. The batch size was set to 32, 300 epochs were used, and Darknet53 was used as a backbone. The YOLO model reached a maximum AP of 36.32%, much lower than the AP achieved by the Faster R-CNN model with data augmentation and a 15 cm buffer.

In addition, other widely used object detectors, such as RetinaNet and Single Shot Detector (SSD), were evaluated using ArcGIS Pro software (version 3.3.1) implementation. A class arcgis.learn.SingleShotDetector() allows the user to create an SSD object detector with the specified parameters, including grid sizes used for creating anchor boxes, zoom scales and aspect ratios of the anchor boxes, backbone models for feature extraction, and dropout probability. The SSD implementation supports the backbones from the ResNet, DenseNet, and VGG families. A class arcgis.learn.RetinaNet() creates a RetinaNet object detector with the specified zoom scales and aspect ratios of anchor boxes, as well as backbone models. Only backbones from the ResNet family are supported by RetinaNet implementation. The aspect ratio parameters were set to 1:1 to define the square shape of the labels. Multiple zoom parameters were tested to define the appropriate scale of the anchor boxes to cover the sizes of the detected people in the image.

Table 5. Settings for the training of the SSD and RetinaNet models using data augmentation.

Model type	Ratios	Backbone	Batch Size	Learning Rate	Zoom/ Scales	Grids	Dropout
SSD	[1.0, 1.0]	VGG16, VGG19, ResNet152, and DenseNet201	2, 4, 8, 16, 32, and 64	Define by lr_find() function	[0.1–0.5]	4,8,16,32	0.2
Retina-Net	[1.0, 1.0]	ResNet50, ResNet101, and ResNet152	2, 4, 8, 16, 32, and 64	Define by lr_find() function	[2–8]	NA	NA

summarizes the parameters setting used for the SSD model and the RetinaNet model.

The above testing in ArcGIS Pro implementation yielded very poor results, with a maximal average precision value of only 0.1% for the SSD model employing the VGG19 backbone, a batch size of 16, a zoom of 0.5, and a grid of 8.

Based on the poor results of the YOLO model and ArcGIS Pro implementations, only the Faster R-CNN model was further tested.

2.4. Validation Process

After applying the methods, it is necessary to determine how accurate each of the methods are. The schema of validation process is shown in the Figure 6. First, it is necessary to determine the center of gravity for each predicted polygon. This is an approximation of the most probable location of a predicted person (PP). Next, the shortest distances between the PP and the ground truth data (GT) are compared. This is calculated as follows: A Euclidean distance for each pair of GT and PP locations is determined and the nearest distance for each GT is found. Both points are then excluded from the list to allow for a subsequent search, where the next shortest distance between points is found until the shortest distances for all points have been determined. A new script to provide such a function was prepared in the Python language since the available software lacked a suitable function. The results then undergo statistical evaluation to determine the quality of the methods. The precision, recall, and F1 scores are calculated, for which it is necessary to evaluate TPs (true positives), FPs (false positives), TNs (true negatives), and FNs (false negatives). When evaluating the precision, recall, and F1 scores, the distance limits between the ground truth points and the detected points must be established. The following distance limits were set: 15 cm (corresponding to a 1/2 pixel), 30 cm, 45 cm, and 60 cm.

3. Results

Table 6. Results of people detection without data augmentation and without a buffer.

Method	Limit (cm)	TP	FP	FN	Precision (%)	Recall (%)	F1 (%)
Faster R-CNN without mask	15	7	107	207	6.1	3.3	4.3
	30	33	81	181	28.9	15.4	20.1
	45	46	68	168	40.4	21.5	28
	60	51	63	163	44.7	23.8	31.1
Faster R-CNN with mask	15	6	48	208	11.1	2.8	4.5
	30	16	38	198	29.6	7.5	11.9
	45	30	24	184	55.6	14	22.4
	60	38	16	176	70.4	17.8	28.4

shows the results of people detection without using data augmentation or a 15 cm buffer. The F1 score at 60 cm corresponds approximately to the AP when training the given model (31%). When using the mask, the number of TPs decreased and the number of FNs increased. There was also a lower number of FPs, caused by a smaller number of predicted people; therefore, the value of the F1 score reached only 28%.

Table 7. Results of people detection without data augmentation but with a 15 cm buffer.

Method	Limit (cm)	TP	FP	FN	Precision (%)	Recall (%)	F1 (%)
Faster R-CNN without mask	15	10	177	204	5.3	4.7	5
	30	40	147	174	21.4	18.7	20
	45	58	129	156	31	27.1	28.9
	60	68	119	146	36.4	31.8	33.9
Faster R-CNN with mask	15	11	110	203	9.1	5.1	6.6
	30	34	87	180	28.1	15.9	20.3
	45	54	67	160	44.6	25.2	32.3
	60	67	54	147	55.4	31.3	40

shows the results obtained using a 15 cm buffer without data augmentation. When the buffer is used, an increase in TPs can be seen in both cases, which corresponds to a decrease in FNs. The F1 score for the model without a mask achieved an approximately 3% improvement compared to the model without a buffer. When using the mask, the model utilizing the buffer achieved an F1 score approximately 12% higher.

Table 8. Detection with data augmentation but without a 15 cm buffer.

Method	Limit (cm)	TP	FP	FN	Precision (%)	Recall (%)	F1 (%)
Faster R-CNN without mask	15	20	279	194	6.7	9.3	7.8
	30	72	227	142	24.1	33.6	28.1
	45	107	192	107	35.8	50	41.7
	60	121	178	93	40.5	56.5	47.2
Faster R-CNN with mask	15	17	219	197	7.2	7.9	7.6
	30	70	166	144	29.7	32.7	31.1
	45	101	135	113	42.8	47.2	44.9
	60	116	120	98	49.2	54.2	51.6

shows the results when using data augmentation. Regardless of whether a mask is used, the F1 score is higher when data augmentation is employed. In the case of using a mask, it is possible to see an increase of more than 20% in the F1 score (F1 score reaches more than 51%) compared to when not using data augmentation. When using data augmentation, it was possible to detect more than 50% of GT within 60 cm.

Table 9. Detection with data augmentation and with a 15 cm buffer.

Method	Limit (cm)	TP	FP	FN	Precision (%)	Recall (%)	F1 (%)
Faster R-CNN without mask	15	21	380	193	5.2	9.8	6.8
	30	79	322	135	19.7	36.9	25.7
	45	123	378	91	30.7	57.5	40
	60	136	265	78	33.9	63.6	44.2
Faster R-CNN with mask	15	18	256	196	6.6	8.4	7.4
	30	75	199	139	27.4	35	30.7
	45	114	160	100	41.6	53.3	46.7
	60	133	141	81	48.5	62.1	54.5

shows the results obtained using data augmentation and a buffer of 15 cm. In this case, more than 60% of GT were detected. For the model not utilizing a mask, the value of the F1 score is smaller than in Table 8, which is due to a larger number of FPs. Meanwhile, in the case of employing a mask, the model with data augmentation reached more than 54%, which is approximately 3% higher than when not using a buffer.

Figure 7 shows the output of people detection using the Faster R-CNN method. Red squares indicate real people (ground truth), while blue squares indicate people detected by the Faster R-CNN algorithm. The Faster R-CNN algorithm was able to detect people in this image with relatively high accuracy. It can be seen that in some cases the squares overlap completely and in others they overlap partially. Nevertheless, there is still room for improvement, especially in terms of the size of the area that the algorithm identifies as human.

Figure 8 documents the results of Table 7, where no mask was used. False positives occurred in areas where the surface appeared lighter due to specular reflection. Additionally, there were various transitions between surfaces where pixels could be misinterpreted as representing people. This issue was particularly evident on rooftops and sidewalks (Figure 8), where the transitions between different surface textures and lighting conditions led to pixel patterns resembling those of human figures, resulting in incorrect detections on the satellite images. This can also be seen in Table 9, where precisely such surface types have a higher density of predicted data than ground truth data (Locations 6 and 8).

Additionally, areas where individuals were in close proximity (Location 7 in

Table 10. The population density calculated for ground truth and predicted data per 1 m².

Location	Ground Truth	Predicted
6—Prague Castle—surface 1	0.014	0.037
7—Prague Castle—surface 2	0.034	0.027
8—Prague Castle—surface 3	0.009	0.014

) exhibited an increased incidence of false negatives. In these situations, the model struggled to accurately classify individuals, resulting in missed detections.

Figure 9 clearly demonstrates that the combination of data augmentation and a buffer with no mask resulted in the highest number of true positive people, with almost 64% detection. Data augmentation and buffers significantly increase the number of TPs, while masking has a less significant effect.

4. Discussion

The results in Table 6, Table 7, Table 8 and Table 9 show the success of detecting people from satellite images. The F1 score results are approximately 30% when the model is trained without the use of data augmentation. The utilization of a 15 cm buffer leads to better results in terms of the F1 score, and models trained on the dataset that employed data augmentation achieve an F1 score exceeding 50%.

Figure 10 shows people detection without using data augmentation compared with using data augmentation. It can also be seen from Figure 9 that data augmentation helped to detect more people. However, without the use of a mask, Faster R-CNN was able to detect more people than with the use of a mask.

Figure 11b shows Faster R-CNN detection using a 15 cm buffer and data augmentation. More people were detected with the buffer than without (Figure 9). With the mask, two additional people were falsely detected, but without the mask two people were missing. It can be seen from Figure 10b that, again, data augmentation improves the results of people detection, and that when using a buffer and data augmentation all the people in Figure 11b were detected with no FPs.

The maximum achieved F1 score was 54.5%, which is not satisfactory. Duporge and Isupova [20] also applied Faster R-CNN for processing WV3 and WV4 images, and they obtained an F2 score of 77.8% in heterogeneous areas and 73% in homogeneous areas. Their higher accuracy may be due to the larger pixel representation of an elephant than a human. Gaszczak, Breckon, and Han [39] achieved up to a 70% detection rate for people detection in thermal imagery based on a cascaded classification technique combining additional multivariate Gaussian shape matching. Sirmacek and Reinartz [1] used GeoEye-1 satellite images to detect people using an adaptive kernel density estimation method and estimated the corresponding probability density function; their results reached an approximately 75% detection success rate. Wang et al. [4] used Mask R-CNN to detect people from cameras in offices and shops and achieved an AP of 56–76%. Using a modified Faster R-CNN method, Y. Ren, Zhu, and Xiao [24] detected small objects, such as planes and ships; the AP for planes was approximately 83% and for ships it was 71%. Yan et al. [40] proposed a method for extracting tailing pond information from high-resolution images (Google Earth images) using Faster R-CNN, and their AP reached 80.1%. Testing models with other architectures (YOLOv3, SSD, and RetinaNet) using ArcGIS Pro software implementation did not achieve higher accuracy than Faster R-CNN.

5. Conclusions

The detection of people from satellite images remains a challenge. Faster R-CNN, YOLOv3, SSD, and RetinaNet in ArcGIS Pro implementation were tested for people detection. A validation method based on the calculation of distances between point-based locations of people (ground truth data) and gravity centers of polygons for predicted people was employed. Variants with buffers, masks, and data augmentation were analyzed. The most promising results were obtained with Faster R-CNN using a 15 cm buffer, no mask, the application of data augmentation, and a 60 cm validation distance, resulting in an F1 score of almost 55%. In the future, results may be improved with the implementation of selected textural features and other CNN models, using image super-resolution methods such as fast Fourier convolutions [41], or using deblurring techniques to increase image quality [42].