The Role of Environments and Sensing Strategies in Unmanned Aerial Vehicle Crowdsensing

Abstract

Crowdsensing has gained popularity across various domains such as urban transportation, environmental monitoring, and public safety. Unmanned aerial vehicle (UAV) crowdsensing is a novel approach that collects extensive data from targeted environments using UAVs equipped with built-in sensors. Unlike conventional methods that rely on fixed sensor networks or the mobility of humans, UAV crowdsensing offers high flexibility and scalability. With the rapid advancement of artificial intelligence techniques, UAV crowdsensing is becoming increasingly intelligent and autonomous. Previous studies on UAV crowdsensing have predominantly focused on algorithmic sensing strategies without considering the impact of different sensing environments. Thus, there is a research gap regarding the influence of environmental factors and sensing strategies in this field. To this end, we designed a $4\times 3$ empirical study, classifying sensing environments into four major categories: open, urban, natural, and indoor. We conducted experiments to understand how these environments influence three typical crowdsensing strategies: opportunistic, algorithmic, and collaborative. The statistical results reveal significant differences in both environments and sensing strategies. We found that an algorithmic strategy (machine-only) is suitable for open and natural environments, while a collaborative strategy (human and machine) is ideal for urban and indoor environments. This study has crucial implications for adopting appropriate sensing strategies for different environments of UAV crowdsensing tasks.

1. Introduction

As technology continues to advance, crowdsensing is becoming increasingly pervasive across various industries and sectors [1,2]. Crowdsensing refers to the process of utilizing a large number of individuals, such as smartphone users and sensor devices, to collectively complete data collection and information perception [3,4]. Conventional crowdsensing aims to gather data from a diverse workforce by leveraging collective intelligence. It involves participants, primarily humans, employing their smart mobile devices for data collection, computation, and analysis [5]. With the rapid advancement of technology, artificial intelligence (AI) is gaining significant power, robotics are progressing at an accelerated pace, and humans are increasingly averse to engaging in repetitive and monotonous tasks. This development brings about a profound transformation in crowdsensing. Consequently, the majority of sensing activities can now be undertaken by robots and unmanned vehicles. In particular, crowdsensing based on unmanned aerial vehicles (UAVs) has started to play a crucial role due to its exceptional flexibility and mobility. The focus of this study lies in UAV crowdsensing. We posit that, within the era of AI, human workers may transition into roles as high-level operators who issue directives to AI systems and delegate remaining tasks to AI and robots. However, there is one thing that always exists—the crowdsensing environment, where UAV crowdsensing campaigns take place.

Researchers in the field of crowdsensing have primarily focused on issues such as task assignments, incentive mechanisms [6], privacy protection [7], data quality [8], and other related topics [9,10]; however, they have often overlooked or oversimplified the impact of environmental factors [11,12]. Nevertheless, it is important to acknowledge that different environments may exert varying effects on crowdsensing performance. In addition, whether in the past or present, most crowdsensing-related studies have remained largely theoretical, and practical crowdsensing applications are still relatively rare [4]. The primary challenge in the practical application of crowdsensing may be diverse and often unpredictable environments [13]. Traditionally, crowdsensing campaigns easily take place in urban environments, where it is feasible to find a sufficient number of human participants [14]. There may not be such favorable innate conditions for crowdsensing in other environments. Nowadays, the development of UAVs has led to a shift in the crowdsensing paradigm, which previously relied on humans or fixed sensor networks. Crowdsensing is convenient to put into effect because of the widespread use of UAVs. As long as UAVs can reach and explore the area, UAVs can be easily controlled and effectively utilized to collect data in various environments.

In addition to the environment, sensing strategies can also affect crowdsensing activities. There are multiple sensing strategies, such as opportunistic and algorithmic [15,16,17,18,19,20,21,22]. Different strategies perform variably across environments [23]. However, there is a lack of detailed research on their different roles in UAV crowdsensing [24]. Since the influence of environment and sensing strategies in crowdsensing has remained unexplored, we know little about their impacts on the efficiency and effectiveness of crowdsensing task completion. We, thereby, delve into this exploration through our work. In this paper, we address the following research question: how do environments and sensing strategies affect crowdsensing efficiency and effectiveness?

To address the knowledge gap, we first analyzed and classified crowdsensing environments into four basic categories: open, urban, indoor, and natural, to represent most crowdsensing scenarios to the best of our knowledge. Then, we summarized three typical crowdsensing strategies from the literature, which were opportunistic [15,16,17], algorithmic [18,19,20,21], and collaborative [22]. Next, we designed a study incorporating these four types of environments and conducted experiments to understand the role of environments when applying three different strategies. The experimental task was to use a swarm of UAVs (up to 100 UAVs) to explore an unknown environment and achieve maximum coverage. All experiments were performed on a prototype system. Particularly, we recruited 18 participants to complete crowdsensing campaigns based on a collaborative strategy. The prototype system recorded participants’ clicks to the crowdsensing campaigns.

Additionally, we measured the performance of the crowdsensing campaign in terms of effectiveness, efficiency, and human intervention. The statistical analysis revealed significant differences based on the environment and sensing strategy employed, highlighting their substantial impact on crowdsensing performance. For both efficiency and effectiveness, we argue that an algorithmic strategy could be a suitable option in open and natural environments. In contrast, a collaborative strategy is best suited for urban and indoor environments if the cost is not taken into account. This work has important implications in terms of improving the effectiveness and efficiency of crowdsensing tasks by providing insights in terms of choosing an appropriate sensing strategy.

The remainder of the paper is organized as follows. Section 2 reviews the related literature. Section 3 focuses on the study design, encompassing the environment classification, sensing strategies design, introduction to the experimental procedure, and definition of evaluation metrics. Section 4 displays and analyzes the relevant experimental results, which is followed by discussions identifying potential future research avenues in Section 5. Finally, Section 6 concludes the presented study.

2. Related Work

We introduce the related literature from five aspects: crowdsensing environments, crowdsensing strategies, crowdsensing applications, crowdsensing challenges, and data quality.

2.1. Crowdsensing Environments

Crowdsensing plays an important role in different environments. In this part, different crowdsensing activities in various environments are introduced.

The majority of crowdsensing campaigns take place in urban environments due to the good foundation and high demand for crowdsensing in the city. People with smart devices, the vehicles regularly moving around city streets, taxis, buses, and all forms of transportation can be crowdsensing nodes [25,26]. Fan et al. [27] discussed the importance of achieving fine-grained spatiotemporal sensing coverage in vehicular urban sensing by utilizing a hybrid approach. This approach combines for-hire with dedicated sensing vehicles. The document [28] presented a novel approach to reuse delivery drones for urban crowdsensing. Alvear et al. [29] discussed the emergence of smart cities and the important role of crowdsensing in addressing environmental monitoring, particularly air pollution levels in urban areas. GasMobile [30], CommonSense [31], Third-Eye [32], AirSense [16], and 3M’Air [33] are examples of urban air pollution monitoring systems using only off-the-shelf sensors available on citizens’ smart devices.

Crowdsensing systems are also widely used to provide location services in indoor environments. Lashkari et al. [34] investigated the degree of user contribution in existing crowd-powered techniques. They presented a classification of crowd-powered indoor localization solutions to delineate the specific crowdsourcing approaches employed in each indoor localization solution. A prior article [35] introduced a novel WiFi domain adversarial graph convolutional network model for indoor localization. This model can be trained with a small quantity of labeled site survey data and extensive unlabeled crowdsensing WiFi fingerprints.

In the natural environment, crowdsensing plays an important role in wildlife tracking, natural disaster monitoring, and ecological protection. The authors in [36] showed how crowdsensing technologies were used in biodiversity conservation, covering monitoring of wildlife and habitats, conservation management, and fighting illegal activities. João et al. [37] discussed the development of a mobile-based citizen science project for forest fire prevention and combat. By utilizing UAVs equipped with aerial imaging technology, the monitoring system can provide real-time and remote monitoring of forest fires, particularly in challenging terrains like forests and mountains [38].

Table 1. Crowdsensing in different environments.

Literature	Environment	Application Scenario
[27]	Urban	Vehicular urban sensing
[28]	Urban	Reuse delivery drones
[29]	Urban	Environmental monitoring
[16,30,31,32]	Urban	Air pollution monitoring systems
[34,35]	Indoor	Indoor localization
[36]	Natural	Biodiversity conservation
[37]	Natural	Monitoring forest fires
This paper	Urban, indoor, natural	Various application scenarios

summarizes the environments in which crowdsensing appears in the above literature. Crowdsensing has been used in various environments. However, these studies do not refer to the role of environments in crowdsensing tasks.

2.2. Crowdsensing Strategies

Typical crowdsensing strategies include opportunistic strategies and algorithmic strategies.

Opportunistic crowdsensing refers to a method that utilizes various sensors (such as GPS, microphones, accelerometers, etc.) carried within mobile devices (such as smartphones and wearable devices) to collect environmental data. Opportunistic crowdsensing leverages users’ opportunistic capture of sensor data to overcome data quality limitations [15]. This approach is related to the larger idea of opportunistic sensing, which aims to collectively leverage sensors to sense the target within the preferred time frame [17]. In [15], the authors solved the problem of automatically determining a location’s category using sensor data from smartphones through the opportunistic crowdsensing method. Dutta et al. [16] presented the AirSense system, an opportunistic crowdsensing-based air quality monitoring system aimed at collecting and aggregating sensor data to monitor air pollution in both indoor and outdoor environments.

With the continuous development of intelligent vehicle technology, unmanned aerial vehicles (UAVs), unmanned marine vehicles (UMVs), and unmanned ground vehicles (UGVs) are becoming smarter and more affordable. This trend is driving the widespread application of intelligent vehicles in various fields. Intelligence vehicles can autonomously execute tasks under the guidance of an algorithm. A prior work described the use of two versions of flood-fill algorithms, basic and modified, for autonomous mobile robot maze navigation using ArduinoUNO cards and ultrasonic sensors [18]. Deepak et al. [19] focused on developing an efficient path planner for an autonomous mobile robot using particle swarm optimization (PSO). The authors in [20] introduced an approach based on ant colony optimization (ACO) for solving the minimum time search (MTS) problem using a fleet of UAVs searching for a target in uncertain domains. Husain et al. [21] explored the application of bio-inspired techniques, specifically ACO and Dijkstra’s algorithm, in optimizing autonomous agents’ search and rescue efforts in critical scenarios, such as disasters in unknown environments. Traditional methods based on decision rules were limited in covering diverse environments and sensor properties. Therefore, a deep reinforcement learning-based decision algorithm was introduced, utilizing a deep neural network to learn exploration strategies from partial maps [39].

Opportunistic and algorithmic strategies do not apply to all kinds of situations in reality. The work landscape is evolving towards greater knowledge intensity and intricacy, surpassing the capacities of individual humans [40]. Concurrently, AI, especially machine learning, has made significant advancements in recent years, sometimes surpassing human capabilities. Nevertheless, these AI systems are constrained to precise, narrowly outlined tasks, falling short of achieving a widespread level of intelligence [41]. Human–AI collaboration refers to the collaboration and interaction between humans and AI systems in various processes, particularly machine learning applications. This collaboration involves combining humans with the computational power of AI methods to enhance decision-making and problem-solving capabilities [42,43]. The strengths and weaknesses of humans and AI systems are complementary, leading to a mutually beneficial relationship when combined, allowing the team to collectively accomplish more than what would be individually achievable [44]. Therefore, a collaborative strategy (mainly human–AI collaboration) may have a better effect on solving complex tasks in comparison to the other two strategies.

Relevant crowdsensing campaigns can be organized using a three-layer architecture. Organizersare participants who plan the campaigns, make decisions, and control the direction. In novel vehicular crowdsensing campaigns, organizers will mostly be biological participants—in other words, humans will play these important roles here [45]. Assistants are participants who design specific strategies, assign tasks, predict progress, and aggregate/evaluate outcomes. Assistants will mostly be digital, most probably artificial intelligence. Assistants are driven by advanced AI models and knowledge bases, to understand human needs and produce corresponding responses/assistance. Workers are participants who receive tasks, execute tasks, and provide raw task outcomes. Workers will mostly be robotic, as robots will do the dirty work. Due to the exploratory nature of crowdsensing campaigns, these robotic workers are mostly intelligent vehicles capable of moving, such as UAVs, UGVs, or UMVs. Some of the literature also reflects this architecture. For example, Zhao et al. [22] introduced a crowd-powered source search approach that combines human intelligence with AI algorithms to improve the agents’ source searching effectiveness and efficiency.

Table 2. Sensing strategies for crowdsensing.

Literature	Sensing Strategy	Application Scenario
[15]	Opportunistic	Determining a location’s category
[16]	Opportunistic	Air quality monitoring
[18]	Algorithmic	Mobile robot maze navigation
[21]	Algorithmic	Autonomous search and rescue
[22]	Collaborative	Source searching
This paper	Opportunistic, algorithmic, collaborative	Various application scenarios

shows that diverse sensing strategies are employed for a variety of crowdsensing tasks in the above-mentioned literature. However, there is a gap in research when it comes to studying how environments affect sensing strategies.

2.3. Crowdsensing Applications

There are many crowdsensing applications. This section will provide a concise overview of existing crowdsensing applications and describe their key characteristics. These applications connect devices, share data, and extract information to map common phenomena, which generally fall into four categories: healthcare, environmental, infrastructure, and social [46].

In healthcare applications, smart wearable devices and portable devices (smartphones) can connect patients with medical services through mobile communication networks to achieve sensing and diagnostic functions. The authors in [47] mentioned the TrackYourTinnitus mobile crowdsensing mHealth platform, developed to gather data on tinnitus, a chronic disorder with substantial economic and patient-related burdens. In environmental applications, the main phenomena to be sensed are air quality, forest fires, etc. Alvear et al. [29] discussed the growing importance of crowdsensing in smart cities for environmental monitoring, particularly air pollution levels in urban areas. Numerous urban air pollution monitoring systems, such as GasMobile [30], CommonSensee [31], Third-Eye [32], AirSense [16], and 3M’Air [33], have emerged, utilizing only off-the-shelf sensors available on citizens’ smart devices. João et al. [37] developed a mobile-based citizen science project for forest fire prevention and combat. The system employs UAVs with aerial imaging technology to provide real-time and remote monitoring of forest fires, especially in challenging terrains like forests and mountains [38]. Infrastructure encompasses various systems and facilities that are essential for the functioning of a community, city, or country. A crowdsourcing-based model for evaluating and indexing road surface quality was proposed in the literature [48]. The model utilizes the power spectral density analysis of vertical acceleration data from smartphone sensors in vehicles to measure road roughness and apply blind source separation to eliminate the effects of vehicle parameters.

2.4. Crowdsensing Challenges and Limitations

Crowdsensing, a novel and widespread paradigm in the digital age, presents exciting opportunities and faces several challenges and limitations that must be addressed for it to reach its full potential. One of the primary challenges is the issue of data quality and reliability [6]. As individual users contribute data from their personal devices, there can be significant variations in the accuracy, precision, and timeliness of the information provided. This can lead to inconsistencies and inaccuracies in the aggregated data, undermining the overall value and usefulness of the crowdsensing insights. Another limitation is the uneven distribution of participants, which can result in biased or skewed data. Certain demographics or geographic regions may be underrepresented, leading to blind spots in the data and potentially inaccurate conclusions. Addressing these gaps in participation requires innovative strategies to engage a more diverse and representative set of contributors. The authors in [49] proposed an incentive mechanism based on behavioral economics (BE) and aimed to address the uneven participation in location-based crowdsensing (LCS) systems. The risk of compromising citizens’ privacy could lower their willingness to participate in the crowdsensing campaign [25].

2.5. Data Quality

Data quality and validation are essential for data management, ensuring accurate, complete, and reliable information. This is also critical in crowdsensing. Establishing robust data quality standards and implementing rigorous validation processes are crucial to prevent errors, inconsistencies, and anomalies in datasets. Effective data validation can identify and address issues such as missing values, outliers, data type mismatches, and violations of business rules or constraints. Proactively addressing data quality concerns enhances the trustworthiness of data, improves decision-making, and unlocks the full potential of information assets. Samulowska et al. [8] addressed the problem of detecting and removing crowdsourced data bias in air pollution symptom mapping, a prerequisite for better-quality open-data output. Luo et al. [50] proposed a cross-validation approach to address data quality issues in mobile crowd sensing and the Internet of Things. The key idea was to use a validating crowd to ratify data contributed by the original crowd and reshape the data into a more credible representation of the ground truth.

3. Study Design

To understand the role of crowdsensing environments and sensing strategies, we designed a study, classifying environments into four common categories, to represent as many crowdsensing scenarios as possible to the best of our knowledge.

Moreover, we used a prototype system with graphical user interfaces (GUI) to conduct simulation experiments to complete some tasks. The tasks involved using a swarm of UAVs (up to 100 UAVs) to explore an unknown environment and maximize coverage. And, every task had restrictions on the number of steps. In the experiment, it was always assumed that the UAVs flew on a two-dimensional plane. Then, we chose three typical strategies to explore environments, namely, opportunistic, algorithmic, and collaborative strategies. Then, three different sensing strategies were used to drive UAV crowdsensing campaigns to discover the impact of environments and sensing strategies on crowdsensing.

Please note that in this empirical study, we did not choose state-of-the-art sensing strategies and extremely realistic environments in the experiment on purpose. Instead, the chosen environmental scenes and sensing strategies were simplified to a large extent, in order to acquire common and representative findings through the experiment.

3.1. Environment Classification

We classified crowdsensing environments into four categories and designed seven experimental scenes representing the situations that participants might encounter. Figure 1 provides an overview of four environmental categories and all the designed scenes for the experiments. The black blocks in the right half of Figure 1 represent obstacles, and the white parts represent where the UAVs could go through.

According to whether it is a man-made environment, an environment can be categorized as an artificial environment or a non-artificial environment. For non-artificial environments, open and natural environments are two typical environments [51]. For artificial environments, they can be divided into indoor and outdoor environments. Outdoor environments mainly refer to urban environments [52,53,54].

• The open environment refers to places without obvious physical restrictions or obstacles, such as parks, grasslands, wild areas, etc. These places usually have a wide view and space. The UAVs can fly freely without avoiding obstacles;
• The natural environment refers to places with natural obstacles such as forests, hills, and mountains, which are not obviously disturbed by human beings. The UAVs can only move in open spaces due to the obstacles;
• The urban environment refers to places with artificial barriers like city blocks and urban parks. Many tall buildings exceed the maximum flight altitude of UAVs. Therefore, UAVs need to avoid building or other obstacles actively;
• The indoor environment refers to the space consisting of multiple rooms with limited space in buildings. Due to the existence of indoor ceilings and walls, UAVs need to avoid these indoor obstacles actively.

3.2. Sensing Strategies

In this section, we will introduce three sensing strategies (opportunistic, algorithmic, and collaborative) in detail. An opportunistic strategy is the most basic and simple strategy. This method allows individual users to collect data on their daily activities without specific data collection devices or tasks, always featuring randomness [55]. An algorithmic strategy (machine-only) is used to complete specific tasks, driven by an algorithm. A collaborative strategy (human and machine) involves incorporating human wisdom into the algorithm. These three strategies develop from simple to complex, as do most of the situations in reality.

Opportunistic crowdsensing is a method of collecting environmental data using various sensors (such as GPS, an accelerometer, a microphone, etc.) carried in mobile devices (such as smartphones and wearable devices). We used a random strategy to simulate opportunistic crowdsensing. The random strategy implies that the likelihood of each step moving in each direction is equal.

There are many algorithms [18,19,20,21,56,57]; we used inverse ant colony optimization (IACO) because it is always used to achieve maximum coverage, aligning with our goal, and it is simple and typical enough to represent many similar intelligent algorithms. In IACO crowdsensing, IACO is deployed in swarm UAVs to make decisions during crowdsensing. When a UAV passes a grid, it produces a pheromone that passes the information that the grid has been explored. UAVs are inclined to move in the direction of low pheromone concentration. At step t, the probability density function (1) is applied to determine the direction of the next step:

$$P_i(t+1)={\displaystyle \frac{{(c+n_i\left(t\right))}^{\alpha }}{{\sum }_{j=1}^4{(c+n_j\left(t\right))}^{\alpha }}}$$

(1)

where $P_i(t+1)$ is the probability of moving in direction i, and $n_i\left(t\right)$ is the quantity of pheromone in grid i. The summation (j) in Equation (1) refers to the four possible directions of movement. Additionally, c is the degree of attraction to an unexplored path, and $\alpha$ is the bias in using a pheromone-concentrated path [58]. IACO parameter values were c = 20, and $\alpha =-2$, as shown in [59].

The roles in the crowdsensing campaign employing a collaborative strategy [22,60] can be categorized into three distinct parts: organizers, assistants, and workers. In this study, human operators served as organizers overseeing the entire campaign. Organizers possessed the capability to intervene in the autonomous movements of UAVs and provided guidance on crowdsensing directions by interacting with the map interface. Assistants refer to AI or algorithms that serve as fundamental tools aiding intelligent UAV swarms in exploring environments. In this study, inverse ant colony optimization (IACO) played the role of assistant. Workers were individual UAVs that could move autonomously under the guidance of IACO. When human organizers wanted to guide UAVs to the proper location, the A* algorithm [61] was used to find the best path between the current location of the UAV and the human-designated destination. In many cases, the A* algorithm was able to quickly find solutions and flexibly adapt to different problems and scenarios.

3.3. Experimental Setup

This section provides a detailed description of the experimental setup and measurements. The experiments on multi-UAV cooperative crowdsensing were conducted on a prototype system. Possible movement directions of intelligent UAVs mainly include forward, backward, left, and right in the prototype system. The prototype system utilizes a GUI as an intermediary between UAVs and human operators for collaborative crowdsensing. It visually displays the UAVs’ current status and alerts operators to any detected issues. Additionally, operators can interact with the exploring UAVs through this interface. The graphical interface was created using tkinter, a standard GUI toolkit for Python that offers essential components for desktop application development, as shown in Figure 2. A legend module in the bottom-right corner briefly explains the graphical elements to users. We outlined the crowdsensing task and communicated issues to the operator by displaying a map, UAV positions, and pheromone concentrations in grid cells. Additionally, we showed a curve illustrating the increasing “coverage rate” over time to enhance the operator’s understanding of task progress. The operator could interact with the UAVs by clicking on the map, and the interface translated their guidance into commands for the UAVs. For algorithmic crowdsensing, we only needed to remove human intervention, just relying on the algorithmic strategy in the prototype system. For opportunistic crowdsensing, we just relied on the opportunistic strategy without human involvement when executing the crowdsensing task. The experimental environment (mainly the prototype system) was deployed on a Windows 10 laptop with an AMD R7-4800U processor and 16 GB of RAM. The laptop, manufactured by Lenovo, was sourced from China.

The experimental process is shown in Figure 3. Opportunistic crowdsensing and algorithmic (IACO) crowdsensing ran independently 10 times in each scene. Furthermore, we recruited 18 participants and carried out 126 human–UAV interaction crowdsensing campaigns. All participants needed to guide UAVs to complete the corresponding crowdsensing tasks under the seven scenes. Task completion time, coverage rate, and mouse click data were recorded during the experiments. The collected data were subsequently analyzed to assess the performance of each crowdsensing method. Statistical methods were employed to compare task completion times and coverage rates among three crowdsensing strategies. Additionally, the mouse click data provided insights into organizer interactions and decision-making processes during collaborative tasks.

The opportunistic strategy has an equal probability of moving in four directions, incorporating randomness. The algorithmic strategy based on IACO determines the movement direction using Equation (1), also with an element of randomness. Human behavior is inherently difficult to predict [62]. The collaborative strategy is not only based on IACO but also embraces human behavior, introducing an element of randomness. Because of the randomness of each run, experimental data may vary even within the same environment and strategy. Repeated experiments are one of the important methods to overcome randomness in scientific research; they help to confirm whether a certain observation or effect exists universally, rather than just representing an accidental result. Due to the stochastic nature of the three sensing strategies in this paper, we also conducted statistical hypothetical tests using environmental categories based on experimental data to deeply understand the influence of the environment on sensing activities. The null hypothesis is that two sets of experimental data are drawn from the same distribution. The alternative hypothesis is that two sets of experimental data are drawn from different distributions. Before testing for statistical significance, we performed Shapiro–Wilk normality tests for each pair of experimental data. We used t-tests for pairs that were normally distributed, and the Mann–Whitney U test for pairs that were not. The significance level was set to 0.05. Moreover, we aimed to visualize the results through comprehensive graphs and charts. By plotting the experimental outcomes against environmental categories, we sought to illustrate the distribution of performance metrics—such as effectiveness and efficiency. These visual tools aided in presenting our findings clearly, making it easier to identify trends and anomalies that warranted further investigation. The codes involved in the experiments were implemented in Python (the source code is available at https://github.com/summerqinye/role (accessed on 13 September 2024)).

3.4. Evaluation Metrics

We evaluated the role of environments and sensing strategies in UAV crowdsensing campaigns in three aspects, that is, efficiency, effectiveness, and human intervention. Effectiveness consists of two evaluation metrics in this study, i.e., coverage rate and success rate.

• Measure 1. The coverage rate [63,64] is defined as the ratio of the number of explored grids ($\#cell{s}^{exp}$) to the total number of all accessible grids ($\#cell{s}^{all}$), as shown in Equation (2). A greater coverage rate indicates more thorough scene exploration;
• Measure 2. According to our experimental results, all scenes usually take similar execution times to consume all steps. Based on this, we define the methods’ coverage speed as the coverage rate ratio to the task completion time [65], as shown in Equation (3). The larger coverage speed indicates better performance;
• Measure 3. Our study considers a crowdsensing task successful if the UAVs explore at least 99% of the open area (excluding obstacles) within a maximum number of allowed crowdsensing steps. The success rate [66,67] is calculated by dividing the number of successful tasks ($\#task{s}^{suc}$) by the total number of tasks ($\#task{s}^{all}$), as shown in Equation (4);
• Measure 4. An additional metric for evaluating the collaborative strategy is human intervention frequency [68,69], which is defined as the ratio of the total number of clicks ($\#clicks$) to the task completion time (completion time), as shown in Equation (5). The frequency of human intervention aligns closely with the overall operational cost of the UAV crowdsensing campaigns. A lower intervention frequency suggests reduced human effort needed to complete a crowdsensing task successfully. A higher frequency suggests a greater dependence on human oversight, which can introduce more cost. Therefore, minimizing the human intervention frequency should be a goal, aiming to develop more autonomous systems capable of executing complex crowdsensing tasks with minimal guidance.

$$coveragerate={\displaystyle \frac{\#cell{s}^{exp}}{\#cell{s}^{all}}}$$

(2)

$$coveragespeed={\displaystyle \frac{coveragerate}{completiontime}}$$

(3)

$$successrate={\displaystyle \frac{\#task{s}^{suc}}{\#task{s}^{all}}}$$

(4)

$$interventionfrequency={\displaystyle \frac{\#clicks}{completiontime}}$$

(5)

4. Results

To figure out the role of different strategies and environments in UAV crowdsensing, we show the experimental results in this section. A total of 266 data entries were used for the experimental data analysis. We pay close attention to the effectiveness (coverage rate and success rate), coverage speed, and human intervention frequency. We use two-way ANOVA tests to show the main and interaction effects of the sensing strategies and environments considered in this study. The results of the statistical tests are shown in

Table 3. Results of two-way ANOVA ($\alpha =0.05$, Type I) for the efficiency and effectiveness.

Metrics	Factor	Degree of Freedom	F-Value	Pr (>F)
Efficiency (coverage speed)	Environment (E)	3	76.360	3.1 × 10−35
	Strategy (S)	2	1505.350	1.4 × 10−141
	E × S	6	48.323	2.4 × 10−39
Effectiveness (coverage rate)	Environment (E)	3	96.008	1.5 × 10−41
	Strategy (S)	2	201.346	4.1 × 10−53
	E × S	6	6.389	2.8 × 10−6

. We found that the efficiency and effectiveness of environment crowdsensing show a significant difference in terms of the environment (p = 3.1 × 10⁻³⁵ for efficiency, p = 1.5 × 10⁻⁴¹ for effectiveness), sensing strategy (p = 1.4 × 10⁻¹⁴¹ for efficiency, p = 4.1 × 10⁻⁵³ for effectiveness), and their interaction effect (p = 2.4 × 10⁻³⁹ for efficiency, p = 2.8 × 10⁻⁶ for effectiveness), meaning that three factors have significant impacts on crowdsensing performance. The interaction effect in a two-way ANOVA represents the mutual influence between the sensing strategy and environment. When explaining the influence of independent variables on dependent variables, two independent variables and their interaction must be considered at the same time. Therefore, in the subsequent analysis, we analyze the performance of different strategies in the same environment. A comparative analysis often leads to the generation of valuable insights. Then, a more detailed analysis can be conducted according to different evaluation metrics.

4.1. Efficiency

In this part, we further analyze the difference in the efficiency of the sensing strategies across different environments.

The distributions of the coverage speed for different sensing strategies in various environments are shown in Figure 4 using boxplots. It is apparent from the figure that the coverage efficiency of the opportunistic strategy is far behind those of the other strategies for the four environments. However, the difference between the algorithmic and collaborative strategy is not that big. Additionally, the results suggest that the collaborative strategy demonstrates an edge over the algorithmic strategy in terms of coverage speed.

In addition to the boxplots, we drew an estimation plot [70] (Figure 5) to provide a more detailed comparison across various environments. It is worth noting that each data point depicted in the figures represents the result of an individual crowdsensing task. The estimation plot illustrates the efficiency discrepancies between the algorithmic strategy and the other two strategies across various environments, as depicted in Figure 5. Each subplot corresponds to a specific environment and exhibits the estimated mean value with error bars signifying the standard deviation. The x-axis denotes the various strategies, and the y-axis illustrates the value for each environment. From Figure 5, we can observe minor differences in the coverage speed between the algorithmic strategy and the collaborative strategy in the open and natural (non-artificial) environment. However, the differences in coverage speed between the two strategies are obvious in indoor environments, and the mean value for the collaborative strategy is significantly higher than that of the algorithmic strategy. The performance of the opportunistic strategy in coverage speed lags behind that of the algorithmic strategy in the four environments, particularly in environments characterized by increased complexity.

The significance test was used for the further analysis. According to normality tests, some data do not follow a normal distribution. So, we needed to conduct different statistical tests depending on the diverse situations to find differences in the efficiency. An independent t-test (two-tailed, $\alpha =0.05$) is appropriate when analyzing two data groups with normal distributions. In cases where the data fail to meet the assumption of equal variances, Welch’s t-test is the preferred method. If two data groups do not follow normal distributions, it is reasonable to use Mann–Whitney U tests. The Bonferroni correction was utilized to control the Type-I error due to multiple comparisons (various sensing strategies) carried out among four datasets (four environments).

Table 4. Efficiency ($\mu \pm \sigma$: mean and standard deviation; unit: percentage increase per second) and p-values across different environments and sense strategies.

Coverage Speed (%/s)	Opportunistic (vs. Algorithmic, Collaborative)	Algorithmic (vs. Collaborative)	Collaborative
Open	$0.428\pm 0.058$ ($p=$ 9.9 × 10−10, 1.8 × 10−5)	$1.300\pm 0.142$ ($p=0.46$)	$1.252\pm 0.218$
Urban	$0.096\pm 0.015$ ($p=$6.8 × 10−8, 7.9 × 10−10)	$0.447\pm 0.040$ ($p=$ 0.10)	$0.525\pm 0.138$
Indoor	$0.248\pm 0.063$ ($p=$ 1.8 × 10−22, 7.9 × 10−10)	$0.557\pm 0.125$ ($p=$ 2.9 × 10−6)	$0.812\pm 0.198$
Natural	$0.239\pm 0.101$ ($p=$ 6.8 × 10−8, 7.9 × 10−8)	$0.739\pm 0.275$ ($p=$ 0.20)	$0.869\pm 0.353$

shows not only the mean and standard deviation of the coverage speed across the 12 experimental conditions (four environments × three sensing strategies) but also the significance test results.

Related statistical tests revealed that there is no significant difference between algorithmic strategy and collaborative strategy in open, urban, and natural environments (corresponding p-values are 0.46, 0.10, and 0.20, which indicates that we cannot reject the null hypothesis). So, we can use an algorithmic strategy in open, urban, and natural environments when only taking efficiency into account, which is beneficial for reducing human participation. In indoor environments, a collaborative strategy is better.

4.2. Effectiveness

We evaluated the effectiveness of scene crowdsensing from two perspectives, namely, the success rate and the coverage rate. Figure 6 shows the coverage rate of different sensing strategies across four environments (open, urban, indoor, and natural) in the form of boxplots. It is apparent from Figure 6 that a collaborative strategy achieved a good performance in the coverage rate in the four environments. And, there is little difference between the algorithmic strategy and the collaborative strategy in open environments.

The estimation plot displays an experimental dataset’s complete statistical information regarding the coverage rate, as shown in Figure 7. Jitter plots display the experiment results for the coverage rate across different conditions. Here, we used the baseline condition—the algorithmic strategy—as a control group in the plots to make a comparison with all the other experimental conditions. The estimation plots also show the resampling distribution of the difference in means, representing the effect size (the effect size is zero, indicating that both groups have identical mean data). The results clearly show that the opportunistic strategy is at a distinct disadvantage compared to the other two strategies in the four environments. Both algorithmic and collaborative strategies achieved a high coverage rate in the open and natural environment, with minor differences in the coverage rate between the two strategies in the open and natural (non-artificial) environment. The collaborative strategy performed better than the algorithmic strategy in indoor and urban (artificial) environments.

More in-depth information regarding the coverage and success rate can be found in

Table 5. Effectiveness–coverage rate ($\mu \pm \sigma$: mean and standard deviation; unit: %) and p-values across different environments and sense strategies.

Coverage Rate (%)	Opportunistic (vs. Algorithmic, Collaborative)	Algorithmic (vs. Collaborative)	Collaborative
Open	$76.8\pm 10.4$ ($p=$ 1.5 × 10−4, 1.4 × 10−5)	$99.2\pm 0.1$ ($p=$ 0.03)	$99.3\pm 0.2$
Urban	$17.5\pm 2.5$ ($p=$ 8.4 × 10−26, 7.3 × 10−10)	$81.6\pm 5.9$ ($p=$ 3.6 × 10−9)	$97.7\pm 3.6$
Indoor	$44.4\pm 11.4$ ($p=$ 6.8 × 10−8, 7.1 × 10−10)	$91.8\pm 7.3$ ($p=$ 1.3 × 10−5)	$98.9\pm 1.2$
Natural	$43.0\pm 18.1$ ($p=$ 6.4 × 10−8, 6.3 × 10−10)	$95.8\pm 4.9$ ($p=$ 0.18)	$98.1\pm 2.3$

and

Table 6. Effectiveness–success rate (unit: %) across different environments and sense strategies.

	Opportunistic	Algorithmic	Collaborative
Open	0	100	100
Urban	0	0	81
Indoor	0	25	92
Natural	0	50	78

, respectively. The statistical test procedures for the coverage rate were similar to tests for the coverage speed. As can be seen from Table 5, relevant tests (expected $\alpha =0.05$, two-tailed, corrected by the Bonferroni method) between the algorithmic and collaborative strategies show that the coverage rate has no significant difference in open and natural environments (corresponding p-values are 0.03, 0.18, and two values are less than the Bonferroni-adjusted alpha level of 0.017 (0.05/3), which indicates that we cannot reject the null hypothesis). In other situations, we found a statistically significant difference.

The opportunistic strategy was tested 10 times in all scenes. However, it failed to successfully complete any tasks, as shown in Table 6. While the cost of implementing an opportunistic strategy is fairly low and the execution process is straightforward, the successful completion of tasks related to opportunistic crowdsensing, particularly within the constraints of limited steps and the number of UAVs, presents a considerable challenge for the strategy. The algorithmic strategy achieved the same success rate as the collaborative strategy in open environments. In other environments, there is a big gap between the two strategies. This may be because we defined a successful task strictly (a coverage rate of over 0.99). It can be seen in Table 5 that the gap in the coverage rate is not as big as the gap in the success rate between the two sensing strategies. When pursuing the success rate of the task, an algorithmic strategy is suitable for open environments and a collaborative strategy is a preferred choice in other environments. When pursuing the coverage rate of the task, an algorithmic strategy is suitable for open and natural (non-artificial) environments and a collaborative strategy is a preferred choice in indoor and urban (artificial) environments.

4.3. Human Intervention Frequency

The results pertaining to the overall human intervention frequency are shown in Figure 8. As we can see from the figure, the mean value of intervention frequency in open environments is lower mean values than in other environments. This may be because the open environment is simple and does not require much human involvement.

Table 7. Intervention frequency in collaborative crowdsensing by environmental categories ($\mu \pm \sigma$: mean and standard deviation; unit: times per second, $\#clicks/s$).

Measure	Environmental Categories
	Open	Urban	Indoor	Natural
Human Interventions (intervention frequency)	$0.118\pm 0.132$	$0.369\pm 0.542$	$0.225\pm 0.232$	$0.197\pm 0.211$

shows more detailed results about the human intervention frequency. From the table below, we can see that the mean value and standard deviation are both minimal in the open environments.

The estimation plot displays an experimental dataset’s complete statistical information about the human intervention frequency, as shown in Figure 9. Jitter plots show the overall human intervention frequencies, and how they distribute, across experimental conditions in Figure 9. From the figure, it is clear that the human intervention frequency distribution in urban environments differs significantly from that in open environments, with a considerable mean difference in the intervention frequency between the two environments. This is possible because urban environments are more intricate than open environments and often necessitate increased human intervention.

In addition to boxplots and estimation plots, the significance test was used for additional analyses. According to the normality tests, the human intervention frequency distributions did not come from a normal distribution, so we applied pairwise Mann–Whitney U tests to carry out the significance tests ($\alpha$ values were adjusted by Bonferroni corrections, $0.05/6=0.008$). The results obtained from the significance test are summarized in

Table 8. Significance test results (p-value) for the human intervention frequency in collaborative crowdsensing. ** in the table indicates statistical significance with Bonferroni correction, and * indicates a p-value < 0.05 that was not significant after Bonferroni correction.

	Open-Urban	Open-Indoor	Open-Natural	Urban-Indoor	Urban-Natural	Indoor-Natural
Interventions Frequency	0.001 **	0.022 *	0.152	0.154	0.036 *	0.351

. From the table below, we can see there is a significant difference in the intervention frequency between open and urban environments. The difference in the complexity of the environments may be an important factor contributing to the statistically significant difference.

5. Discussion

Based on the results presented in Section 4, it is clear that the environment category and sensing strategy both have a significant impact on the performance of UAV crowdsensing. In this section, we provide valuable insights into the experimental results.

5.1. The Role of Environments and Sensing Strategies

Different characteristics of environments underscore the importance of selecting an appropriate sensing strategy to optimize crowdsensing performance across various scenarios.

The role of environments. The impact of the environment on crowdsensing is significant. The performance (the effectiveness, efficiency, and intervention frequency) across all sensing strategies is better in open environments than in other environment categories. The open environment is likely ideal and conducive for UAV crowdsensing. Future research should consider environmental factors, particularly for urban or indoor crowdsensing. We recommend creating a more realistic experimental environment to evaluate the performance of UAV crowdsensing.

The role of sensing strategies. Opportunistic strategies can utilize the behavior of UAVs without requiring large-scale resource allocation and management, while algorithmic strategies require more resources to manage and control the behavior of individuals. Collaborative strategies require human participants to pay attention to the complete workflow of crowdsensing tasks. Therefore, the human effort needed for collaborative strategies is relatively high compared to the other two strategies. Considering the level of human involvement and the complexity of strategy execution, among the three sensing strategies, the overheads of collaborative strategies are the highest, followed by algorithmic strategies, and opportunistic strategies. The opportunistic strategy generally performed poorly in the four environments and showed a significant gap compared to the other two sensing strategies, likely due to its reliance on chance rather than a systematic approach, potentially missing opportunities. Usually, the opportunistic strategy performed worse in more complex environments. Relying solely on randomness makes it exceedingly challenging to explore complex regions fully. This approach often leads to repetitive exploration within already traversed areas. As a special sensing strategy, the opportunistic strategy still has its own significance. It is suitable for low-cost data acquisition with low coverage and time requirements. In contrast, the other two sensing strategies, i.e., algorithmic and collaborative, involve deliberate planning and action to gather information and make decisions. These strategies are more reliable and effective in achieving better outcomes.

Implications for strategy selection. Collaborative crowdsensing has demonstrated an ability to achieve high performance in effectiveness and efficiency across diverse environmental contexts. A collaborative strategy can be directly selected when the budget is adequate. We learned that the performance difference between collaborative and algorithmic strategies was not statistically significant in non-artificial environments, which means the algorithmic strategy demonstrated relatively comparable efficiency and effectiveness to the collaborative strategy. However, in artificial environments, particularly in intricate indoor settings, the advantages of the collaborative strategy became pronounced, enabling it to capitalize on the wisdom of humans. Thus, an algorithmic strategy can be directly applied in non-artificial environments. In artificial environments, the cost must be considered when choosing appropriate sensing strategies.

5.2. Limitations and Caveats

The simplification of the experiment settings. UAVs usually try to keep at the same level when flying. This is because maintaining a stable flight altitude can reduce flight resistance, thereby saving energy and prolonging flight time, which is crucial to the success of flight missions. In addition, a stable flight altitude contributes to the accuracy of flight control and navigation systems. Therefore, experiments on the prototype system were simplified to a two-dimensional plane. Such simplification also helped to extend the obtained conclusions to UMV and UGV crowdsensing. The real-world scenarios where UAVs operate in three-dimensional space can be considered in the follow-up study. In addition, we did not take moving obstacles into consideration. When a swarm of UAVs complete the environment crowdsensing task together, the individuals of each UAV may become each other’s moving obstacles. There may also be other moving objects. How to sense environments with moving obstacles deserves research. This research is crucial for the development of autonomous UAVs and other technologies that rely on accurately sensing and navigating through dynamic environments.

The diversity of the sensing strategies. This study focuses on three sensing strategies: opportunistic, algorithmic, and collaborative. Specifically, we conducted experiments based on inverse ant colony optimization for the algorithmic and collaborative strategies, with the potential for further experiments using other AI algorithms. With some valuable findings from this study, an adaptive sensing strategy based on three typical strategies can also be considered in future studies. An adaptive sensing strategy means that the system can adjust the sensing strategy according to the changes in the environment to obtain more information effectively. Sometimes, a whole experiment crowdsensing task does not just take place in one environmental category. By allowing the system to dynamically adjust its sensing strategies in response to varying environmental conditions, the adaptive strategy may showcase its prowess in maximizing information acquisition efficiency and reducing crowdsensing costs. In the intricate landscape of diverse environments, each with its unique challenges and opportunities, the adaptive strategy stands poised to harness the strengths of different sensing strategies.

The sample size of the participants. The sample size of the participants was not large or representative enough; however, this is a minor problem since the number of total task executions was large enough. We can expand the scale of the experiment and recruit more volunteers for collaborative crowdsensing to increase the number of samples. We can further enrich the dataset by diversifying our crowdsensing strategies and incorporating more AI algorithms. This approach may enable us to uncover more hidden patterns and trends that may not be apparent in the current dataset. Moreover, by iteratively refining our crowdsensing strategies and leveraging advanced analytical tools, we can systematically enhance the quality and breadth of our dataset. Through this iterative and systematic process, we aim to extract more valuable insights that can propel our research efforts toward greater innovation and discovery.

6. Conclusions

In this paper, we studied how sensing strategies and environment categories can affect environment crowdsensing performance. However, our study primarily relied on simulation experiments, and future research should consider real-world scenarios where UAVs operate in three-dimensional space. In order to achieve this purpose, we implemented a serial empirical study involving categorizing sensing environments into four major categories and carrying out experiments to investigate the effect. Finally, we provided some suggestions for how to choose appropriate sensing strategies based on different needs in different environments in future environment crowdsensing. An algorithmic strategy is suitable for non-artificial environments (open, natural), while a collaborative strategy is ideal for artificial environments (urban, indoor) with an adequate budget. By understanding the differences of each sensing strategy in each environment, researchers and practitioners can enhance their decision-making processes when selecting appropriate strategies. These findings pave the way for future advancements in UAV crowdsensing, providing valuable insights for efficiently exploring diverse environments.