Flight-Based Control Allocation: Towards Human–Autonomy Teaming in Air Traffic Control ^†

Abstract

It is widely recognized that airspace capacity must increase over the coming years. It is also commonly accepted that meeting this challenge while balancing concerns around safety, efficiency, and workforce issues will drive greater reliance on automation. However, if automation is not properly developed and deployed, it represents something of a double-edged sword, and has been linked to several human–machine system performance issues. In this article, we argue that human–automation function and task allocation may not be the way forward, as it invokes serialized interactions that ultimately push the human into a problematic supervisory role. In contrast, we propose a flight-based allocation strategy in which a human controller and digital colleague each have full control authority over different flights in the airspace, thereby creating a parallel system. In an exploratory human-in-the-loop simulation exercise involving six operational en route controllers, it was found that the proposed system was considered acceptable after the users gained experience with it during simulation trials. However, almost all controllers did not follow the initial flight allocations, suggesting that allocation schemes need to remain flexible and/or be based on criteria capturing interactions between flights. In addition, the limited capability of and feedback from the automation contributed to this result. To advance this concept, future work should focus on substantiating flight-centric complexity in driving flight allocation schemes, increasing automation capabilities, and facilitating common ground between humans and automation.

1. Introduction

Many domains, including air traffic control (ATC), are moving towards higher levels of automation to increase capacity, safety and efficiency, while coping with predicted staff shortages. To enable this, automation needs to move beyond its role as a decision support tool that integrates information but cannot act autonomously. Unlike in other parts of the aviation system, e.g., the flight deck, decision-making automation is relatively sparse in ATC. Consequently, both the Single European Sky ATM Research (SESAR) program and its United States equivalent, the Next-Generation Air Transportation System (NextGen), aim for higher levels of ATC automation in the coming decades [1,2], with fewer people expected to handle more traffic in larger airspaces [3].

Advances in computing power and data exchange technologies (e.g., the increased use of Controller Pilot Data Link Communications (CPDLC)) are enabling the development of smarter and more autonomous agents that can make independent decisions and adapt to circumstances [4]. Automation is no longer limited to performing fixed preprogrammed tasks that a human has requested, and can now perform tasks without external (i.e., human) involvement. The evolution towards a fully autonomous system where no human involvement is required at all will be gradual, in line with the evolution of technology. Meanwhile, it is expected that human air traffic control officers (ATCOs) will continue to play an important and vital role for decades to come, collaborating with decision-making automation. But how should human–automation collaboration in ATC look?

The traditional approach towards full automation is to gradually increase the level of automation (LOA) of specific functions and/or tasks. In such a function-based allocation strategy, there is a continuum between ‘no’ and ‘full’ automation, with many intermediate steps involving part of a (sub)task performed by a human and part by automation [5,6]. For example, automation is able to narrow down a solution to a problem to a few options (i.e., decision selection), while humans are required to evaluate the presented options, select one, and execute it. This leads to serialized human–automation interactions where the automation requests the human operator to come back into the control loop at certain events. When more functions and (sub)tasks are assigned to a computer, humans are increasingly relegated to a passive supervisory role where they are only required to intervene in critical situations that are beyond the capabilities of the automation. As pointed out by Hancock [7], “If you build systems where people are rarely required to respond, they will rarely respond when required.” A wealth of research and operational experience with supervisory control has revealed a number of significant human performance problems, such as ’out-of-the-loop’ situation awareness, transient workload peaks, skill erosion, boredom, and reduced job satisfaction [8]. These drawbacks are known as the “ironies of automation” articulated by Bainbridge [9], which remain largely unresolved to this day [10].

An alternative approach to higher LOAs in operations which has been hypothesized to avoid the above problems is to create a parallel team composition with two or more agents working autonomously (and interdependently) towards shared goals, for example, surveillance and inspection operations where humans and autonomous systems (e.g., robots) each cover their own (geographical) area and do not interfere with each other [11]. Here, the general idea is that humans will remain involved and engaged in all facets of their work while autonomous systems help to increase productivity and alleviate workload. Leading experts in the cognitive engineering community are also advocating a shift from human–automation interaction towards Human–Autonomy Teaming (HAT) [12]. The HAT perspective has only recently come to the attention of the ATC community, and practical examples of its potential and implications are scarce.

In this article, we propose flight-based human–automation control allocation as an alternative to function-based control allocation to guide evolution towards autonomous ATC in ways that keep human controllers engaged and proficient. This concept, illustrated in Figure 1, draws inspiration from Flight-Centric ATC (FCA) [13]. FCA does not change the basic responsibilities and tasks of the human ATCO, i.e., to ensure a conflict-free flight and the means to achieve this; instead, it places the responsibility for each flight with a different human ATCO. Such flight-based rather than sector-based allocation provides a high degree of flexibility in the distribution of flights (i.e., workload) to controllers. Our proposed system can be regarded as the next evolutionary step in FCA, where full control over individual flights is allocated to either a human ATCO or a digital (computer) ATCO. Note that due to the expected benefits of a parallel system, flight-based control allocation is under consideration by the Maastricht Upper Area Control Centre (MUAC) [14].

The aim of this study is twofold: first, to discuss the potential benefits and challenges of flight-based control allocation as a step towards HAT in ATC; and second, to gain initial empirical insights into how such a system is perceived by human ATCOs and how different flight allocation schemes may affect the performance of the combined human–automation system. In a simulation exercise, ATCOs were confronted with a predefined allocation, but had the final say on which flights to delegate to or take back from automation. The results may shed light on how certain preconditions, such as existing airspace structures, personal preferences, and automation capabilities, can lead to a successful allocation strategy. We conclude with some recommendations for future directions to be explored.

2. Flight-Based Control Allocation

2.1. Theoretical Underpinnings

In the strive towards higher LOAs, many (transportation) domains, such as maritime, rail, and automotive, have prompted the development of LOA taxonomies over the past decades [15]. The vast majority of these taxonomies consider function-based automation schemes where the LOA on task levels gradually increases for all vehicles in step with the advancements of automation capabilities over time. Examples of this can be found on the flight deck and in the automotive industry, where Tesla has been gradually increasing the capabilities of its ‘autopilot’ systems [16]. Not surprisingly, the ATC community has long considered a similar function-based evolution toward fully autonomous ATC [1]. However, while function-based automation might be appropriate for supporting and/or taking over skill- and rule-based tasks (e.g., stability augmentation, autopilot, and warning systems), it may not be equally beneficial for supporting higher-level cognitive work.

Function-based automation invokes serialized interactions in which the human operator needs to monitor the computer and/or accept or reject solutions proposed by the computer [8], as illustrated at the bottom of Figure 1. In such a setup, described by Millot and Mandiau [17] as a vertical system, the human is mostly backing up the computer, leading to reiteration of a large part of the work. Serial automation at the decision-making and execution stages is often not efficient [8], as operators may prefer to or even need to re-analyze a situation. For example, if ATCOs need to check whether a solution presented by the computer is feasible, they may perform a similar or even more time-consuming analysis compared to when they had to come up with the solution themselves. Similar to on-the-job training, where experienced ATCOs sit alongside trainees to monitor their actions, automation is degraded to a ‘student’ that requires close supervision. Therefore, the desired workload reduction is often not attained with serial automation in complex tasks [8]. In addition, serial processes require high levels of conformance with respect to the ATCO’s individual style in order to increase acceptance [18,19]. Serial automation is often implemented as a hierarchy, with the human governing and delegating tasks to automation.

To circumvent the problems associated with serial automation, some transportation domains are considering an alternative constraint-based strategy. Here, high(er) LOAs are promptly attained in a constrained environment and this environment is gradually expanded. Examples include trains and shuttle buses in dedicated lanes, which can operate at very high LOAs or even fully autonomously [20,21]. In ATC, certain flights could receive ATC services at a higher LOA than others, as illustrated in the upper half of Figure 1, before making the higher LOA available to additional flights as the system matures. However, proving that automation can work under any condition and in any environment is no easy feat. Thus, the most advanced autonomous cars, such as those of Waymo, currently operate at Level 4 at most on the five-level automotive industry standard defined by [22]. These cars can only operate within geofenced areas (e.g., city centers or highways) and under specific (weather) conditions. For Level 5, these constraints need to be lifted to allow the car to safely operate anywhere at anytime.

Constraint-based control allocation leads to a parallel (or horizontal [17]) system that evokes a heterarchy, that is, a level playing field where both human and automation can take initiative and execute actions [4,23]. The importance of keeping operators engaged has been recognized [24] and is known to improve failure detection [25], situation awareness [26], motivation, and job satisfaction [8]. ATCOs who are satisfied with their jobs are in turn more willing to accept new forms of automation [27]. When human operators rarely perform a task, their cognitive skills in manually performing this task can deteriorate. During the COVID-19 pandemic, for example, ATCOs reported skill erosion due to historically low traffic demand [28].

In ATC, the most extreme form of parallelism can be achieved by fully automating an entire sector while a neighboring sector remains completely under manual control. In practice, flights in both sectors can have an impact on each other, meaning that true parallelism can only be approximated. A common frame of reference [29] or team orientation [30] is a prerequisite for efficient parallel systems, making joint communication equally or even more important [31]. Experienced human–human teams often rely on implicit communication, such as body language and voice nuances, which is naturally very difficult for computers [30]. In current-day operations, with a human ATCO in every sector, mutual performance monitoring and backup behavior is provided by placing the ATCOs responsible for adjacent sectors side-by-side or on direct phone lines with each other [32]. Insufficient communication can lead to compensatory behavior and a (temporary) return to a serial system. The main challenge in creating a parallel system is how to organize the overlap between the ATCO and automation work domains to ensure that there are not too many serial processes.

2.2. Towards a Parallel ATC System

In highly automated systems, critical moments often occur after periods with little human involvement followed by a sudden need for the human to intervene. This intervention is made difficult when the automation has largely been working independently from the human, creating a disconnect between the two agents. Numerous studies suggest that humans (including ATCOs) should be kept in the loop by establishing teamwork between the human and automation [8,33,34]. If this is not done, the work domains of the two agents barely touch each other (Figure 2a), with only minimal exchange of information, a clear example of a brittle team that would be unable to successfully handle situations when the automation’s capabilities are inadequate.

Despite the fact that much ATCO work is currently performed as a human–human team effort [35], most of the automation solutions proposed or implemented thus far are aimed at supporting individual ATCOs [14,36,37,38]. In the sociotechnical community, there is an increasing belief that applying human–human teamwork concepts in human–automation teams is the way forward [8]. By increasing the exchange of information between both agents to establish a common frame of reference [29], the overlap is increased (Figure 2b). While backup behavior is a part of teamwork [39], the overlap should not be too large in order to prevent adding a significant amount of workload solely to establish and maintain teamwork.

To assign work to either agent, the Venn diagram from Figure 2 cannot only be applied to the agents themselves, but can also be applied to the physical world that the agents are acting in. For example, the circles can represent airspace sectors, with one sector fully automated and the other controlled by a human ATCO (Figure 3a). Then, the boundary between the two sectors will solicit a certain overlap between the agents in both sectors (Figure 3b) to ensure a streamlined transfer at the sector boundaries. In current-day operations, comprehensive letters of agreement are established between adjacent units [40]. These reduce the number of locations where flights interact across sector boundaries or shortly after entering a new unit, thereby requiring less cognitive workload than an airspace with many possible conflict locations. However, the reliance on designated coordination points, altitude ranges, and additional separation criteria is in opposition to many of the benefits seen in flight-centric operations.

At a smaller scale, each circle could represent an individual flight. As long as there is no interaction between these flights, for instance when they fly in different sectors or directions, they can more or less be considered as isolated entities (Figure 4a). As soon as the flights have conflicting trajectories or trajectories that limit the other flight’s solution space, they have a certain overlap (Figure 4b). If the flights are then handled by different agents (i.e., human and automation), it becomes necessary for these agents to create a similar overlap in their work domains in order to establish efficient teamwork. Such mixed conflicts are one of the key challenges of FCA, where it is proposed to have a ‘less impacted flight’ algorithm determine which ATCO should take action to resolve the conflict with minimal deviation from the planned trajectories [41].

If both flights are instead assumed by a single ATCO who delegated one of the flights to automation, it might be undesirable to assign the resolution of these mixed conflicts in the same way as in FCA. Unlike FCA’s human–human setting, a human–automation team has much larger asymmetry between the two agents. Humans might prefer to always solve these conflicts themselves, as they can then stick to their plan (a plan the computer is likely to be unaware of). However, humans are, expected to have better situation awareness with respect to flights that are under their active control compared to delegated flights, which could complicate and/or increase the time and effort required to manually resolve mixed conflicts.

Apart from these two extremes (allocation at the individual flight level or at the sector level), an intermediate form could consist of only delegating flights in a certain stream of traffic to either automation or human, as in [42]. Overflights, for example, could be delegated to automation, as they are generally less susceptible to conflicts than climbing and descending flights that need to be merged with other traffic. When automation is assumed to be capable of handling certain areas of the airspace or specific flights, there will still be some interaction between these flights and those controlled by a human. To support the human in understanding, supervising, and/or assessing the automation’s actions in these interactions, systems can be introduced to increase automation transparency [43]. Interacting with these systems inevitably comes with substantial mental demands for its operator [44]. By tuning the allocation, the overlap between the two agents can be adjusted, which manifests itself in changes in human supervisory control performance in terms of attention allocation, workload, situation awareness, etc.

A concept along these lines is currently under development at MUAC in their ongoing ATC Real Groundbreaking Operational System (ARGOS) automation project [14]. In ARGOS, ‘basic’ or ‘routine’ flights are envisioned to be allocated to a computer system while ‘non-basic’ or ‘complex’ flights are still handled by human ATCOs [45]. This ‘basic’ and ‘non-basic’ terminology is used to distinguish between flights requiring little ATCO attention and intervention (e.g., high-altitude overflights) and flights requiring (multiple) vertical changes and/or those which may interact with multiple other flights. The ARGOS philosophy is explicitly not to replace the ATCO completely but rather to “Let ATCOs focus on the real, challenging work, to do what they are the best at, and leave the routine work to the machine” [45]. The workload reduction and other benefits that this is hypothesized to bring about can then be used to work larger or busier sectors with the same number of staff [46]. In the long term, this could evolve into single-controller operations by replacing the current human–human ATCO dyad with a computer–human dyad [47].

2.3. Technology Enablers

While fully automated and autonomous ATC may still be far on the horizon, this does not preclude the introduction of new forms and higher levels of automation in the forthcoming decades. The realization of our concept hinges mainly upon technological advancements and ongoing developments in communication, navigation, and surveillance systems. Here, high-quality and high-bandwidth digital data links facilitate the exchange of information between air and ground systems, which is important for communication (e.g., CPDLC [48]) and surveillance (e.g., Automatic Dependent Surveillance) purposes. Improvements and innovations in navigation systems (e.g., quantum navigation [49]) are making trajectory predictions more reliable, and in particular are increasing the accuracy of predicted vertical and velocity profiles [50]. These advances pave the way for performance-based navigation and 4D trajectory-based operations. The key infrastructure and concepts mentioned above are currently (partially) operational, and it is reasonable to expect them to be fully operational within the next decade(s).

In tandem with the developments of improved trajectory prediction and data exchange infrastructures, rapid advances in AI-based technology are paving the way for the creation of digital ATCOs to safely and efficiently direct flights through the airspace [51]. Examples include experimental data-driven approaches to ATC conflict detection [52] and conflict resolution [53] that can augment well-established physics-based models and algorithms. Currently, the use of AI-based technology in tactical ATC operations is not yet permitted. Legal barriers and certification hurdles prohibit the use of any technology for automated decision making and execution, leaving the human ATCO with the ultimate responsibility [54]. Therefore, it is expected that advanced data-driven and physics-based algorithms will initially be integrated into decision-support tools to assist rather than replace the ATCO. However, this does not preclude experimentation with autonomous computer agents through simulation exercises to study their impact on human–automation teamwork.

3. Exploratory Experiment

3.1. Overview and Goal

As most of the existing research focuses on automation at a functional level, it is not yet known whether sharing full control over part of the traffic in an airspace would be a workable and acceptable situation from an ATCO perspective. To gain initial empirical insights into the envisioned parallel ATC system, this section describes the design of an exploratory experiment with six professional MUAC ATCOs. Due to a lack of established flight-centric complexity metrics, initial suggestions were provided for each flight based on individual flight and airspace characteristics. The suggestions either aimed to create a parallel system (complete sectors allocated to either agent) or a more mixed system with flights in the entire airspace under mixed control. ATCOs had the final say on which flights to delegate to automation or not. The purpose of this flexibility was to gain insight in preconditions which could lead towards a successful and acceptable allocation strategy, including existing sector-based structures, personal preferences, and automation capabilities.

3.2. Participants

Six professional en route ATCOs (age M = 38.3, SD = 10.0, years of experience M = 14.8, SD = 8.7) from MUAC voluntarily participated in a real-time simulator experiment. Four of them had active licenses for the Delta and Coastal (DECO) sector group, one for Hannover, and another for Brussels.

3.3. Apparatus

SectorX, a TU Delft-built Java-based simulator (Figure 5), was designed to mimic the operational MUAC interface in order to ensure that the participants could focus on working with the experimental automation. Figure 6 shows the setup, consisting of a 1920 × 1920 pixels 27″ monitor and a standard computer mouse for control inputs.

Aircraft performance was modeled using Eurocontrol’s Base of Aircraft Data (BADA) version 3.10 [55]. The ATCOs could issue clearances for heading, route (direct-to or adding an intermediate waypoint), and altitude. Speed and vertical rates were not controllable. All clearances were uplinked through a datalink, removing the need for voice transmission over radio, and were instantaneously executed by the simulated pilots. Using MUAC’s Verification and Advice tool (VERA), the ATCOs could request the predicted minimum distance between any two flights (based on linear extrapolation of the current tracks) as well as the time-to-go until that minimum distance was reached. VERA’s graphical representation of the conflict geometry and corresponding flight positions was not included in this particular experiment.

3.4. Airspace and Traffic Scenario

Participants were responsible for air traffic above FL245 in the combined Delta and Jever sectors above the Netherlands and part of Germany. This combination of sectors from the DECO sector group is not used in real operation, but was selected for the experiment because it encompassed two comparably sized sectors. Each ATCO experienced the same real-time traffic scenario: a typical radar snapshot of flights passing through the controlled sectors on an average day in February 2020, prior to the COVID-19 pandemic and the associated traffic reduction. Figure 7 shows the scenario’s traffic density, with a clearly visible hotspot in the Delta sector.

There were between 15 and 30 (M = 21, SD = 4) flights in the combined sectors at any time (Figure 8), totaling to 104 flights for the entire 95 min scenario. MUAC ATCOs would currently handle 20–25 flights in a combined sector the size of Delta and Jever. A higher peak value was chosen for the experiment to compensate for an expected workload reduction caused by offloading some flights to the automation and by the absence of voice communication. As two of the participating ATCOs were not licensed for this sector and the automation was new to all participants, the traffic density was not artificially increased beyond current levels (e.g., as sometimes done to simulate future traffic densities).

All flights followed standard routing or direct routes (24 flights) to their designated exit points and flew at a constant indicated airspeed of 250 knots. In addition to overflying traffic, arrivals and departures to several airports within or close to the sector were included. In total, 22 flights had to climb within the sector, 15 flights had to descend, and the remaining 67 flights had identical entry and exit levels. There was no wind.

3.5. Automation

During the exercise, the ATCOs were accompanied by an ‘automated colleague’. When flights entered their sector, the ATCOs had to decide whether to manually assume the flight or delegate it to automation (Figure 9). The allocation remained flexible, allowing the participants to re-assume manual control or delegate flights at any time anywhere in the sector. To enforce an explicit transfer of responsibilities, all flights had to be manually transferred to the next sector, including those delegated to automation.

For experimental control, simple rule-based automated solvers were used to provide a basic level of automation that was both predictable and easy for the ATCO to understand. This diminishes or even removes the requirement for the highly independent automation to extensively communicate its intentions to the human operator, for which (complex) displays would be needed. The design of such displays is an entire research topic on its own, while this study’s focus is the impact of flight allocation on teamwork rather than the impact of inter-agent communication.

The automation was fully capable of acting without human involvement and automatically executing actions to ensure safe air traffic. The automation performed the following tasks on flights delegated to it:

• Ensuring sufficient separation between automated flights (5 NM horizontally, 1000 ft vertically).
• Delivering flights at their exit point and transfer level while climbing as early as possible and descending as late as possible.
• Descending arrivals to FL260 for transfer to lower area control.

The computer solved conflicts between automated flights in the vertical plane only. It would never issue any heading clearances or direct-to’s, meaning that flights allocated to automation would continue along their planned routes (or routes modified by the ATCO). If both conflicting flights were already at their planned exit level, one of the flights would be instructed to climb or descend 1000 ft to solve the conflict. Thereafter, when clear of the conflicting traffic, the flight would be instructed to return to the exit level. All human–automation conflicts had to be solved by the ATCO, under the presumption that automation would not know the ATCO’s plans or intentions. Apart from showing the issued clearances in the flight labels, the automation did not provide any other feedback on its intentions.

3.6. Procedure

All participants followed the procedure outlined in Figure 10, starting with a short briefing and pre-experiment questionnaire about their stance on automation. Next, each participant received ten minutes of training, during which the automation was introduced and they could familiarize themselves with the interface and practice their designated tasks. Both human–automation and automation–automation conflicts were shown in order to demonstrate how automation would handle both situations. The training was concluded with a short questionnaire.

Next, the experimental run started with a five-minute period simulating a shift take-over, during which scripted clearances were automatically executed without the ability to issue manual clearances, followed by 90 min of real-time simulation. Throughout the run, the experimenter observed and asked the ATCOs to explain their actions and what they were taking into consideration. Every three minutes, the ATCOs had to rate their instantaneous self-assessed (ISA) workload by clicking on an on-screen 0–100 scale [56]. The scale showed their previous rating for reference. After the experiment, they completed a questionnaire with several open and Likert-type scale questions.

3.7. Independent Variable

There was one independent variable, namely, the suggested flight allocation, which was unique for each ATCO, as specified in Figure 11. Rather than including a complexity metric driven by interactions between flights, the suggestions were either solely based on each flight’s required absolute flight level change $|\Delta \mathrm{FL}|$ from the entry level (NFL) until the transfer level (TFL) or based on the sector where the flight first entered the controlled airspace. Flights with $|\Delta \mathrm{FL}|\le$ 2000 ft were considered overflights in this context. For ATCOs 3 and 4, flights crossing the internal sector border would automatically change the responsible agent during the take-over period to adhere to the suggested allocation.

The suggestions were shown by the color of the flights’ labels and radar position symbols upon approaching the sector (green = manual, blue = automated, in accordance with the colors on the MUAC radar screen, where green flights are under control of the ATCO and blue flights are within that ATCO’s area of responsibility but controlled by a different ATCO). The ATCOs were not briefed on which scheme was applied to them. They could ignore the suggestions and re-allocate flights at any time, even after delegating them to automation. Note that the chosen allocation color replaced that of the suggested allocation once a flight had been allocated by the ATCO.

3.8. Control Variables

The following control variables were the same for each participant:

• Airspace and traffic sample: As described in Section 3.4.
• Atmospheric conditions: International standard atmosphere without wind.
• Automation capabilities: As described in Section 3.5.
• No voice communication: All instructions were transmitted via CPDLC.
• No pilot or CPDLC transmission delays.
• ATCO support systems: Only VERA and short-term conflict alert (STCA).

3.9. Dependent Measures

The following measures were collected in the experiment:

• Pre-experiment questionnaire: Prior to the simulation session, a short questionnaire using both open-ended and Likert-type scale questions aimed to probe the ATCOs’ stances and preconceptions on what automation could offer them. They were also asked to indicate which tasks and functions they would like or expect automation to support and/or take over.
• Post-training questionnaire: After brief exposure to automation in a flight-based control allocation context, the ATCOs expressed their initial opinions on the new concept.
• Experiment trial:
  • Chosen flight allocations: Each ATCO was presented with a unique initial flight allocation, but was free to revise the suggested allocation as they saw fit by either delegating flights to automation or taking back control from automation at any time. As such, their level of appreciation for the initial allocation could be observed.
  • Control activity: The number, type, and timing of issued clearances (altitude, heading, and direct-to).
  • Perceived workload: This was measured through an instantaneous self-assessed (ISA) rating on a 0–100 scale every 3 min during the experiment trials [56].
• Post-experiment questionnaire: After the experiment, the ATCOs provided their opinions on the automation, flight-based control allocation concept, and simulation in general after having worked with it during the 90 min trials.

4. Results

Because the number of participants was small and the primary goal of the experiment was to provide a first insight into the feasibility and challenges of delegating part of the traffic to automation, we focused on providing a qualitative analysis of the raw data, observations, and questionnaires rather than engaging in inferential statistics. Results are presented in accordance with the four data collection phases defined in Section 3.9.

4.1. Pre-Experiment Questionnaire

The results of the questionnaire at the start of the experiment revealed that the participating ATCOs had mixed opinions on automation and its involvement in their work (Figure 12). Answers into this and other questionnaire figures that are either relatable to human or automation have been colored according to the agent; a green bar corresponds to a more human-favorable answer, while a blue bar relates to an automation-favorable answer. Most ATCOs trusted automation and expressed the opinion that it lowers their workload in general, but were also strongly of the opinion that a human should ultimately be in charge.

While this experiment focused on flight-based allocation, a human–automation team may also be created by sharing tasks according to a more conventional function-based allocation scheme. We replicated part of the study from Prevot et al. [3] to see what kind of tasks the ATCOs would prefer to do themselves and which they would share with or completely delegate to automation if function-based allocation was used. In line with the findings of [3], Figure 13 shows that the ATCOs indicated that a considerable number of tasks can be either shared with or completely delegated to automation. Transfer of control can be automated as a first step towards more automation, but ATCOs would like to be able to reject auto-transfers as well as to initiate early transfers. The ATCOs preferred to keep short-term tactical actions under their control, while suggesting that more strategic long-term planning and routine tasks can be (partially) delegated to automation.

4.2. Post-Training Questionnaire

After a brief introduction and exposure to the experimental automation, the ATCOs had mixed opinions on whether it would be a useful asset in their work, as shown in Figure 14. Furthermore, contrary to their high level of trust in automation in general (Figure 12), the ATCOs indicated very low trust in this particular form of automation.

4.3. Experiment Trial

4.3.1. Chosen Flight Allocation

All ATCOs delegated at least 50% and up to 100% of the flights that were simultaneously in the airspace to automation, regardless of the suggested allocation (Figure 15). Had all ATCOs adhered to the suggested allocations, the figures on the right would have been the inverse of those on the left (i.e., all flights that were green for ATCO 1 would have been blue for ATCO 2 and vice versa). Note that despite the identical traffic scenarios, the total number of simultaneously assumed flights varies slightly between ATCOs due to individual assume and transfer timings. Whereas most ATCOs largely ignored the suggested allocation, ATCO 3 tried to follow it after realizing that one of the sectors was completely handled by automation. Moving on to ATCO 4, the stark shift around 60 min from a nearly completely automated airspace towards a substantial number of manual flights followed a self-proclaimed “test of the automation” by this ATCO who deliberately re-directed flights manually. While the same ATCO claimed to be “comfortable with purely monitoring a completely automated scenario”, this ‘test’ may have been a sign of boredom. For ATCO 6, all flights were suggested for delegation; however, they did assume some of the flights (or at least part of their paths) manually. Note that none of the ATCOs who received the suggestion to delegate flights (everyone but ATCO 5) assumed more flights manually than was suggested to them. For all ATCOs combined, flights were manually assumed for 23% of the total flight time.

The strategies suggested to the ATCOs and those they ultimately adopted can be more clearly seen when comparing the ground track density maps and required vertical movement $|\Delta \mathrm{FL}|$ with the associated distribution of flight time (i.e., the duration that a flight was allocated to either agent) in Figure 16. Note that flights under control by either agent shared the same airspace in the experiment, and have only been split per agent in the figure for visualization purposes. The left (green) and right (blue) figures always add up to 100% of the traffic for each ATCO. As an example, ATCO 1 delegated 73% of the flight time in the Jever sector and 81% of the flight time in the Delta sector to automation. For the remaining time, 27% and 19%, respectively, the flights were under manual control. In terms of vertical movement, only 24% of the flight time of flights with $|\Delta \mathrm{FL}|\le$ 2000 ft (LT) was under manual control by this ATCO, with the remaining 76% being automated. Following the suggested allocation would have led to 100% manual flight time, as indicated by the green outline of the ‘LT’ bar. The ’GT’ bars resemble the flights with $|\Delta \mathrm{FL}|>$ 2000 ft. The suggestion was that ATCO 1 automate all of these flights, they instead chose to take manual control for 17% of the flight time.

As already hinted at by Figure 15, ATCO 3 exhibited the greatest adherence to the proposed allocation strategy among all ATCOs. This ATCO even delegated flights as they transitioned from Delta to Jever, commenting that solitary manual flights in a predominantly automated area were difficult to handle. This resulted in 95% of the flight time in Jever being delegated to automation, approaching the suggested 100% in this sector-based allocation. Interestingly, the same ATCO did not consistently re-assume automated flights that entered Delta from Jever, resulting in a considerable 46% of the flight time in Delta being delegated to automation instead of the suggested 0%. All other ATCOs had a considerably more even distribution in both sectors.

For ATCOs 1 and 2, the suggested allocation was based on the required $|\Delta \mathrm{FL}|$ between sector entry (NFL) and transfer (TFL) rather than on the entry sector. ATCO 2 appears to have followed the suggestions slightly better than ATCO 1, as shown by a 97% delegation of overflights and considerably lower share of 66% for flights with $|\Delta \mathrm{FL}|$ > 2000 ft. It must be noted that the bars in Figure 16 are based on the $|\Delta \mathrm{FL}|$ between sector entry and exit; therefore, flights that were flying level for the majority of their flight time may have predominantly contributed to the blue GT bar even tough they were assumed manually during the actual (short) climb or descent phase. Upon closer inspection, all ATCOs frequently delegated flights as soon as they had (almost) reached their TFL and were clear of any remaining conflicts. ATCO 3 is again a noticeable outlier, with manual control used for a relatively large share of flights with small $|\Delta \mathrm{FL}|$ for a prolonged time (43%) due to this ATCO’s adherence to the suggested sector-based allocation.

One of the ATCOs kept a low and slow flying Beluga cargo flight with $|\Delta \mathrm{FL}|$ = 0 ft under manual control throughout the sector. This ATCO commented that this was mainly because of the flight’s close proximity to lower airspace that is not controlled by MUAC, and the potential disruption it may cause to or receive from traffic in that airspace. If such traffic had been present in the experiment, the other ATCOs may have also decided to manually control the Beluga.

4.3.2. Control Activity

The ATCOs issued 51% of all clearances (30% of altitude clearances), leaving the rest to the computer (Figure 17). As discussed in Section 4.3.1, most ATCOs took manual control for a short period of time in order to issue a clearance before delegating the flight for the remainder of its trajectory. Here, 55% of the flights did not change agent after being assumed, while 43% of the flights that did spent less than one minute with the ATCO (Figure 18). This was especially true for flights that could benefit from a direct-to, which the computer could not issue. Interestingly, ATCO 3 hardly sent any flights on a direct-to, while ATCO 5 did so for over 50 flights. ATCOs 1 and 3 both issued intermediate-level clearances to up to 25 flights, resulting in an above average total number of altitude clearances.

4.3.3. Perceived Workload

The ISA workload ratings in Figure 15 do not show a significant correlation with the fraction of automated flights. A higher share of automated flights did not provide the workload reduction that ATCOs generally expect with automation (Figure 12). ATCO 5 reported a very consistent workload, ranging between 15 and 24 (on a 100-points scale), whereas for ATCOs 3 and 4 it varied considerably. In general, the ATCOs commented that their workload was low due to a relatively low traffic demand (when considering that part of the tasks were performed by automation) and the absence of voice communication. Because every ATCO only experienced one of the suggested allocation schemes, and as workload ratings are subjective, no further within-participant comparisons can be made with respect to the suggested or followed allocation schemes.

Despite 51% of the control actions being performed by the ATCOs, only 23% of all flight time was manually assumed. This discrepancy may explain why the number of flights allocated to either agent did not correlate with the perceived workload.

4.4. Post-Experiment Questionnaire

4.4.1. Flight Allocation

At the end of the experiment, the ATCOs were asked what percentage of flight time they believed themselves to have delegated to automation over the entire run. All ATCOs were able to estimate this within eight percent point of the actual median (

Table 1. Self-reported and actual flight time delegated to automation over the entire run.

	ATCO
	1	2	3	4	5	6	Mean
Self-reported (%)	71	86	66	92	95	71	80
Actual mean (%)	77 (+6)	85 (−1)	60 (−6)	88 (−4)	80 (−15)	77 (+6)	78 (−2)
Actual median (%)	79 (+8)	86 ( = )	58 (−8)	94 (+2)	91 ( −4)	75 (+4)	81 (+1)

), indicating a good match.

The questionnaire provided further insight into how the ATCOs determined whether flights should be delegated in the experiment trial (Figure 19). Note that the ATCOs could classify factors as ‘not considered’ (this ranged from three to six per ATCO), meaning that not all ATCOs ranked the same number of factors. Traffic directly surrounding a flight was considered especially important when there were many nearby manual flights. Delegating a single flight to automation would then have added (too) much uncertainty. Overflights are generally considered to be more predictable than arrivals and departures, making the type of flight another important factor. Special flight types, such as the slow and low-flying Beluga cargo flight, also played a role here. The suggested allocation was given low priority or even ignored by most ATCOs. ATCO 3 ranked this as the most important factor and acted accordingly, as confirmed by Figure 15 and Figure 16. If automation would have been capable of giving direct-to’s, the ATCOs commented that they would have delegated more flights in this experiment. Four ATCOs included the automation capabilities, while only two included their workload (which was relatively low, as discussed in Section 4.3.3). Surprisingly, some ATCOs mentioned that the distance to the next sector should be included in an allocation algorithm, despite none of them considering it in their own allocation decision-making during the experiment.

Although every ATCO only received suggestions from a single flight allocation scheme in the experiment trial, the questionnaire asked their opinion on all of the schemes from Figure 11. Grouped per type of allocation-driver, their feedback was as follows:

• Vertical change: The ATCOs unanimously agreed that ‘complex’ climbing/descending flights need to be handled manually (potentially with support tools). They indicated a strong preference for delegating ‘basic’ (over)flights to automation. For most ATCOs, this was also reflected in the time that they delegated such flights; 70% of the flights with identical entry and exit levels ($|\Delta \mathrm{FL}|=0$) were automated for 95% of their flight duration, versus only 50% of the flights requiring some level change. Although some ATCOs commented that a 5000 ft level change would have been a more appropriate threshold to divide traffic in basic and complex than the used 2000 ft, this was not directly reflected in their chosen allocation strategy. All traffic that had to change levels evoked more manual control than overflights, and as such could be considered at least somewhat ‘complex’. This corresponds to the allocation suggested to ATCO 2.
• Sector: Allocating flights per sector was outright rejected by three ATCOs, who commented that the choice of whether or not to delegate a flight should depend on the situation rather than the geographic sector. Two ATCOs (including ATCO 3) did see some use in it when one of the sectors was busy and/or required more concentration, while one ATCO refrained from commenting.
• Full manual or automation: Four ATCOs praised the fully manual scheme for giving them full authority over which flights to delegate to automation and when (e.g., after turning and climbing). One ATCO preferred to have overflights always proposed to automation, while the remaining ATCO simply disliked this scheme. Finally, the fully automated scheme received favorable comments from five ATCOs, provided that the automation functioned well and that the supervising ATCO could take over at any moment. One ATCO criticized it on the basis that there will always be flights that need human involvement due to their flight profile or because they pass through traffic hotspots.

4.4.2. Perceived Impact of the Automated Agent

Figure 20 shows that the ATCOs believed the automation as implemented in the experiment to have somewhat worsened their situation awareness and work style. Nevertheless, all ATCOs classified their situation awareness as ‘okay’, the middle score on a five-point Likert scale from ‘poor’ to ‘very good’. All ATCOs mentioned that they paid (much) less attention to the blue automated flights, akin to transferred flights, even though they knew that they were still responsible for these flights. At the only (not explicitly programmed) occurrence of a short-term mixed conflict in the experiment, the involved ATCO was surprised by the STCA. He explained that he had not spotted the conflict because the automated flight was emerging from “a sea of blue aircraft”. Any other potential mixed conflicts were avoided by the ATCOs in a timely way.

The ATCOs reported the lack of feedback from the automation regarding what actions it would take and the associated uncertainty as the largest contributor to their negative experience. All ATCOs would have liked the automation to at least show its intentions about where on the trajectory it would start and end a climb or descent. Interestingly, they also reported that the automated agent’s actions were not surprising, suggesting that the implemented rule-based automation was in fact predictable. However, several ATCOs reported that they proactively took manual control at times (e.g., when approaching a top of descent) in order to forestall any potential surprises from the automation.

In the end, the automation did have a positive impact on their overall perceived workload and was considered to be somewhat useful by most ATCOs, exceeding expectations as reported in the post-training questionnaire (Figure 14). Nonetheless, only ATCO 4 considered collaborating with the automation to be a form of teamwork.

4.4.3. Simulation Fidelity

This experiment was a first test of the SectorX simulator’s MUAC style. The ATCOs were asked to rate the fidelity of various aspects when compared to the operational human–machine interface and traffic (Figure 21). The most missed interface feature was VERA not showing the conflict geometry at the closest point of approach.

The aircraft behavior was rated as unrealistic by two ATCOs, who particularly mentioned the simulated climb rates. This can be partly attributed to the fact that in the absence of operational data, all flights were assigned a constant reference mass from the BADA performance model, making them exceptionally light on departure and heavy on arrival. All aircraft were flying at a constant indicated airspeed, and did not follow the standard climb/descent profiles. In addition, no pilot delay was modeled, meaning that all clearances were immediately executed.

Finally, the traffic scenario was considered realistic, although somewhat low in traffic, leading to relatively low workload ratings throughout the runs.

5. Discussion and Future Work

The experimental results of our initial exploration of flight-based control allocation in en route ATC show promising results in terms of concept feasibility and ATCO acceptance. Followup studies should address the limitations and assumptions of our study, e.g., by introducing pilot delays, wind, and voice communication. Together with a more demanding traffic scenario, in which unaided manual ATC would lead to excessive workload, this is hypothesized to better demonstrate the benefits of offloading flights to automation in conjunction with the distinctive problem-solving abilities of ATCOs. In addition, we recommend that future research focus on the three research areas outlined below in order to bring the concept one step closer to operational implementation.

5.1. Automation

At the start of the experiment, all ATCOs reported having a high level of trust in automation in general, but were nevertheless suspicious of the experimental automation after the (short) training. Despite this, their trust was largely restored through the experiment trial (Figure 22). According to the ATCOs, this trust buildup was mainly due to their seeing the automation perform well. The rule-based form of automation (programmed to be ‘perfect’), clear separation of responsibilities, and absence of uncertainties such as wind and pilot behavior further contributed to this assessment.

Fostering a high level of trust in the automation is of paramount importance in successfully creating a parallel system. In this experiment, the ATCOs needed some time to observe and monitor the behavior of their digital colleague before a sufficient level of trust was gained. Occasionally, they saw a need to intervene by taking back control over a flight, issue a clearance, and delegate it back to automation again. Monitoring and intervention are typical of supervisory control environments, which invoke the more serialized interactions that flight-based control allocation seeks to avoid.

Based on ATCO comments and observations during the experiment, the need for monitoring and intervention appeared to be caused by the limited capabilities of the automation combined with the ATCO’s responsibility for resolving mixed conflicts as well as the rather simple and pragmatic flight allocation schemes. To achieve a more desirable parallel system, automation should be able to perform all ATCO tasks and should have the responsibility and authority to resolve mixed conflicts. The potential downside of the latter, where the automation avoids all other computer- and human-directed flights, is that the two agents might engage in oscillating behavior by reacting to each other’s actions. One way to avoid this issue is to use a smarter flight allocation strategy based on predicted interactions between flights rather than on individual flight and/or sector characteristics. If mixed conflicts do occur, a ‘less impacted flight algorithm’, as developed for FCA [42], could quickly identify which agent can resolve the conflict with minimal effort and disruption and alert the ATCO if it is their responsibility.

5.2. Flight Allocation Suggestions

In general, the ATCOs did not follow the suggested flight allocations, with half of the ATCOs explicitly reporting that they ignored them (Figure 19). The suggestions were based on simple pragmatic schemes, and did not take into account the actual interactions between flights. While the ATCOs indicated that the automation capabilities and suggested allocation were equally important, the former seemed to be prevalent in their chosen allocations. The high number of lateral control actions issued by the ATCOs illustrates the lack of lateral control ability on the part of the automation. Nevertheless, half of the ATCOs reported that they did not consider the automation’s capabilities in their allocation.

To provide more fitting (and consequently more accepted) flight allocation suggestions, it may be beneficial to take the actual (predicted) complexity of flights into account. In contrast to the abundance of research on sector-based complexity (e.g., dynamic density, interval complexity, fractal dimension, input/output approach, Lyapunov exponents, and trajectory-based complexity [57,58]), the complexity of individual flights is less understood. Flight-centric complexity is a prerequisite for the automated allocation of flights in both FCA and our proposed operations. In FCA, a predicted workload increment per flight is used to allocate flights while balancing the workload between ATCOs [41], whereas in our proposal low-complexity flights are allocated to the automated agent. In both cases, interacting flights are best assigned to the same agent in order to prevent ‘mixed conflicts’ between flights under the responsibility of different agents and avoid excessive communication and coordination efforts between agents.

Unlike sector-based complexity, the perceived complexity of a single flight of interest is not defined by the number of actions required for this flight. Instead, it is centered on spatiotemporal interactions with the surrounding traffic. Following the experiment presented in Section 3, an initial study investigating the underlying parameters confirmed that similar to workload, the perceived flight-centric complexity is susceptible to ATCO perceptions and experiences, although it centers on interaction parameters (e.g., overlapping altitudes and crossing routes) [59]. Tuning the algorithm for individual ATCOs may increase acceptance [60], but could also complicate shift take-overs and require that the automated agent deal with a wider range of potential interaction complexities. Fortunately, there is a substantial level of consensus between ATCOs regarding flights on either end of the complexity spectrum, providing sufficient basis for an allocation algorithm [59].

The complexity of a flight is not necessarily constant throughout its time in the sector. Section 4 showed that ATCOs frequently delegated flights after they had passed the ‘challenging’ part of their route, i.e., climb, descent, or conflict situations. External factors such as adverse weather conditions and associated reroutes can also play a role as trajectory uncertainty increases. Thus, flights could be re-allocated based on changes in their complexity beyond a certain threshold.

5.3. Feedback and Communication

In the experiment, the ATCOs reported that they did not like the lack of feedback from the automation, especially with regard to planned tops of climb/descent. Lack of communication is a common pitfall in automation design that hinders the establishment of productive human–automation teamwork [61]. Consensus within the cognitive engineering and AI communities also points to the requirement that automation should disclose information on its capabilities, limitations, what task(s) it is currently doing, why and how it is doing the task(s) in the specific way(s) that it is, and what it plans to do next. This type of feedback is commonly referred to as “seeing-into” transparency [62,63]. However, opening the ‘black box’ may also come with new challenges related to clutter and the complexity of (visual) representations, potentially leading to increased workload and delayed responses [64,65]. Especially for ATCOs, who prefer to have a clean and uncluttered radar screen, ‘minimalistic’ feedback solutions are sought that, for example, can communicate machine intentions via decision-support tools (e.g., VERA) that ATCOs are already using today.

Interestingly, several empirical studies in ATC have reported limited benefits of automation transparency. For example, Westin et al. [66] showed that ATCOs’ acceptance of machine-generated resolution advisories was more affected by matching them to human preferences and strategies than to transparency. This suggests that understanding and accepting machine intentions can also be achieved by matching the automation’s behavior to ATCO best practices, preferences, and expectations [19]. In addition, some form of ‘letter of agreement’ between human and digital ATCOs could further reduce the need for inter-agent communication and coordination, similar to how Standard Instrument Departures and Standard Terminal Arrival Routes minimize the need for communication between ATCOs and pilots. Eventually, when automation is sufficiently reliable and has proven itself in a variety of challenging circumstances, the need for “seeing-into” transparency may diminish altogether. This is no different from high-performing human teams, where team members do not need to understand and communicate each other’s intentions as long as the work is done well.

6. Conclusions

This exploratory study has yielded useful insights into human–automation teaming in a realistic ATC setting. We show that after initial skepticism, professional en route ATCOs are not averse to sharing their work in an airspace with automation. In a simplified situation lacking uncertainties due to wind, emergencies, and pilot requests, a high level of delegation to automation was reached under the condition that flights were on direct routes and free of conflicts. The ATCOs generally ignored the suggested allocations, hinting not only at the need for a different form of allocating flights but also at the impact of automation capabilities.

While some ATCOs may simply prefer to make their own division when flights come in, the majority of the participants welcomed an automated allocator. However, the automation should be able to perform all tasks in order to prevent the serialized interactions found in systems requiring human supervision. This includes issuing direct-to’s, assuming/transferring flights, and solving or communicating about conflicts between human-directed and automation-directed flights. To further minimize interactions between the human and computer, future research should take a closer look at determining the complexity of individual flights and consequently classifying them as ‘basic’ or ‘complex’ such that a fitting allocation scheme can be applied. Together with empirical studies on the various forms of task sharing and distribution, this can help establish human–automation teamwork in a shared ATC environment.