White Paper on Adaptive Situational Awareness Enhancing Augmented Reality Interface Design on First Responders in Rescue Tasks

Abstract

The advance in the development of augmented reality technologies has attracted interest in their applicability in rescue scenarios. The characteristics of the different missions covered by First Responders, as well as the different objectives they can cover in a rescue operation, condition the importance of the additional information they can receive in these rescue processes through technology. This white paper aims to analyze the difficulties encountered when converging on the design of an interface that is adaptable to the professional and contextual circumstances in a rescue task.

1. Introduction

During the last few decades, augmented reality and virtual reality technologies have been tested in order to support and boost the efficiency of medical and rescue tasks of First Responders (FRs) [1,2,3,4,5,6]. The European projects such as FASTER [7,8], RESCUER [9,10,11] and RESPONDA [5,12] also indicate the technological interest in evaluating augmented reality technologies to enhance situational awareness and early risk identification through mobile and wearable technologies, ameliorating mission management and personalizing information delivery to FRs [7,9]. The research [9] summarizes the implementation of an Augmented Reality (AR) interface as a component of smart helmets for First Responders (FRs). In contrast, the project [7] considers providing the AR interface through both mobile phones and AR glasses (e.g., HoloLens) by overlaying sensor-provided data onto the real-world view. The performance evaluation research [3] proposes a training framework that leverages augmented reality and virtual reality technologies to improve the accuracy and speed of task execution, which is a novel design validated in an emergency medical services use case.

In order to evaluate the feasibility of innovative solutions in FR operations, the International Forum to Advance First Responder Innovation [13] has identified ten recurring global capability gaps. This list establishes essential technical requirements that will guide the development of innovative technologies specifically designed to improve the security, competence, and overall effectiveness of FRs. Between those gaps, real-time information sharing seems primordial to track the location of FRs [6,14] as well as monitor and analyze threats and hazards. The incorporation of information available from multiple and nontraditional sources has been detected as an important capability gap, including the option to monitor physiological signs of FRs. Apart from sensor-based information, interoperable communications with FRs are supposed to enhance FRs’ safety and efficiency independent of environmental conditions [15,16].

In addition, the real-time monitoring of physiological signs of FRs can provide useful cognitive information about FRs in rescue scenarios. Indeed, recent studies evaluate the relationship between vital signs and mental task execution [17,18], indicating that mental resources can indirectly be measured in rescue situations that are physically and mentally demanding [19]. Cognitive load, defined to measure mental resources management when dealing with novel information [20], can be an indicator to expose limited data to FRs in rescue scenarios. These measures are indispensable to prevent an overwhelming display of information for rescuers. Recent studies [10] describe the created technology to adapt information exposition to the FRs according to their cognitive processing capabilities.

The success of AR user interface design strongly depends on its adaptability to its field of application [21]. Therefore, testing different prototype applications in real scenarios is necessary for validation [22]. In this context, the research [10] collects the effort to create a modular AR design that can adapt to the distinct requirements of First Responders in various rescue circumstances. It summarizes an icon system designed to represent distinct functionalities, facilitating the comprehension and integration of sensor-based and artificial intelligence-based information within their rescue context. The key to creating a personalized interface was an external dashboard manipulated by the control center that could select/eliminate the visualization of interesting functionalities according to the FR biological and environmental circumstances.

The roles and tasks that FRs have to perform in rescue operations vary according to their professional capabilities and the circumstances of the accident environment. As [23] indicates, Emergency Medical Services, Firefighters and Police Officers are professionals who participate in different ways in these rescue tasks. In addition, emergency response teams exhibit varied configurations [24], which are often tailored to the specific requirements of each state or autonomous community. For instance, within Spanish mountain rescue operations, the composition of teams differs significantly according to region: medical professionals are integrated within rescue teams in the Central Pyrenees [25], while in regions such as the Basque Country, a dedicated sector of security forces assumes primary responsibility for initial medical assistance [26].

In this context, different perspectives emerge when prioritizing information in rescue operations due to the nature of various tasks, professional capabilities, and rescue team configurations. The main goal of this project was to create a standardized and robust configuration of the modular interface introduced in [10] that adapts to the needs of First Responders (FRs) according to their different roles and rescue scenarios. In order to concrete these configurations, a survey was designed to collect information criteria from FRs across five different profiles and three rescue scenarios. During the integration and testing pilot conducted as part of the H2020 European project RESCUER [11], this survey was completed by various FRs belonging to rescue partners. This white paper analyzes the challenges encountered when translating statistical data into concrete configurations.

2. Survey to Collect Information Prioritization Requirements According to Operation Modes and User Profiles

This section summarizes the technical and descriptive details of the context in which the survey was conducted as part of the European H2020 RESCUER project [11]. The aim was to define an AR display configuration for the smart helmet based on distinct rescue operation requirements. The primary focus was on analyzing, through a survey, the prioritization of environmental and biosignal information based on user profiles and task types, leveraging the modular configuration of the display. The ultimate goal was to create standardized interfaces for use in rescue scenarios.

This section organizes the context of the developed questionnaire in several parts. The first subsection provides a statistical description of the First Responder (FR) professionals who participated in the survey. Then, the distinct profiles of the FRs and the different rescue tasks considered in this analysis are detailed. The following subsection explains the smart helmet that has been tested and analyzed by end-users within the European H2020 RESCUER project [11] in order to evaluate the AR modular interface display configurations within distinct rescue scenarios. The awareness-enhancing modules that conform to this display are briefly detailed in Appendix B with the respective symbols. The last subsection summarizes the questions collected in the study.

2.1. Population

Within the European RESCUER project, a series of pilots were conducted to perform progressive integration tests in the three characteristic rescue scenarios outlined in the project: earthquakes, mountain rescue, and tunnel rescue. At the end of February, in Navacerrada, a pilot was carried out to test the integration and functionality of the different technological modules developed in the mountain rescue scenario. Various First Responder profiles detailed in Section 2.2 attended this event to participate and provide feedback on the technology under development. The smart helmet, which featured a monocle displaying simulated information from biological and contextual sensors, was tested during a simulated rescue scenario in the dark. The test was conducted in an enclosed space with low levels of artificial and natural light and a controlled amount of smoke. First Responders were tasked with locating victims using the information provided by the smart helmet. This scenario focused on creating realistic low-visibility conditions rather than stimulating actual human stress responses. Upon completing the tests, participants filled out a questionnaire as depicted in Figure A1 for three distinct operation modes: SURVIVAL, SEARCH, and COMPLETE, as detailed in Section 2.2.

The survey was completed by 33 different FRs from the institutions ARC, ENSOSP, FEM, GERA, ISEMI, SDIS, SUMMA 112, and UME. A description of the institution acronyms, as well as the number of participants for each institution, is summarized in Appendix A in

Table A1. Number of participants in the survey grouped by institution.

Acronym	Institution	Country	Number of Participants
ARC	Österreichisches Rotes Kreuz	Austria	3
ENSOSP	École Nationale Supérieure des Officiers de Sapeurs-Pompiers	France	1
FME	Federación Madrileña de Espeleología	Spain	1
GERA	Grupo Especial de Rescate en Altura	Spain	10
ISEMI	ISEM-Institut pre Medzinárodnú Bezpečnosť a Krízové Riadanie	Slovakia	1
SDIS	Service Départemental Incendie et Secours de la Savoie	France	2
SUMMA 112	Servicio de Urgencias Médicas de Madrid	Spain	12
UME	Unidad Militar de Emergencias	Spain	3

.

2.2. First Responders and Operation Type Description

The different types of missions have been grouped into five main categories: Firefighting, Search and Rescue, Natural Disasters, Emergency Medical Services, and Hazardous Materials (Hazmat) Incidents.

• Firefighter: The main task of a firefighter is to control and extinguish fires occurring in settings such as residential, commercial, and wild-land areas. Frequent circumstances in these operations include combustible materials, with major risks including smoke, oxygen deficiency, elevated temperatures, poisonous atmospheres, and violent air flows, creating toxic and dangerous environments [27].
• Search and Rescue: They coordinate efforts to locate and assist persons, vehicles, other conveyances, or buildings considered in imminent danger. Participation in these tasks requires prior training and knowledge of the use of appropriate resources to locate persons in distress and provide them with the necessary assistance, including initial medical care, to ensure their safety and well-being.
• Natural Disasters: The main task of these FRs is to provide assistance in natural disaster situations such as forest fires, floods, extreme snowstorms, landslides, earthquakes or hurricanes. This assistance takes the form of evacuation tasks, search and rescue operations, and health care, among others.
• Emergency Medical Services (EMS): They provide urgent pre-hospital treatment and stabilization for serious illnesses and injuries that occur in accidental circumstances. In some situations, the transport of injured victims is the next point of care, typically to an emergency department of a hospital.
• Hazardous Materials (Hazmat) Incidents: Mishaps occurring as a result of the release or spillage of hazardous materials involve a high risk to public health and the environment, given the toxicity or radioactivity of the scenario produced by the chemicals, gases or radioactive substances released. The FRs trained to assist in these rescue tasks have the objective of controlling and decontaminating the hazardous materials as well as managing the relevant health care.

In order to design a system that caters to the diverse requirements of various rescue situations, three distinct operational scenarios have been distinguished:

• Survival: In this scenario, the survival of the First Responder (FR) is prioritized, so the amount of information exposed at the interface is minimum quantity to avoid overwhelming the FR with unnecessary information details and focus on what is essential for survival. The main purpose of this information is to facilitate the FR’s immediate exit from the scenario.
• Search: This scenario is characterized by the fact that the additional information facilitates the search for victims within a rescue scenario. As this is an a priori activity, the amount of information provided is of a medium level.
• Complete: This scenario differs from the previous scenarios in that it is a non-threatening situation, where the FR is fully capable of receiving complete mission status information. That is, it is considered the maximum amount of information that an FR can receive in a rescue scenario.

The primary objective is to determine unique combinations of awareness modules tailored to each scenario, optimizing the FR’s response capabilities and ensuring the effective handling of various rescue situations.

2.3. Smart Helmet and Display Interface

The smart helmet combines a one-eyed transparent display designed to visualize augmented reality information on top of the firefighter’s view. Figure 1a depicts the prototype generated within the European project, where the smart helmet is connected to several sensors such as a thermal camera and a GoPro attached to the helmet structure. Its main objective is to provide additional information to the firefighter during missions without compromising the integrity of their original equipment or obstructing their sight.

The display primarily functions as a screen with the peculiarity that black pixels are transparent to the viewer. The information is rendered inside the display using the AR Interface. This interface was programmed using Angular, as detailed in [10], and has been designed modularly. This allows information from different sources to be displayed or removed independently, considering the data required for each situation based on feedback from the FRs.

The need to adapt the quantity of information displayed in the interface during a realistic rescue scenario has been highlighted by the FRs [9,10]. In this context, approximate measurements of cognitive load to regulate the quantity of information have been analyzed [20]. However, which awareness modules should be prioritized or eliminated according to the requirements of the end-users was not explored. The main goal of this study has been to establish a standard modularity of awareness modules according to the distinct information needs defined by the different FRs, depending on their roles and rescue task configurations.

2.4. Survey

In the questionnaire, respondents were asked about the types of missions in which they typically participated.

Table A2. Number of participants in the survey grouped by indicated types of mission.

Type of Mission	Number of Participants
Emergency and Medical System	15
Firefighter	1
Firefighter, Search Rescue	2
Firefighter, Search and Rescue, Hazmat Incidents	1
Firefighter, Search and Rescue, Natural Disasters	10
Hazmat Incidents	1
Search and Rescue	1
Search and Rescue, Emergency and Medical System	1
Search and Rescue, Emergency and Medical System, Natural Disasters	1

summarizes the multi-choice responses provided by these individuals. As shown in Table A2, there are only two groups or combinations of missions that can be considered representative, as in all other configurations, there are at most two FRs. Therefore, the responses have been categorized by groups of missions, duplicating and triplicating the responses of those FRs who indicated their participation in more than one mission. With this preprocessing of the data, the number of respondents according to mission type is summarized in the following

Table 1. Number of participants on emergency response categories.

Category	Number
Firefighter	10
Search and Rescue	14
Natural Disasters	12
Emergency and Medical System	17
Hazmat Incidents	2

.

The survey was structured into three repetitive modules, which are each associated with scenarios distinguished by the survival risk or danger levels defined in Section 2.2: Survival, Search, and Complete. Each of the modules was configured by the same questions, as outlined in Appendix B Figure A1. The initial closed three questions aimed to quantify or measure the different criteria for prioritizing the awareness modules based on rescue profiles and tasks, while the last open question goal was to collect additional suggestions made by the FRs:

1.

How frequently should the following information appear?

• 1 = Never, 2 = Only if there is anomaly info, 3 = Constantly

2.

What is the maximum number of functionalities visible at one time?

• Select a number between 1–13. Max number = _____________

3.

What is the priority order of the functionalities to be shown?

• Order from 1 to Max number for the following list of functionalities.

4.

What else would you add, remove or change?

• Include biosignals of paired FR, or the level of the oxygen tank.

3. Results

This section presents the results obtained from the surveys, which were organized according to two main characteristics: the type of FR mission and the hazard scenario. The results are organized according to the order in which the questionnaire was completed: Survival, Search, and Complete. It should be noted that all categorical variables corresponding to Questions 1–3 have been converted into numerical forms to save space in the surveys (see Figure A1) and in the results (see, for instance, Figure 2). On the other hand, the comments and suggestions provided in response to Question 4, What else would you add, remove, or change? are discussed in detail in Section 4, as they cannot be translated into numerical form.

3.1. Survival

The answers to Questions 1–3 collected in the survival scenario have been displayed in Figure 2, Figure 3, Figure 4, Figure 5 and Figure 6, and their respective statistics have been collected in

Table 2. Statistics of appearing frequency results corresponding to all participants in survival mode.

Awareness Module	Mean	Std
Robust vision	2.42	0.71
Breath rate	2.36	0.70
Heart rate	2.36	0.70
Battery level	2.33	0.61
Radar	2.31	0.74
C2-FR connection	2.29	0.76
Augmented olfaction tool	2.15	0.65
People	2.05	0.83
Temperature, humidity	2.05	0.80
Signs of life	2.00	0.84
Wireless finder	1.84	0.74

,

Table 3. Statistics of number of awareness modules according to mission types in survival mode.

Mission Type	Mean	Std
Emergency and Medical System (EMS)	6.47	3.24
Firefighter (F)	5.35	1.80
Natural Disasters (ND)	5.96	3.35
Search and Rescue (SR)	6.46	3.39
Hazmat Incidents (HI)	9.50	7.78

and

Table 4. Statistics of priority ordering results corresponding to all participants in survival mode.

Awareness Module	Mean	Std
Heart rate	3.47	2.71
Radar	3.71	2.48
Breath rate	4.08	2.33
Augmented olfaction tool	4.23	2.68
Robust vision	4.73	3.08
Temperature, humidity	4.88	2.78
People	4.96	2.97
C2-FR connection	5.03	2.90
Signs of life	5.19	2.88
Battery level	5.51	2.28
Wireless finder	7.00	3.14

. The results corresponding to Questions 1 and 3 have been contrasted collectively (see Figure 2 and Figure 5) and segmented according to task types (see Figure 3 and Figure 6).

For the sake of simplicity, the frequency of occurrence of the awareness modules has been limited to three responses: “Never”, “Only if there is anomaly info”, and “Constantly”. Figure 2 and Table 2 collect the visual and statistical distribution of these responses by First Responders. Specifically, Table 2 collects the mean and variance of the awareness modules in descending order, indicating that the Robust vision is the tool that should appear more frequently in the survival mode. At the same time, the Wireless finder is the one that should appear less frequently. This descending order based on mean metric has characterized the vertical ordering appearance of awareness modules in Figure 2.

According to Table 2, the mean of appearing frequency of all tools is between the values $[1.84,2.42]$, indicating that the appearance criterion of the tools, in general, would be “Only if there is anomaly info”. However, looking at Figure 2 in detail, regarding the Robust vision, Breath rate, Heart rate, and C2-FR connection tools parameters, the “Constantly” option accumulates more votes than “Only if there is anomaly info”. In the case of the Battery level, Augmented olfaction tool, and Wireless finder, the majority of votes indicate that the most appropriate frequency of appearance of these tools in survival circumstances should be “Only if there is anomaly info”. Finally, according to these data, the modules Black box info (people, temperature, humidity) and Signs of life “Never” should appear in these circumstances.

However, the configuration of these responses may vary according to the roles or tasks that FRs have in rescue operations. Figure 3 contains the answers to how often the awareness modules should appear in survival mode. Still, the results have been distinguished according to the tasks of the FRs: Emergency and Medical System, Firefighter, Natural Disasters, Search and Rescue, and Hazmat Incidents. In Figure 3, the sizes of the circles are proportional to the number of FRs surveyed. As summarized in Table 1, the number of EMS FRs is larger than the rest of the roles, especially compared to Hazmat Incidents FRs, in which two responses exist. Therefore, the size of the circles is representative only of those with the same colors.

Upon examining the responses provided by the Hazmat Incidents FRs, it is observed that the only modules that would be constantly activated are Robust vision, C2-FR connection, Augmented olfaction tool, and People Quantity (Black box). The rest of the modules should be activated “Only if there is anomaly info”. On the other hand, if the focus is on EMS FRs, there is a wide variety of opinions. The Breath rate, Heart rate, Battery level, C2-FR connection, and Augmented olfaction tool modules should be turned on “Constantly” or “Only if there is anomaly info”. For the rest of the tools, there is a general discrepancy. The distribution of responses corresponding to the Firefighter, Natural Disasters, and Search and Rescue tasks are similar: Robust Vision, Breath Rate, Heart Rate, Battery level, Radar, and C2-FR connection should appear in the interface “Constantly” or “Only if there is anomaly info”, while for the rest of the tools, there are scattered opinions.

Figure 4 and Table 3 show the number of modules FRs would deploy at the interface in a survival situation. Firefighters and Natural Disaster specialists have a more unified opinion: they would deploy between four and six modules. Indeed, the Firefighter responses have the lowest mean and std as summarized in Table 3. Search and Rescue specialists would concentrate on between four and eight, while EMS FRs have a more dispersed opinion, indicating that they would display between four and ten modules. The two Hazmat Incidents specialists also disagree, indicating either six or 12 modules, corresponding to the highest mean and std in Table 3.

The order of priority assigned by the FRs to the different modules is shown in Figure 5 and Figure 6 as well as in Table 4 and

Table 5. Pondered weight for priority ordering results configured in survival mode.

Awareness Module	Emergency and Medical System	Hazmat Incidents	Natural Disasters	Search and Rescue	Firefighter
Aug. olfaction tool	3.07	1.0	5.40	5.00	6.25
Battery level	6.20	8.0	5.00	5.40	5.00
Breath rate	4.33	3.0	3.57	4.67	3.57
C2-FR connection	5.54	9.0	4.38	4.80	4.57
Heart rate	4.00	2.0	2.71	4.11	2.71
People	5.30	4.0	4.80	4.86	4.80
Radar	4.67	9.0	2.57	3.00	2.83
Robust vision	6.25	2.0	3.83	4.12	3.83
Signs of life	4.58	5.0	6.00	5.33	6.00
Temp., humidity	4.83	7.0	4.67	5.00	4.67
Wireless finder	7.00	6.0	7.50	6.50	7.50

. Table 4 collects the mean and std of prioritization criteria assigned to distinct modules by the FRs, which are ordered in a descending manner based on mean statistics. This module ordering has been extended to Figure 5 and Figure 6, where the results have been grouped according to the mission types. While Figure 5 summarizes the general information prioritization, Figure 6 provides more narrow information through task segmentation. Table 5 summarizes the priority ordering concluded for the distinct rescue task. In order to extract numerically comparable information, in Table 5, the weighted priorities (the average of the priority ordering) assigned to the different awareness modules according to the different task profiles have been collected. Finally, the graphs in Figure 7a–e show the distribution of these weighted priorities for each task. The contraction/expansion metric is the inverse of the weighted priority (displayed on the edges), so that the highest priority modules are near the center, and the lowest priority modules are at the periphery.

Suppose we observe the priority distribution assigned by the EMS FRs to the different modules. In that case, we can see that there are more data than in the rest of the tasks for two reasons: it is the task that has more representatives (see Table 1) and it is one of the profiles that would show more tools in this survival situation (see Table 3). The modules with the highest priority for this profile are Heart rate, Breath rate and environmental information of the Black Box (temperature, humidity). The lowest weighted priority values also correspond to these three modules, as shown in Table 5 and in Figure 7a. On the contrary, the awareness modules with the highest weighted priority values, and therefore, the least prioritized, are Battery level, Robust vision and Wireless finder. In the graph visualized in Figure 7a are the nodes farthest from the center with the lowest shrinkage values between 0.14 and 0.17.

The priority results associated with the Hazmat Incidents task are almost invisible in Figure 6 due to being two unique representatives. The resulting average priority ordering is more clear, as values spread over a bigger range of values (from 1 to 9) as detailed in Table 5 and depicted in Figure 7b. Augmented olfaction, Robust vision and Heart and Breath rates are the most prioritized modules, while Battery level, Radar and C2-FR connection are the less highlighted ones.

The modules that the FRs with Natural Disasters profile have prioritized are the Heart Rate and Radar, according to the weighted priorities in Table 5, and these are visualized in Figure 7c. However, the modules they prioritize the least are the Wireless finder and Augmented olfaction tool.

The priorities indicated by the FRs with a Search and Rescue profile have similar results to those of Natural Disaster tasks, as seen in Figure 6, but with more variability. The overall distribution in Figure 6 shows that the Battery level and C2-FR connection data have the most variability. The result of this variability is seen in Table 5 and in the graph in Figure 7d, as Heart Rate and Radar had a contraction coefficient of 0.5 in the graph in Figure 7c, while in Figure 7d, the Battery level coefficient has dropped to 0.33, and the Breath rate coefficient has dropped to 0.25. The main observation is that the priority order is less clear with more data variability.

Finally, the weighted priority corresponding to firefighters, summarized in the last column of Table 5, provides almost identical results as the Natural Disasters profile. The resemblance is evident in Figure 7c–e.

3.2. Rescue

The rescue scenario is characterized by the fact that the objective of the task is the rescue of the victims. In general, the results collected in Figure 8 and

Table 6. Statistics of appearing frequency results corresponding to all participants in rescue mode.

Awareness Module	Mean	Std
Signs of life	2.76	0.43
Robust vision	2.69	0.57
Wireless finder	2.60	0.49
Radar	2.58	0.50
People	2.51	0.72
C2-FR connection	2.44	0.60
Heart rate	2.35	0.64
Battery level	2.33	0.47
Breath rate	2.29	0.63
Augmented olfaction tool	2.15	0.76
Temperature, humidity	2.15	0.62

indicate that the average frequency with which awareness modules should appear in the rescue scenario is higher than in the survival scenario. Tools that help to identify victims (Signs of life, Robust vision, Wireless finder, Radar and People) should appear Constantly. Figure 9 extends these results grouping them by mission task and confirms that regardless of the profiles, the modules that should appear constantly are those previously indicated. For the rest of the modules, there are divided opinions: they should appear “Constantly” or “Only if there is anomaly info”.

The number of modules that should appear in rescue circumstances has increased to nine modules according to most profiles (see

Table 7. Statistics of number of awareness modules according to mission types in rescue mode.

Mission Type	Mean	Std
Emergency and Medical System (EMS)	9.26	4.27
Firefighter (F)	7.95	4.18
Natural Disasters (ND)	9.12	4.67
Search and Rescue (SR)	9.46	4.59
Hazmat Incidents (HI)	10.00	7.07

and Figure 10). Firefighters would reduce this number to 8, and Hazmat Incident FRs would increase to it 10. However, the variability of the responses indicates that the number of modules to be displayed in the interface could be reduced to 6 or up to 14 modules depending on the different users.

According to the average order of priority they would assign to each module (summarized in

Table 8. Statistics of priority ordering results corresponding to all participants in rescue mode.

Awareness Module	Mean	Std
People	3.46	1.85
Signs of life	4.34	2.67
Radar	4.58	2.18
Robust vision	4.59	2.27
Heart rate	4.71	3.07
C2-FR connection	4.94	3.32
Wireless finder	5.34	2.79
Breath rate	5.50	2.81
Augmented olfaction tool	6.00	3.35
Battery level	6.30	1.96
Temperature, humidity	6.33	2.30

and Figure 11), the priority modules would be People, Signs of life, Radar and Robust vision, following that order, and the tools that provide information, contextual, temperature or battery level would be the lowest priority. Looking at these results grouped by task in Figure 12 and quantified in

Table 9. Pondered weight for priority ordering results configured in rescue mode.

Awareness Module	Hazmat Incidents	Emergency & Medical System	Search & Rescue	Firefighter	Natural Disasters
Aug. olfaction tool	1.0	3.70	8.00	9.33	9.33
Battery level	8.0	6.56	6.00	6.00	6.00
Breath rate	5.0	6.00	5.33	4.50	4.50
C2-FR connection	6.0	6.18	4.43	4.00	4.00
Heart rate	5.0	5.70	4.25	3.33	3.33
People	2.0	3.67	3.38	3.43	3.43
Radar	3.0	4.92	4.57	4.33	4.33
Robust vision	2.0	5.50	4.43	4.00	4.00
Signs of life	2.0	4.67	4.43	4.17	4.17
Temperature, humidity	6.0	5.70	6.33	8.00	8.00
Wireless finder	1.0	5.92	5.29	5.17	5.17

, the prioritization of the modules varies quite a lot according to the different tasks. The People, Radar and Signs of life modules are the most prioritized tools in all cases; the Augmented olfaction module is important for EMS and Hazmat Incident FRs, while not for the rest, and inversely with C2-FR connection and Heart rate; Robust vision is quite high priority for all but EMS FRs, while Wireless finder is essential only for Hazmat Incidents. The lowest priorities overall are Battery level, contextual information (ambient Temperature, humidity) and Breath rate.

The distribution of weighted priorities is more evident in the graphs in Figure 13a–e, where the visual analysis is performed by task.

3.3. Complete

In the complete scenario illustrated in Figure 14 and detailed in

Table 10. Statistics of appearing frequency results corresponding to all participants in complete mode.

Awareness Module	Mean	Std
Robust vision	2.89	0.31
Signs of life	2.73	0.56
Wireless finder	2.67	0.58
Battery level	2.62	0.59
Radar	2.56	0.60
C2-FR connection	2.55	0.60
Augmented olfaction tool	2.45	0.50
People	2.44	0.66
Heart rate	2.38	0.78
Temperature, humidity	2.35	0.64
Breath rate	2.31	0.79

, according to the responses, the majority of the awareness modules should be continuously displayed. The modules Augmented olfaction, People, C2-FR connection and Temperature and humidity show variability between two options: to show information “Constantly” or “Only if there is anomalous information”. The extension of this information analyzed by task depicted in Figure 15 confirms this conclusion about the frequency of occurrence of the awareness modules. The additional information provided by this figure is the visualization of rare situations where users have indicated that the frequency of appearance of some modules should be Never.

The number of modules that should be deployed in the interface in general is around 9, as can be seen in Figure 16 and

Table 11. Number of awareness modules according to mission types in complete mode.

Mission Type	Mean	Std
Emergency and Medical System (EMS)	8.03	2.44
Firefighter (F)	8.80	4.32
Natural Disasters (NDs)	9.83	4.59
Search and Rescue (SR)	10.07	4.48
Hazmat Incidents (HIs)	13.00	2.83

. The EMS FR profile reduces its average to 8, and the Hazmat Incidents go up to 13 with respect to the numbers collected in the rescue scenario shown in Table 7.

The prioritization statistics in

Table 12. Statistics of priority ordering results corresponding to all participants in complete mode.

Awareness Module	Mean	Std
Signs of life	3.81	1.96
People	3.97	2.48
Robust Vision	4.29	2.63
Wireless finder	4.52	2.26
Radar	4.53	2.32
Augmented olfaction tool	4.93	2.85
C2-FR connection	4.93	3.04
Battery level	6.00	2.07
Heart rate	6.00	3.24
Temperature, humidity	6.14	2.71
Breath rate	6.59	2.94

and Figure 17 also resemble the results of the rescue scenario. The prioritized modules are those related to victim detection, such as Signs of life, People, Robust vision, Wireless finder and Radar. Analyzing these results in more detail in Figure 18 and

Table 13. Pondered weight for priority ordering results configured in complete mode.

Awareness Module	Emergency and Medical System	Hazmat Incidents	Firefighter	Natural Disasters	Search and Rescue
Augmented olfaction tool	4.15	1.0	5.80	5.80	5.83
Battery level	6.45	4.0	5.83	5.83	5.86
Breath rate	6.45	1.0	7.33	7.33	7.25
C2-FR connection	5.70	3.0	4.50	4.50	4.86
Heart rate	6.08	1.0	6.25	6.25	6.40
People	3.86	2.0	4.17	4.17	4.14
Radar	4.88	3.0	4.00	4.00	4.60
Robust Vision	5.57	1.0	3.33	3.33	3.86
Signs of life	4.73	2.0	2.80	2.80	3.50
Temperature, humidity	5.60	3.0	7.33	7.33	6.50
Wireless finder	5.57	1.0	3.60	3.60	4.17

, where the priority order is arranged by tasks, it is observed that firefighters, Natural Disaster and Search and Rescue FRs share the same priority results, where the Signs of life module is given a little more priority by Firefighters and Natural Disaster FRs. The other prioritized modules are Robust vision, Radar, People and C2-FR connection. EMS FRs, on the other hand, prioritize the following awareness modules: People, Augmented olfaction, Signs of life and Radar. Finally, Hazmat Incident FRs prioritize Augmented olfaction, Breath and heart rate, Robust vision and Wireless finder. These distributions are even more evident in the task graphs visualized in Figure 19.

4. Discussion

The objective of this section is to recognize the difficulties encountered in determining and validating the appropriate interface configuration for different tasks and situations.

On the one hand, the amount of data available is too limited to draw definitive conclusions. However, initial processing has been carried out to ensure that the different tasks considered are sufficiently represented. This also responds to the reality that a First Responder is often involved in different tasks depending on the region, administration and profession. As a consequence, it is observed that the preference results for firefighters and natural disaster professionals are identical, and in the case of rescue professionals, almost identical. For the task distinction to make sense in this context, the number of professionals per task should be larger, and their responses should be collected in separate rounds. In this survey, the results of the Hazmat Incident FRs are the least significant, as they consist of only two respondents.

On the other hand, those who perform different rescue tasks (in buildings, mountains or tunnels) have indicated that, depending on the nature of the rescue, certain modules would make more sense than others (the augmented olfaction module or the Black box contextual information would not make sense in the mountain, while in a tunnel or building, it may be essential). Therefore, the strategy to divide the display configuration according to different task has been well considered.

In general, there has been a general controversy regarding the number of scenarios (survival, rescue, full) considered. Some respondents have indicated their disagreement with the number of scenarios: many EMS have suggested considering only one scenario, since their collaboration in rescue tasks does not include the rescue of the victim and, therefore, no distinction is made between scenarios where their life may be in danger. In this sense, the first step would be to conduct a preliminary survey to determine a number of scenarios for each type of task with objective characteristics and contrasted by the professionals. The second step would be to administer the same survey, considering the different scenarios, to a significant group of professionals.

As for the conclusions, which should be contrasted with larger representative groups, they indicate that the firefighter profile tends to visualize a smaller number of modules, while the EMS FRs choose to deploy a larger number of tools in the different rescue scenarios. The other profiles in general have chosen intermediate quantities. However, in the complete mode, the EMS FRs have indicated a lower number of tools to deploy than the other profiles. When analyzing the results, we have seen that there were some surveys with this section empty, and the impact on the results is evident.

Another noticeable detail in the distribution of the data is that the EMS FRs have more diverse opinions, and therefore, a greater variability in the response data. In contrast, the other profiles show less variability and more conclusive data. This is essential for extracting useful information, such as the order of priority of the modules. Even if a ranking is performed based on average, a large variability in the data implies that there is no consensus. Therefore, instead of generalizing the interface, it should be kept flexible and adapted individually. If there is unanimity, standard interfaces could be generated.

The different profiles of First Responders have shown an interest in proposing information not considered in the survey to facilitate the rescue tasks. On the one hand, some surveyed firefighters have also indicated that they would prioritize having access to the biosignals of their fellow rescuers. Especially in rescue operations where respiratory support is used, communication with the partner is often limited, and they consider it more important to know the partner’s vital information than their own. They also indicated that displaying the oxygen tank level on the interface would be very helpful. The firefighters also indicated that it would be very useful to store the traceability of their rescue operations so that when the signal of any rescuer is lost, the point where the signal was lost and the way to reach that location can be quickly located.

On the other hand, the EMS FRs have indicated that their own biosignals do not seem important to them, as they work in situations where their life is not in danger. They have prioritized this type of information in the design (heart and respiratory rate) because they consider that it could be essential for those professionals working in victim rescue. For their part, they showed that for the health-care team, it would be interesting to see the constants of patients or victims.

Another feature to take into account are the proposals made by the FRs regarding the hierarchy of roles and tasks. For example, firefighters entering a building should have an interface that displays modules focused on victim detection, while those leading these intervention groups should have another interface that includes the FRs’ biosignals as well as their traceability and victim information. The senior leaders who manage all resources should have access to all information.

5. Conclusions

One of the main conclusions of this study is that the flexibility and modularity of an interface should not be replaced by standardized models unless they are based on robust results that converge toward a concrete and concise list of priorities and module selection. The main conclusions regarding information prioritization for each mission mode are summarized in

Table 14. Summary of survival, rescue, and complete scenarios.

Question	Survival Scenario	Rescue Scenario	Complete Scenario
1. How frequently should the following information appear?	- Most tools: “Only if anomaly” - Robust vision, Heart rate, Breath rate: “Constantly” - Wireless Finder, Black box info: “Never”	- Higher frequency than survival - Signs of life, Robust vision, Wireless finder: “Constantly” - Varied opinions on other tools	- Majority: “Constantly” - Augmented Olfaction, People, C2-FR Connection, Temp/Humidity: “Constantly” or “Only if anomaly” - Rare cases: “Never”
2. Maximum number of functionalities visible at one time?	- Firefighters: 4–6 - Search and Rescue: 4–8 - EMS: 4–10 - Hazmat Incidents: 6 or 12	- Generally higher than survival - Firefighters: 8 - Other tasks: 9–10 - Variability: 6–14	- Around 9 on average - EMS: 8 - Hazmat Incidents: 13 - Other tasks: 8–10
3. Priority order of functionalities to be shown?	1. Heart rate 2. Radar 3. Breath rate 4. Augmented olfaction tool … 11. Wireless finder	1. People 2. Signs of life 3. Radar 4. Robust vision … 11. Temperature, humidity	1. Signs of life 2. People 3. Robust vision 4. Wireless finder … 11. Breath rate
4. What else would you add, remove, or change?	- No specific additions/removals mentioned. Focus on varied tools depending on tasks.	- Similar diversity in priorities - Augmented olfaction important for EMS and Hazmat, less so for others.	- Priority varies by task - EMS: People, Augmented olfaction - Hazmat: Augmented olfaction, Breath, Heart rate - General: Focus on victim detection

, but in order to define standardized configurations according to types of missions, roles, and scenarios, it is necessary to have a group of professionals who, in addition to being representative, are familiar with the situational awareness modules investigated in the present European Rescuer project.

To optimize efforts, constant interaction with the different FR profiles is recommended so that both the interface design and the surveys to adapt its configuration are aligned with the diverse realities faced by FRs in the different rescue scenarios. This will save time and resources in generating prototypes that fit the needs of the end-users.

Despite the discrepancy in converging toward standardized interfaces, it is worth noting that the experience of collecting feedback from the different FRs has been very productive, since the professionals themselves contributed many ideas for improvement. Moreover, the existence of this type of tool is beyond their imagination. Therefore, this type of project helps to generate a bridge between technology developers and rescue professionals. In addition, the rescue in the dark scenario is a realistic simulation to represent low visibility circumstances, but consideration should be given to adding some factor that generates a human stress response to simulate a realistic cognitive load so that the interface can be designed more appropriately. This could be achieved by developing additional technologies, such as virtual reality crisis simulations, to test the interface design in more immersive rescue scenarios.

It is also important to mention the concern shown by some respondents about the implicit disadvantages of using these types of technologies. They indicated that the use of these tools could replace human references (collected through touch, smell and other senses) and that, in the event of blows or circumstances in which the technology becomes useless, the FR could find itself in more vulnerable situations than if it had used these references.

In conclusion, despite the difficulties encountered in reaching a consensus on a standardized interface, through collaborative work with FRs, it is possible to design interfaces for smart helmets that help different professionals to perform rescue operations with more information than is available to them today.