“Share Your Tools”—A Utility Study of a Norwegian Wildland-Fire Collaboration Exercise

Abstract

Based on the assumption that crisis collaboration exercises lead to better team-integration and more efficient problem solving, the aim of this study is to test whether there is a relationship between exercise participation and perceived levels of learning and utility. Online survey data was collected from participants in a 2018 two-day, full-scale, wildland-fire collaboration exercise in southeastern Norway. The instrument of choice was the collaboration, learning, and utility (CLU) scale. Findings indicate a strong covariation between participation in Norwegian wildland-fire collaboration exercises and the perceived level of learning, with a medium to small covariation between perceived learning and utility. The results indicate the importance of giving clear instructions, focus on collaboration, and sufficient forms of discussion during and after the exercise in order to gain learning. However, learning had a limited impact on utility. The study indicates joint evaluations, improvising, and testing of new and alternative strategies across sectors are important when exercises are constructed. The data was retrieved from a questionnaire, observations and interviews can add more and comprehensive insight into the studied phenomenon.

1. Introduction

In 1996, Karl Weick [1] published his allegory of organizational studies “Drop your Tools.” In his paper, Weick points to 10 possible reasons why 27 wildland firefighters lost their lives when they failed to follow orders to drop their tools and run when overrun by exploding fires. His study is based on values developed by James D. Thompson [2] and data collected from two main wildland fire disasters: Mann Gulch in Montana (1949) and South Canyon in Colorado (1994). Weick shows that the willingness of endangered firefighters to leave equipment behind and run was overdetermined due to deeply rooted routines and identity. An important conclusion was that fighting wildland-fires requires skills in cross-sectional collaboration to break familiar and iterative routines during extra-ordinary situations. Such skills develop through collaboration exercises. There exists an assumption in emergency preparedness that exercises lead to better team-integration and more efficient problem solving. However, though few in number, all relevant international studies indicate that the perceived effects of exercises by participants are rather limited [3,4,5,6,7].

As the effects of collaboration exercises and specially wildland-fire collaboration exercises are little researched, this should be considered a gap in the collaboration literature. As a contribution to closing this gap, this study focuses on the perceived effects of such exercises with an emphasis on learning and usefulness. Further, this study has a practical utility value, as it defines perceived problem areas, which can help exercise organizers and managers to further develop collaboration learning and usefulness. The following research questions were developed: (1) To what extent is there a relationship between participation in Norwegian wildland-fire collaboration exercises and perceived level of learning? (2) To what extent is there a relationship between participation in Norwegian wildland-fire collaboration exercises and perceived level of utility? A collaboration exercise is here defined as an exercise where multiple emergency stakeholders come together to develop preparedness, team-integration, and behavioral response [8]. In Norway, where this data is collected, collaboration is one out of four national emergency preparedness principles. The principle imposes on all relevant emergency stakeholders the commitment to ensure the best possible collaboration with other relevant actors across all phases of emergency preparedness [9]. The three other principles of responsibility, equality, and proximity are also formalized in the other Scandinavian countries, but thus far Norway is the only country that has adopted collaboration by law. This paper first starts out by reviewing literature on collaboration, learning, and utility. Second, it outlines the study’s materials and methods, before summarizing and presenting the study’s results. The discussion section then describes and interprets the findings, with the final section summarizing the answers to the stated research questions, outlining the theoretical and practical implications, and making recommendations.

1.1. Collaboration

Collaboration is here defined as a horizontal process where stakeholders, both at the organizational and individual level, share competences, unite resources, and unprestigiously work together towards a common goal. [10]. Compared with coordination and cooperation, collaboration calls for more frequent interaction, higher embeddedness, and a larger will of risk sharing [11]. There are multiple motives for engaging in collaborative processes, but possible explanations include an overall assessment of advantages vs. disadvantages [3], a desire for individual or social benefits [12], and the sharing of risks [13]. As a working form, it has over the years become popular across multiple fields and branches and received considerable attention, especially in management literature [14,15,16,17]. Well-functioning collaboration processes are often presented as a solution to task allocation and regulatory fragmentation [18], thus they represent the leading perspectives within fields such as team-development, coaching, and integration [19]. In the field of crisis management, collaboration is viewed as a key success criterion [20,21,22] and has been found to positively affect the overall outcome of a crisis [23]. We define a crisis as a situation or incident that outsources available resources [24] and is not confined within administrative, geographical, or physical boundaries [25]. Collaboration, or more specifically cross-sector collaboration, which is the main focus of this study, is viewed in the crisis literature as a core concern as it helps both crisis managers and societies to effectively deal with adverse consequences [23] and meet societal expectations [26]. On the contrary, if managers fail in their quest, a lack of collaboration may lead to less resilience, flexibility, and efficiency [27]. While there are examples of sudden, informal collaboration processes during major disasters such as Hurricane Katrina [28] and the 2011 terrorist attacks in Norway [29], collaboration is something that, in most cases, needs to be learned, developed, and exercised. Thus, there exists an assumption that cross-sector collaboration exercises develop, train, and test joint preparedness efforts and response [8]. The problem, however, is that recent studies have found that such exercises tend to have limited perceived levels of learning and utility [30,31,32,33,34,35]. Sources today are conflicting as to why this perceived limitation occurs, but cited reasons include a lack of focus on variation [36], lack of trust [37], and insufficient focus on collaboration learning and utility enhancing elements [3,6,7,38]. Collaborative strategies can, however, differ depending on the current situation. On a scale of less to more collaboration sequential, parallel, and synchronous types of collaboration have been identified. Sequential strategies are often used when it is optimal to go through official channels and stick to routines. Parallel routines are when tasks are carried out simultaneously, while acting “on their own”. Such a subtle type of collaboration is used when members do not go in and help each other across. It is characterized by the standardization of developed roles and established procedures. Synchronous collaboration means stepping over the boundary into the unfamiliar and flexibly covering for others where needed, even if this does not lie within a specific area of competence. Synchronous collaboration is the idealized seamless form of collaboration referred to when governing bodies stress their ability to interact, but it is also a challenging and exhausting type of collaboration. These types of collaboration have been identified as useful depending on the current situation. The synchronous type of collaboration is seldom used in everyday practice, but merely when there is a lack of resources such as during mass casualty scenarios [3].

1.2. Learning and Utility

The goal of learning is to acquire new knowledge [39]. The idea of collaboration learning during exercises is, in this study rooted in, and limited to, Johan Stein’s [40] and Klabber′s [41] perspectives on how institutions learn, hence the differences between first- and second-order learning. First-order learning is when participants learn new things during the exercise but are unable or unwilling to transfer knowledge to practice. Second-order learning, on the other hand, is when participants manage to acquire new knowledge and apply that knowledge in real situations [31]. While the goal of obtaining new collaboration skills and understanding during exercises may seem obvious, Berlin and Carlström [3] found that while it was considered theoretically and socially correct to support collaboration engaging processes, participants, in practice, prefer their everyday, standardized working patterns, which results in a decoupling between theoretical structures and practice [42]. To increase exercise utility, planners should focus more on integrating collaboration-learning elements, hence creating new configurations of thoughts that bridge exercise learning and real life practice [43]. Bourgeois et al. [44] discovered that the promotion of collaboration enhancing factors increased the sense of team-belonging and the ability to learn, thus substantiating Borell and Eriksson’s [36] later argument on how perceived crisis learning is greatly dependent on the design and applied exercise model. When designing an exercise, there are multiple ways to enhance both collaboration learning and exercise utility. Firstly, the exercise needs to have a purpose and primary objective [45], hence the exercise needs a clearly defined training content with associated learning outcomes [46]. Participants must be informed that the main goal is collaboration development, not only complex scenario solving. Secondly, the organizers need to provide clear collaborative instructions and ensure that the exercise participation feels relevant and is free from long or unnecessary waiting periods [31]. Thirdly, it is important that all participants at all times have an overview of the ongoing scenario development [45] and feel that their opinions matter. Moreover, they must be included in formative assessment and collective reflection processes throughout all stages of the exercise [39]. On that note, organizers and managers also need to take into consideration that collaboration learning processes are not always either black or white. Some professional boundaries are, and will always be non-negotiable due to e.g., jurisdiction or complexity of task [47].

2. Materials and Methods

This study reports on data collected from a 2018, two-day, full-scale, wildland-fire collaboration exercise in southeastern Norway. The main goal of the exercise was collaboration development, but included also technical, logistical and managerial elements. Participants included the Norwegian Civil Defense, County Wildland-Fire troops, local fire and rescue personnel, the Norwegian Directorate for Civil Protection, and local fire planes and helicopters. The exercise scenario ran from alarm to completion and included air- and land-based fire extinguishing, controlling of fires, and establishment of fire ditches. Logistical exercise elements included crew reception, parking, and housing/catering establishment. The exercise planning and directing staff was composed of senior representatives from the participating organizations. There were no performed joint evaluations following the exercise. The exercise was conducted in May, and data was collected in early fall. The exercise had 184 participants (N = 184) representing local full-time and part-time fire and rescue services, regional wildland-fire troops, civil defense, and wildland-fire planes and helicopter personnel.

The sample included both the operational and tactical level personnel. The selection of sample participants was based on an assumption that relevant personnel in need of collaboration exercise participated, and that the exercise had a relevant and clearly defined collaborative purpose and primary objective.

2.1. Data Collection and Procedures

A G*Power 3.13 analysis [48] calculated the appropriate sample size to be 82 participants. A t-test, linear bivariate regression—one group, two-tailed, with an alpha significance level of 0.05 [49], a statistical power of 0.80, and an effect size of 0.3—was applied. The collection of data occurred through the use of a validated online survey instrument. The collaboration, learning, and utility scale (CLU-scale) [31] became the instrument of choice as it is especially designed to measure the perceived effects of collaboration exercises, with an emphasis on learning and usefulness (

Table 1. The collaboration, learning, utility scale (CLU-scale).

C	The exercises were focused on collaboration
C	Sufficient forms of discussions were provided
C	There were opportunities to improvise
C	Personnel in need of exercise participated
C	I performed well-known activities
C	Collaboration was initiated immediately
C	Clear instructions of collaboration were presented
C	My points of view were regarded
L	I learned new things during the exercise
L	I learned about other’s organizational aspects
L	I learned about other’s communication patterns
L	I learned about other’s prioritizing of activities
L	I learned other’s concepts and abbreviations
U	Based on what I learned, the exercises were useful to real-life activities
U	Based on what I learned, the exercises were useful to command officers
U	Based on what I learned, the exercises were useful to ordinary operative staff
U	Based on what I learned, the experiences from the exercises impact my daily work

Dimensions: Collaboration (C), Learning (L), Usefulness (U). Source: Berlin and Carlström [31].

). Also, as the CLU has been applied in similar studies [7,31,32], a comparison with other collaboration exercises are made possible. CLU is a Swedish developed survey tool. An expert group of five academic instrument-developing experts together with three emergency practitioners, representing blue-light response organizations, developed it. The instrument was developed in different stages based on Meyer and Rowan′s 1977 decoupling theory [42], Berlin and Carlström′s theories on sequential, parallel, and synchronous collaboration [50], and Stein′s [40] learning theories, which have their outspring from Klabber′s [41] perspectives on how institutions learn, hence the differences between first- and second-order learning. Before completion, the CLU-scale was tested in multiple pilot-studies. The final product consisted of 17 items measuring the three dimensions collaboration (C), learning (L), and utility (U). The C dimension measures the perceived collaboration characteristics, the L dimension emphasizes collaboration related lessons, and the U concerns transfer of value to real-life scenarios. The CLU-scale is based on a 5-point Likert scale with the values 1 (strongly disagree), 2 (mildly disagree), 3 (neutral), 4 (mildly agree), and 5 (strongly agree). The instrument’s homogeneity was tested through a calculation of Cronbach’s alpha, showing an alpha of 0.88 [51]. Analysis stems from descriptive data and bivariate and multiple regressions. Means and standard deviations were included for descriptive purposes. The instrument has earlier been applied in multiple, similar studies [7,30,31,32,33,52]. To ensure appliance with ethical research standards, the researchers sought permission from the Norwegian Center for Research Data (NSD) prior to data collection.

Prior to contacting the defined sample population, permission was sought from the participating organizations. The email contained information about the study, methods, instrument, and purpose. The letter emphasized volunteerism and assurance of anonymity. After obtaining written permissions and email addresses from the organizations, an invitation to participate was sent out to the population sample. Apart from a hyperlink to the survey designed in the online survey platform “Nettskjema” [online-form] hosted by the University in Oslo, the invitation contained information about the study, data handling, and how to contact the researchers. Volunteerism and anonymity were highlighted, as well as the option to at any time withdraw from the study without facing consequences. To ensure further anonymity, only demographical questions related to gender, age, professional experience, and professional affiliation were asked. Age and experience were divided into groupings, and affiliation was limited to public or NGO sectors. Age groupings were 1 = 18–26, 2 = 27–35, 3 = 36–44, 4 = 45–53, 5 = 54–62, and 6 = 62+. Choice of affiliation was 1 = private, 2 = public, and 3 = volunteer. Years of professional experience was divided into 1 = 0–5, 2 = 6–11, 3 = 12–15, 4 = 16–20, and 5 = 21+. Number of collaboration exercises attended over the last five years were divided into 1 = 1–3, 2 = 4–7, 3 = 8–11, and 4 = 11+. The participants were asked to complete the survey within three weeks. During this period, two reminders were sent out. After collection, all data was uploaded into Statistical Package for the Social Sciences (SPSS) and analyzed. Identifiable information in the data set was removed and replaced with a number. The scrambling key was stored on a safe drive at the University of South-Eastern Norway accessible only to the research team. The key and identifiable information were deleted after the completion of this research project.

2.2. Analysis

The analysis process commenced with the exploration of demographical information, followed by an analysis of the collaboration, learning, and usefulness (CLU) characteristics (

Table 2. Mean values—CLU-dimensions.

	Mean Values	Standard Deviation
Collaboration	4.12	0.547
Learning	3.97	0.850
Utility	3.68	0.543

Note: n = 71, sign = p < 0.05.

). Here means and standard deviations were identified for describing data distribution variations [53]. The empirical relationship between the CLU variables were tested by performing two bivariate regression analyses, where the first one tested the relationship between collaboration (C) and learning (L), and the second tested the relationship between learning (L) and usefulness (U). Demographical data was collected exclusively for description purposes, thus not integrated into the regression analysis. The working assumption was that focusing on collaboration enhancing elements during a collaboration exercise leads to collaboration learning, which again leads to usefulness [50]. In this first test, collaboration was defined as an independent variable relative to learning (dependent), while in the second test, learning was the independent variable to usefulness (dependent). Pearson correlation coefficients (Pearson’s r) were calculated to measure the linear dependence between the variables. Coefficients of determination (r2) were calculated to determine what proportions of the variance in the dependent variables could be considered predictable from the independent variables [54]. To test the difference in parameters between the observed results and the stated hypotheses, standard errors and F-values were calculated. Further, the significance level (p-value) was calculated to determine the probability of rejecting the null hypothesis. Statistical significance was set to p < 0.05, and all tests were two-tailed [49]. In the next step, the relevant predictor and criterion variables that were found significant were tested in multiple regression analysis [48]. Here collaboration and learning were used as integrated independent variables relative to usefulness (dependent variable). The standardized coefficient values (beta), and the differences between them, were calculated both for the bivariate and multiple regression analyses to analyze the strength of the effects of each CLU variable. Towards the end, p-values were calculated together with the performance of a Shapiro-Wilk test [55]. The last was done to ensure that the CLU variables met normal assumptions for regression analysis.

3. Results

3.1. Demographics

Seventy-one persons participated in the study; 90.1% percent were males and 9.9% were females. The overall response rate was 38.6%. Most age groups were represented, besides the 62+ group. Most respondents belonged to the 45–53 group (52.1%), while the mean age group was 36–44. Ninety-seven percent of the participants belonged to a public organization, and the number of years of professional experience ranged from the 0–5 group to the 21+ group. Most (38%) belonged to the 0–5 group. The number of collaboration exercises attended over the last five years ranged from the 1–3 group to the 11+ group. The majority (57%) had participated in 1–3 exercises during the last five years.

3.2. Collaboration

In total, 84.3% of the sample population either strongly or mildly agreed that the exercise focused on collaboration; none strongly disagreed (M = 4.43, SD = 0.791). On the question regarding whether sufficient forms of discussions were provided, 14.1% strongly agreed and 35.2% mildly agreed. Further, 15.5% mildly disagreed, while 1.4% strongly disagreed (M = 3.47, SD = 0.985). Over half of the sample population (60.6%) either strongly or mildly felt that there were opportunities to improvise, while 21.1% remained neutral (M = 3.74, SD = 1.112). When it came to perceptions of whether the collaboration had been immediately initiated, most mildly (42.3%) or strongly (36.6%) agreed. None of the participants strongly disagreed (M = 3.93, SD = 1.10). In total, 53.5% strongly agreed that they had performed well-known activities. Nobody mildly or strongly disagreed (M = 4.37, SD = 0.790). A total of 85.9% either strongly (64.8%) or mildly (21.1%) agreed that personnel in need of exercise participated, while 11.3% remained neutral (M = 4.55, SD = 0.697). Of the respondents, 77.5% either mildly or strongly agreed that instructions about collaborative practice were presented during the exercises (M = 4.28, SD = 0.922). Finally, when it came to whether the population perceived that their points of view were regarded, 42.3% strongly agreed, 16.9% mildly agreed, and 40.8% remained neutral (M = 4.01, SD = 0.918). The overall mean for the collaboration dimension was 4.12 (SD = 0.547).

3.3. Learning

Most of the respondents either strongly (69.0%) or mildly (18.3%) agreed that they had learned new things during the exercise (M = 4.51, SD = 0.980). A total of 74.6% either strongly (39.4%) or mildly (35.3%) agreed that they had learned about others’ organizational aspects (M = 4.07, SD = 1.041), while more than half (71.8%) had learned about others’ communication patterns. Here, 19.7% remained neutral (M = 4.03, SD = 1.021). As to whether the exercise participants felt that they had learned about how collaborating organizations prioritized their activities, 7.0% mildly disagreed while 2.8% strongly disagreed (M = 3.77, SD = 1.002). Some 45.1% agreed that they had either strongly (16.9%) or mildly (28.2%) learned new concepts and abbreviations (M = 3.43, SD = 1.050). The overall mean for the learning dimension was 3.97 (SD = 0.850).

3.4. Utility

Most of the sampled population (94.4%) found the exercise useful for real-life activities (M = 4.69, SD = 0.623). While 18.3% strongly and 19.7% mildly agreed that the exercise was useful for commanding officers, 29.6% remained neutral (M = 3.17, SD = 1.248). Some 42.3% either mildly or strongly agreed that the exercises were useful for ordinary operative staff. Here, 42.3% remained neutral (M = 3.40, SD = 0.900). Finally, under half (45.1%) agreed that their experiences of the exercises would affect their daily work, while 33.8% remained neutral (M = 3.40, SD = 1.102). The overall mean for the utility dimension was 3.68 (SD = 0.543).

3.5. Bivariate and Multivariate Regression Analysis.

RQ 1:

To what extent is there a relationship between participation in Norwegian wildland-fire collaboration exercises and perceived level of learning?

Bivariate analysis. The most pronounced significance was found between learning and the item “Clear instructions of collaboration were presented” (r = 0.501, r2 = 0.239, F = 20,761, p ≤ 0.000). The item “The exercises were focused on collaboration” had the second highest pronounced significance (r = 0.451, r2 = 0.191, F = 16.349, p ≤ 0.000), followed by “Sufficient forms of discussions were provided” (r = 0.421, r2 = 0.164, F = 13.564, p ≤ 0.000). “My points of view were regarded” had an R-value of 0.398 (r2 = 0.145, F = 12.034, p ≤ 0.001), while “there were opportunities to improvise” had an R-value of 0.282 (r2 = 0.65, F = 5.508, p ≤ 0.022). “Collaboration was initiated immediately” had somewhat lower significance level (r = 0.255, r2 = 0.050, F = 4.439, p ≤ 0.039). The final item with a pronounced significant was “Personnel in need of exercise participated” with an R-value of 0.253 (r2 = 0.049, F = 4.291, p ≤ 0.042). The item “I performed well-known activities” was found insignificant (r = 0.06, r2 = −0.01, F = 0.30, p = 0.58) and was therefore excluded from the multiple regression analysis (

Table 3. Bivariate regression of the collaboration dimensions of learning.

Dependent Variable: Learning. Independent Variables: Collaborative Characteristics of Exercises
	R	R2	F	Sign (p)
The exercises were focused on collaboration	0.451	0.191	16.349	0.000
Sufficient forms of discussions were provided	0.421	0.164	13.564	0.000
There were opportunities to improvise	0.282	0.065	5.508	0.022
Personnel in need of exercise participated	0.255	0.050	4.439	0.039
I performed well-known activities	0.045	−0.014	0.124	0.726
Collaboration was initiated immediately	0.253	0.049	4.291	0.042
Clear instructions of collaboration were presented	0.501	0.239	20.761	0.000
My points of view were regarded	0.398	0.145	12.034	0.001

Note: N = 71, sig = p ≤ 0.05.

).

Multivariate analysis. The joint collaborative characteristics predicted 29.3% (r2 = 0.293) of the learning variance, meaning that the remaining 70.7% of the predicted variance was unaccounted for. Still, the variables “There were opportunities to improvise,” “Collaboration was initiated immediately,” and “Personnel in need of exercise participated” were found to be significant (

Table 4. Multiple regression of the collaboration dimensions of learning.

Dependent Variable: Learning Independent Variables: Collaboration Characteristics
	Biv. reg. Stand.Beta	Mult.regr. Stand. Beta	Sign. (p)
The exercises were focused on collaboration	0.451	0.286	0.056
Sufficient forms of discussions were provided	0.428	0.203	0.129
There were opportunities to improvise	0.282	−0.049	0.722
Personnel in need of exercise participated	0.255	−0.064	0.641
Collaboration was initiated immediately	0.253	−0.025	0.839
Clear instructions of collaboration were presented	0.501	0.308	0.023
My points of view were regarded	0.398	0.101	0.469

Note: N = 71, R = 0.611, 𝑟2 = 29.3 sig = p ≤ 0.05

). The regression analysis indicated a 61% (r = 0.61) covariation between collaboration and learning, which is considered strong [49].

RQ 2:

(2) To what extent is there a relationship between participation in Norwegian wildland-fire collaboration exercises and perceived level of usefulness?

Bivariate analysis (

Table 5. Bivariate regression of the learning dimension of usefulness.

Dependent Variable: Utility Independent Variables: Learning Characteristics of Exercises
	R	R2	F	Sign (p)
I learned new things during the exercise	0.343	0.103	2.874	0.006
I learned about other’s organizational aspects	0.388	0.137	3.316	0.002
I learned about other’s communication patterns	0.323	0.090	2.713	0.009
I learned about other’s prioritizing of activities	0.324	0.091	2.700	0.009
I learned other’s concepts and abbreviations	0.436	0.177	3.841	0.000

Note: N = 71, sig = p ≤ 0.05.

). The most pronounced significance was found between usefulness and “I learned other′s concepts and abbreviations” (r = 0.436, r2 = 0.177, F = 14.755, p ≤ 0.000). This was followed by “I learned about other’s organizational aspects” (r = 0.388, r2 = 0.137, F = 10,998, p ≤ 0.002) and “I learned new things during the exercise” (r = 0.343, r2 = 0.103, F = 8.259, p ≤ 0.006). The item “I learned about other’s prioritizing of activities” received a R-value of 0.324 (r2 = 0.091, F = 7.290, p ≤ 0.009), and “I learned about other’s communication patterns” a R-value of 0.323 (r2 = 0.090, F = 7.358, p ≤ 0.009).

Multivariate analysis (

Table 6. Multiple regression of the learning dimensions of usefulness.

Dependent Variable: Utility Independent Variables: Learning Characteristics
	Biv. reg. Stand.Beta	Mult.regr. Stand. Beta	Sign. (p)
I learned new things during the exercise	0.343	−0.072	0.743
I learned about other’s organizational aspects	0.388	0.256	0.277
I learned about other’s communication patterns	0.323	−0.049	0.826
I learned about other’s prioritizing of activities	0.324	0.078	0.649
I learned other’s concepts and abbreviations	0.436	0.345	0.022

Note: N = 71, R = 0.488, 𝑟2 = 0.170, sig p ≤ 0.05

.). The perceived learning items predicted 17% (r2 = 0.170) of the usefulness variance, meaning that the remaining 83% of the predicted variance was unaccounted for. The only item that was found significant was “I learned about other’s prioritizing of activities” (p = −0.049). Thus, these results indicate a medium to small [49] covariation between learning and utility.

4. Discussion

The results from this study indicate a strong covariation between participation in Norwegian wildland-fire collaboration exercises and perceived level of learning, and a medium to small covariation between perceived learning and utility. Thus, our results support findings in past international studies, all of which indicate that participants’ perceived effects of exercises are rather limited [3,4,5,6,7]. Initially, a significant majority of the sample population in this study (84.3%) agreed that the exercise focused on collaboration. This supports the assumption that it is considered rhetorically correct to be in support of collaboration-enhancing initiatives and exercises [3], hence contributing to maintaining the crisis management assumption that collaboration is viewed as a core concern, as it helps managers and societies to effectively deal with the adverse consequences of a crisis [22]. The results indicate the importance of giving clear instructions and sufficient forms of discussion during and after the exercise in order to gain learning. The study indicates joint evaluations, improvising and testing of new and alternative strategies across sectors as successful. An extensive crisis sometimes requires more focus on flexibility, e.g., synchronous collaboration compared to an everyday emergency, e.g., sequential and parallel collaboration. Exercises are opportunities to train horizontal and vertical working processes, either mechanistic or a more organic approach. Even non-collaborative decisions when a situation requires acts of demarcation or exclusion can be tested and evaluated [56]. The skill of choosing alternative strategies and approaches can be of special value during catastrophes and disasters [45].

Over half (53.3%) strongly agreed to whether they had performed well-known activities, while the rest mildly agreed or remained neutral. This suggests that the exercises contained elements of drilling, which have a focus on developing and repeating discipline-specific key procedures [4]. It should here be taken into consideration that putting out wildland-fires, together with flying planes and helicopters, requires specific skills and training and is thus difficult to entrust to others, something that may explain the finding that 85.9% agreed that personnel in need of exercise participated. This assumption is further supported by the fact that 97% of the participants belonged to a public organization, suggesting that these exercise participants were by and large specialists within their own fields.

A clear majority of participants (87.3%) agreed that they had learned new things. Over half agreed that they had learned about others’ organizational aspects (74.6%) and communication patterns (71.8%). While these are high numbers, we argue for the need to see them in a context involving exercise participants that are highly trained, specialist professionals who already perform well-known activities. It may thus be assumed that the participants, as fire professionals, through everyday interaction and common training, already had a basic knowledge of each other’s way of organizing. That under half (45.1%) agreed that they had learned new concepts and abbreviations suggests, however, that there were organizational differences and a slight lack of focus regarding cross-organizational communication development during the exercise. This, together with the high degree of familiar task performance, indicates that the exercise was dominated by sequential or parallel working patterns [19], where participants either perform their task in a defined order, similar to an assembly-line approach, or work side by side, while maintaining focus on an individual sector specific task. Even if all levels of collaboration are necessary to practice depending on the current situation, there is, according to past studies, a tendency to avoid synchronous collaboration during exercises. The reason can be a desire to avoid challenges associated with having to understand the collaborating organization’s way of working or communicating [3], or as a means to reach individual professional goals [35]. However, in this study, it may also be a result of the practical need to perform tasks in a certain order, such as when to apply air vs. ground resources.

While most of the sample population (94.4%) initially found the exercise useful for real-life activities, the analysis of the other utility items suggests that there was some uncertainty related to who the exercise was for and which organizational level found it most useful. While 38% agreed that the exercise was useful for commanding officers, 42.3% agreed it was useful for ordinary operative staff. However, an interesting observation was that 29.6% and 42.3% chose to respond neutrally to the same questions. This suggests that the exercise lacked a clearly defined purpose, joint discussions, and the presentation of clear instructions. These assumptions are also supported by the fact that barely half (49.3%) agreed that sufficient forms of discussion were provided, and just above half (59.2%) perceived that their points of view were regarded. Overall, under half (45.1%) agreed that their experiences of the exercises would affect their daily work, while 33.8% remained neutral. From a learning point of view, these numbers suggest a potential of improvement. The results can be explained by the discovery that exercise participants have a tendency to distinguish between exercise and real-life behavior [57].

Limitations

First, this study was limited in scope as data was collected from a limited number of exercise participants during a short period. Second, the achieved population sample of 71 resulted in a statistical power of 0.86. Nevertheless, due to the relatively few wildland-fire collaboration exercises involving a large number of participants, data from this exercise gave good indications of the perceived levels of collaboration, learning, and utility. Also, the achieved power is, according to Cohen [49], still to be considered strong. Thirdly, the possible occurrence of a somewhat low term-validity needs to be taken into consideration, as the researchers did not pre-define the meaning of the CLU variables for the exercise participants. Fourth, the time passed between participants’ participation in the exercise and time answering the survey may have led to lower response rate, as well as possible lower response validity. The data was retrieved from a questionnaire, observations and interviews can add more and comprehensive insight into the studied phenomenon.

5. Conclusions

As Karl Weick [1] found in his study, wildland-fire fighters have a strong identity and prefer keeping “their familiar tools in a frightening situation” (p. 7). Weick assumed that this was because they considered the unfamiliar alternative even more frightening. As such, the results from our study may be seen as supportive of Weick’s supposition, as they indicate that the exercise population prefers working in familiar patterns. While this study found a strong covariation between participation in Norwegian wildland-fire collaboration exercises and perceived level of learning, it also found a medium-to-small covariation between perceived learning and utility. The results show that the exercise contributed positively to adding new learning. However, on the basis that the goal was to strengthen collaboration utility value, the study indicates the need to focus more on collaboration building and enhancing elements throughout all phases of the exercise. To ensure an effective and optimized use of available resources in crisis, cross-boundary tasks need to be integrated into the exercise; more room for improvising and testing of unfamiliar and alternative strategies are suggested. As this study was limited in scope and population, we recommend conducting similar studies, preferably through the use of the same instrument.