**5. Results**

The questionnaire revealed several interesting findings. Based on the response, the following table can be presented. The questionnaire was based on a 5-item Likert scale. However, we have merged disagree/strongly disagree and agree/strongly agree in order to make it more readable and to ensure a better overview. The results are presented in Table 1.


**Table 1.** Participants' report on outcome of the exercise. (*n* = 25).

The questionnaire reveals that the exercise produces several important outcomes. However, the number of respondents was low, so we have used it for creating an overall impression. It can be argued based on the response to the questionnaire that the exercise strengthens the collaboration within the group. However, this is reported and is not viewed in practice. Therefore, it is fair that the respondents expect a strengthening rather than actually experiencing a strengthening. It was also reported that room for discussions during and after the exercise is needed and might be structured for CL and social interaction. This has been stressed as very important in order to strengthen collaboration [16]. Debriefing is important in order to strengthen CL. The exercise, as noted earlier, consisted of di fferent types of operations which were concluded before moving to the next operation. We also found support from the cadets' point of view that the exercise provides opportunities for improvisation and trying out di fferent strategies regarding collaboration among the participating organizations. In this regard, the di fferent nations [3] focused on the importance of military leaders' ability to improvise. Another finding is that they reported that they learned about collaborating organizations' priorities. This is a reported outcome but still might assume strengthening the knowledge regarding what decisions are made, but to a lesser degree on how the actual priorities are decided or negotiated in real life. Regarding elements that are not so positive, there are some aspects that stand out, despite an overall positive impression. The cadets were divided down the middle on whether they perceived the exercise as useful in their daily life. This might imply that di fferent cadets or branches of the Royal Norwegian Air Force are better covered. Air operations are complex and covering several scenarios in just four days could be too ambitious. One must ask, how much did they learn in detail from such an exercise? More instructions regarding collaboration could be provided. Having said that, the exercise seems to be most heavily focused on air power planning and decision making. At the same time, it is hard to put too many expectations into an exercise. But there is a potential as we see it to strengthen this part. This could be achieved by having a "time out" focusing on this issue and asking the cadets to write down their reflections regarding collaboration issues given the use of a "Blue Book". The academy uses a so-called Blue Book in which to write down reflections [16]. We did not observe the use of this Blue Book when the first author was part of the exercise. In the exercise, an air raid resulted in severe collateral damage so the success of the mission was, to say the least, very questionable. However, no further discussion was raised. There could have been a possibility to raise questions. But the participants, including the first author, did not raise any questions, nor was it encouraged. We think that what you focus upon is important for addressing di fferent issues. We also observed that the cadets did not report increased trust in each other. This might, however, imply that trust was high before the start of the exercise. An alternative might be that the content of the exercise does not go into any real depth. The case of collateral damage was one where participants were vulnerable. Who is to blame? What should be done to mitigate the damage? A third assumption might be that this was a paper exercise and that, by playing di fferent nations, the cadets were playing roles, rather than truly giving themselves. Being in a more naturalistic and hostile environment, say crossing the mountains as a team in the winter, combating stormy weather, fatigue, and stress would be better suited for building trust. This curriculum was also covered earlier at the RNoAFA. This can be viewed as a negative aspect. In addition, the cadets were together almost every day following a tight schedule and curricula and at the time of the exercise half the class had been together for 1.5 years. All of them were together for half a year (the last ones to enter were experienced o fficers that had earlier gone through the the first year, which is mandatory before going for pilot training in the USA). It can be argued that, for this group, trust was not strengthened. It might already have been at a very high level [27]. The exercise focuses on a strategic planning team that bring to getter personnel from di fferent organizations or di fferent departments in large organizations. So, the question is also relevant for the cadets. They also reported less relevance to daily activity. That could be interpreted as meaning that this exercise is on a high level that is said to be relevant. There might be a greater complexity demanding interdisciplinary collaboration and involving ethical dilemmas that might be less visible in the day-to-day activities at the academy [27]. It stresses the importance of training students in complex scenarios demanding interdisciplinary collaboration and ethical dilemmas. Perhaps it could be argued that more of the curricula at the academy should be linked to the exercise? This means that more resources should be used in order to foster more learning from the exercise and that teachers and facilitators should be used to sca ffold the exercise and later use it as examples in the curricula. A natural development from the UNAPO exercise is that the cadets are put in the role of the commander and writing the intention of missions. In the UNAPO exercise, all cadets play the same role as executors, not the mission commander. Having one group acting as the mission command team could challenge the cadets even more.

#### **6. The Norwegian Defence Pedagogical View (FPG)**

The Norwegian Armed Forces has a comprehensive strategic curriculum with regards to the pedagogical approach and overall learning objectives for learning activity. This is the Armed Forces' pedagogical basic view [27]. This plan focuses consistently on collaboration, development of creativity and problem solving. In addition, emphasis is placed on teaching and exercise methods, where no work is performed towards a defined conclusion, but instead situation-oriented solutions adapted to the challenges and conditions that exist. Competence is important for dealing with unforeseen events. In the preface of the document, it is stated, among other things:

*"* ... *The military profession presupposes the ability to mobilize knowledge, attitudes and skills in both familiar, new and unexpected circumstances in relation to persons with di*ff*erent backgrounds, to equipment and new technology. This makes great demands on the learning activity that is taking place in the Armed Forces in relation to education and training. There are also demands on the individual's ability learn, cooperate, evaluate and adapt to di*ff*erent situations.*

*The changes in the organization and the tasks of the Defense demand inspection of and development of the pedagogical activity. The pedagogical execution of the Defense's branches of education and courses will be an important success criterion to succeed in developing a quality oriented, e*ff*ective, flexible and alteration willing Defense.*

*The Defense already has a good and well-developed educational system, but there may be aspects that have to be improved to meet new challenges. The basic pedagogical view (FPG) of the Defense shall make sure of a common platform for teaching, both internally and what is bought externally. FPG shall be the guideline for all pedagogical activities within the Defense. FPG is based upon the basic values of the Defense, and must also be seen in relation to basic military leadership (GML) the common operative doctrine (FFOD) of the Defense and the personnel manual (FPH). These together are an important force to produce e*ff*ective learning* ... *"* ([27], preface).

Of course, training is also based on basic fixed procedures and solutions, but the concept of learning and pedagogy has several dimensions and is more holistic than that which has traditionally been seen in military organizations (before FPG was introduced). In itself, the development and

implementation of an organization's pedagogical philosophy will be a feature to be determined for the competence development in an organization. Therefore, it is somewhat surprising that several large civilian companies and institutions, and other countries' military organizational zones, do not already have this. This is an important lesson, and we believe that we can strongly state that a well-functioning FPG in the organization is crucial for successful competence development. Without FPG, strategic managemen<sup>t</sup> and training are built without a foundation.

#### **7. The Lessons Learned: Overall Discussions and Implications**

The cadets reported overall positive experiences regarding collaboration and learning. They reported less favorable ones regarding the usefulness in daily activities and trust. However, the number of reports was small, which should also be a consideration. Further distribution of the questionnaire is recommended in order to gather more data and compare the di fferent classes of cadets and thereby create CL in order to strengthen the pedagogical outcome, as pointed out in FPG [27]. We will also advise distributing the questionnaire during the exercise to facilitate more CL. If teamwork was essential and the cadets reported low scores on this issue, more emphasis on this could be introduced during the exercise. In a complex world we aim to find training areas that can prepare the cadets for scenarios that also heavily involve the unforeseen. Improvisation is seen as important for military leaders and the exercise provides a sound arena for this purpose. However, again, it is necessary to ask how they will use improvisation after the exercise. We have seen that, even for a table exercise, important lessons can be learned. The point is not having live planes, but rather solving tasks and garnering feedback and learning. The overall experience is reported to be very positive. In the current study, only the cadets filled out the questionnaire. For further exercises, the instructors might also fill out a questionnaire for creating a more dynamic learning environment and enabling more CL, thereby fostering CL between teachers/facilitators and cadets. It can also be supported by semi-structured interviews by researchers not playing a part of the planning and execution of the exercise. Another point to address is that, in another study [28], it was pointed out that some instructors focus only on the product and not on the process. The students, however, favored instructors focusing on both the process and product [28]. Further development of the UNAPO exercise would benefit from addressing both the product and the processes. Another factor is that CL will demand that instructors and cadets need to stop, reflect and adjust when under way, as [22] when thinking of leadership as putting a hand on the thermostat. The idea of the hand on the thermostat, which either speeds up the pace of the exercise or pauses it to reflect on something emergent, could be a fruitful path to take. If the exercise is perceived either by cadets or teachers/ facilitators as going slow, the "heat" could be turned up to create more stress, or strengthen the complexity, or to reveal an ethical dilemma that could be further explored [25]. In addition, it will be necessary for the organization to have developed a unified pedagogical philosophy, which can form the foundation for strategic management, training and renewal. Not least, this applies to building the right skills and a flexible and adaptable mindset to deal with unforeseen events during situations, including pandemics. The UNAPO exercise is developed within an air power context. We have seen that context is important for the leadership of organizations [7], but we think there are some generic lessons that could be addressed. A paper exercise allows for complex scenarios to be played out. We also know from scenario planning that it contributes to strengthening the relations between the participating actors [29]. In this respect the current study provides an example of the seminal work of De Geus (1988) [29]. Regarding more recent research, we think the current study should be seen in light of the work of Herberg et al. (2019) [15], understanding CL and social interaction. The current study might provide a concrete example to facilitate such processes. The current study might be a case example for contributing to the fight against pandemics, in particular COVID-19, especially in relation to extracting experiences and research finding from military pedagogy and psychology, with implications for handling competence and knowledge strategies to provide infection control [22].

#### **8. Conclusions: Five Crisis Management Principles**

All in all, in this study we have identified five crisis managemen<sup>t</sup> principles for developing competence for interaction under unforeseen conditions and these principles are identified as:


These principles should permeate strategic contingency plans and practical crisis management, to ensure the development of the necessary expertise for interaction under risk and unpredictable conditions. The principles can also serve as quality indicators in evaluation and improvement processes in the organization. However, it is necessary for the individual organization to adapt and concretize the principles to its specific activities and tasks.

**Author Contributions:** The first author (T.J.S.) has planned, executed the data collection and analysed the results. The second author (G.-E.T.) contributed in the planning of the study, analysed the results, deduced conclusions, and contributed in the writing process. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.
