*Article* **FLIGBY—A Serious Game Tool to Enhance Motivation and Competencies in Entrepreneurship**

**Zoltan Buzady <sup>1</sup> and Fernando Almeida 2,\***

<sup>1</sup> Corvinus Business School, Corvinus University of Budapest, 1093 Budapest, Hungary

<sup>2</sup> Faculty of Engineering, University of Porto & INESC TEC, 4200-465 Porto, Portugal

**\*** Correspondence: almd@fe.up.pt

Received: 27 June 2019; Accepted: 17 July 2019; Published: 19 July 2019

**Abstract:** Entrepreneurship is currently one of the most fundamental economic activities in the 21st century. Entrepreneurship encourages young generations to generate their self-employment and develop key soft-skills that will be useful throughout their professional career. This study aims to present and explore a case study of a higher education institution that adopts FLIGBY as a serious game, which allows students to develop entrepreneurship skills in an immersive way and based on real challenges that can be found in business environments. The findings indicate that FLIGBY offers relevant potentials and new possibilities in the development of management, leadership, and entrepreneurship skills. Furthermore, the game allows the inclusion of summative and formative assessment elements, which are essential in the process of monitoring and analyzing the student's performance.

**Keywords:** entrepreneurship education; FLIGBY; Flow; positive psychology; serious games; higher education

#### **1. Introduction**

In the past, it was believed that entrepreneurship could not be taught. In this sense, entrepreneurs were individuals who had an innate capacity, having been born with special characteristics that favored business success. However, the results of several studies over the last two decades have shown that although personal characteristics may facilitate the management of a new business, the entrepreneurial process can be learned and trained [1,2]. According to [3], business success will depend on several internal and external factors to the business, on the entrepreneur's personal characteristics, and on his/her ability to manage daily challenges.

One of the main concerns of entrepreneurial education is to build active entrepreneurs, going beyond the theoretical knowledge of the theme [4,5]. However, Lautenschläger and Haase [6] emphasize that there are aspects of entrepreneurship that may be easy to teach and others not. Skills and competencies such as creativity, innovation, pro-activity, decision-making, and risk propensity are difficult aspects to instill in individuals without this intrinsic propensity [6]. Several authors such as [7,8] advocate a more practice-oriented pedagogical approach as more appropriate for the teaching of entrepreneurship. Therefore, the traditional lecture can be used to review theoretical and cultural aspects of entrepreneurship, directing the other aspects of entrepreneurial action towards richer, more dynamic, and immersive methods and pedagogic resources.

The teaching of entrepreneurship should not exclusively teach how to run a business. In [9], it is mentioned that this type of teaching should encourage creative thinking and promote a strong sense of self-esteem and autonomy. The knowledge that should result from the teaching of entrepreneurship should promote communication, creativity, critical thinking, leadership, negotiation, problem-solving, networking, and time management [10,11]. According to [12], the teaching of entrepreneurship helps to

raise students' awareness of the entrepreneurial career, transferring general knowledge about business management and increasing the level of knowledge and empowerment of students to create their own professional career.

In entrepreneurial education programs, traditional methods are based on theoretical lectures and group work. This approach allows students to understand rigorously and deeply the characteristics of entrepreneurial activity [13]. However, this approach does not allow students to understand the consequences of the actions taken before the commitments and resources to launch a business [14]. Consequently, other teaching methods should complement this theoretical training and encourage the active participation of students, the enhancement of their social and problem-solving skills. Belitski et al. [15] advocate formal education should be complemented by informal teaching methods supported in group activities, simulations, and serious games. In fact, a serious game is defined by [16] as games that have an educational objective but also include three fundamental components: (i) experience, (ii) entertainment, and (iii) multimedia. In this sense, this study intends to explore, through a case study approach, how the use of FLIGBY as a serious game can be used to enhance motivation and competencies in entrepreneurship. This study uses a quantitative approach to assess the success of FLIGBY adoption in an entrepreneurship discipline attended by multidisciplinary students from the management and computer science course at a polytechnic higher education institution in Portugal.

The work is organized as follows: Initially, a literature review on the field of gamification and serious games is performed. After that, the FLIGBY game and the concept of Flow theory is presented. Thereafter, the methodology and adopted methods are presented. Consequently, the main findings are analyzed and discussed. Finally, the conclusions of this work are drawn and some suggestions for future work are given.

#### **2. Background**

Games are typically designed with the primary objective of providing a playful experience and entertaining players. Studies conducted by [17,18] show good results in the application of games to increase the motivation of players when used to make products and services more attractive and engaging consumers. The use of game mechanics, dynamics, and structure in non-game contexts is called gamification [19].

Gamification has been incorporated into several businesses in the promotion of marketing strategies and customer engagement. Deterding et al. [19] address the use of reward systems based on users' reputation ranking. This reward-based approach was then used by [20] to define a model that awards points according to customer participation (e.g., register in products, submit comments and reviews, watch videos, Facebook "likes", share on Twitter, etc.). Edwards et al. [21] suggest the use of gamification techniques incorporated in smartphone applications in the promotion of health behaviors. Due to its popularity, gamification extrapolated the business segment and began to be applied in other contexts such as education.

The purpose of gamification in education is to make learning easier, faster, and more dynamic for the student. According to [22], theoretical content alone does not capture the attention and interest of students in the subjects addressed, hence the retention of this content is low. In this sense, gamification has the fundamental idea of awakening the greater involvement of students and, consequently, improving the quality of teaching. Sailer et al. [23] state the potential of gamification in education is immense, functioning as an element that arouses interest, increases participation, and develops creativity and autonomy. To make use of gamification, it is not necessary to use a game despite the fact that it is one of the possibilities. Gamification in education comprises the idea of adding elements, mechanics, and logic of games to engage people in the learning process [24].

Simultaneously with the growing interest in gamification, the concept of serious games emerged. According to [25], serious games are designed for a specific purpose related to learning, not just for fun. They have the characteristic elements of a playful game, but their goal is to counsel something

predetermined and related to the learning process. Karagiorgas and Niemann [26] state that serious games require players to meet goals in a virtual world, whose student performance has no consequence in the real world, while gamification in education is created to increase student engagement in real tasks.

The concept of serious games is an emerging paradigm in Technology Enhanced Learning (TEL). The characteristics of serious games allow their adoption in the development of entrepreneurial skills and for the teaching of entrepreneurship [27,28]. According to [29], serious games are effective because the acquisition of knowledge is done using a contextual learning approach. Hauge et al. [30] complement this vision by emphasizing that good entrepreneurship serious games should involve students in challenging environments that simultaneously motivate the acquisition of skills relevant to entrepreneurial activity and promote the development of soft skills.

The adoption of serious games in entrepreneurship classes is quite residual. One of the most relevant works in the area was carried out by [31], in which a comparative analysis of eight entrepreneurship serious games (i.e., GoVenture: Entrepreneur, GoVenture: World, Interpretive solutions: Entrepreneur, Entrepreneurship Simulation: The Startup Game, SimVenture, HipsterCEO, Innovative Dutch, and Venture Strategy) was performed. The findings of this study are aligned with the results also obtained by Bellotti et al. [32] and highlight the low degree of maturity of these projects, and the low level of fidelity and engagement. Another game that stands out in the European paradigm is the ENTREXplorer. The main objective of the game is to offer a technological solution, in which students can learn the fundamental concepts of entrepreneurship, the creation of a new business, marketing, and strategic positioning [33]. This game also allows students to build a business plan online. Finally, the work developed by Almeida and Simões [28] identified that entrepreneurship serious games typically fail to support different learning styles and do not include an internal process of assessing learning goals. Therefore, it is important to explore the potentialities of FLIGBY and assess whether the use of FLIGBY increased the students' motivation to attend the entrepreneurship discipline. The following research question (RQ) was established: *RQ1: Is the motivation to attend entrepreneurship classes increased?*

Following the recommendations of [7,8], it is essential to offer students a practical environment for the teaching of entrepreneurship that complements the theoretical exposition of the theme of entrepreneurship. The literature highlights the use of group work in which students can simulate the process of developing a business plan that includes aspects related to the generation of ideas, strategic positioning, feasibility analysis, among others [34,35]. In the process of building a business plan, it is essential that students work in groups as a way to promote communication, the active learning process and the development of critical thinking. Rosen [36] emphasizes the fundamental role of teamwork for entrepreneurs in the maturing of ideas, mentioning that the courses of the main university institutions include team-based project work. In this sense, it is also important to explore whether the use of FLIGBY contributes to increasing students' perception of the importance of group work. The following research question was included: *RQ2: Is the perception of the importance of group work increased?*

The process of developing serious games is based on a diverse set of educational theories, such as cognitive, constructivist, constructionist, and experimental theories of learning. These theories are based on the pillar of contextual learning because the information used in this context tries to simulate reality, and it is supposed to transfer the player to that reality. Based on the culture and identity of each player, serious games can be a way to mediate learning through discussion and reflection [37]. The application of serious games to teaching shows that the motivation before the game, their involvement, and satisfaction while playing depends on the level of their effort and the previous preparation performed by the teacher [38]. This recommendation is important in the definition of mechanisms to introduce students to the fundamental theoretical concepts associated with FLIGBY and to monitor the difficulties experienced by students throughout the game.

Serious games offer the educational community more realistic and interactive educational processes. The educational approach of the entrepreneur based on serious games makes possible to unite playful aspects and specific content, motivating the learning process and contributing to make students feel better prepared for the challenges of creating a new business. The study conducted by [39] shows that entrepreneurship education has contributed to increasing the competencies and intentions of students toward self-employment. Other studies have emerged to explore the type of skills that entrepreneurship education promotes. In [40], the types of competences developed are clustered into two major groups: Academic competencies and personal effectiveness competencies. The first cluster includes competencies regarding reading, writing, science, technology, or critical and analytical thinking; the second cluster includes interpersonal skills, initiative, ambition, integrity, or taking risks. This mapping is not unique, emerging studies that include other groups. In [41], it is suggested the adoption of three clusters: Entrepreneurial attitude (e.g., sense of initiative, self-efficacy, structural behavior), entrepreneurial skills (e.g., creativity, networking, adaptability), and knowledge of entrepreneurship (e.g., understanding of entrepreneurship, role of entrepreneurs, determinants of successful entrepreneurship). In this sense, it is important to explore the role that FLIGBY can play in the development of an entrepreneur's key competencies. The following research question was included: *RQ3: What are the skills in which students performed better or worse?*

The assessment of learning through the use of serious games is another key element. Yüksel and Gündüz [42] emphasize the complementary role of formative and summative assessment. Summative assessment refers to the students' performance at the end of a course or subject in order to assess the final outcome of the learning process, assigning grades and certifications, while formative assessment seeks to monitor the student's learning process throughout the use of a method or tool in teaching [43]. Price [44] states formative assessment is continuous and systematic, allowing the teacher and students to obtain information about the development of learning with a view to adjusting processes and strategies. Serious games should implement final and intermediate assessment mechanisms to assess students' performance. Accordingly, Bellotti et al. [45] advocate that the evaluation of students' performance should consider the different learning styles and provides appropriate feedback throughout the game. Daoudi et al. [46] summarize the evaluation techniques that can be used for learners' assessment, namely the use of interviews, questionnaires, scoring, and multi-agent systems. Within these, the adoption of surveys and questionnaires stands out due to its simplicity and independence from the type of game adopted. However, it is clear that these techniques can be effectively complemented and used in conjunction with mechanisms integrated into the game to measure the performance of the game in real-time. Therefore, it is important that a pedagogical strategy for the use of serious games in the development of entrepreneurial skills includes summative and formative components. Consequently, the following research question was included: *RQ4: What are the summative and formative elements used in the assessment of students' performance?*

#### **3. The FLIGBY Serious Game**

#### *3.1. The Flow Theory*

The flow theory, proposed by Mihaly Csikszentmihalyi from his initial research on states of happiness and creativity, advocates that happiness occurs when individuals experience optimal mental states called "Flow". This state of immersion in which people entered when devoting themselves to a particular task can be achieved from the existence of three conditions [47]:


The various emotional states proposed in [47] in which an individual can be while performing an activity are summarized in Figure 1. Depending on the skill level of the individual and the difficulty of the challenges, the individual will be in a different mental state. The "apathy" is considered the worst state, where there is no challenge to meet and no skill to test. The opposite is exactly the state of flow, where the individual puts all his or her skills into the realization of a challenge that he or she considers both challenging and capable of accomplishing. Most of the time, it is expected that an individual can be at intermediate points. For example, if an individual is in a state of control, then it is important to increase the challenge; if he/she is in a state of anxiety or excitement, then it is important to increase his/her abilities to perform a proposed task.

**Figure 1.** Emotional states in the Flow theory [47].

In the research conducted by [47], which involved a very diverse group of professionals (e.g., monks, executives, doctors), some common elements were found that pointed to the state of flow, respectively: (i) complete involvement in the activities developed, (ii) high focus and concentration, (iii) internal clarity, (iv) calm mind and no worries, (v) total focus on the present moment with loss of the notion of time duration, and (vi) intrinsic motivation.

#### *3.2. Architecture and Structure of FLIGBY*

FLIGBY is a serious game in which the player is recently named General Manager (GM/CEO) of a family business ("Turul Winery") in California in the USA. The player faces the challenging task of having to achieve a state of harmony and cooperation in a team significantly weakened by internal conflicts due to the dysfunctional leadership style of the previous GM.

A key task is to create an environment that promotes teamwork and improves Flow. Consequently, one of the main goals of the game is to bring as many colleagues as possible, even for a short time, to a state of Flow. The dilemma is when to support a colleague and when to protect the interest of the team, its stakeholders, and prudent environmental management. At the same time, decisions about strategic issues of the company's future must be made according to the expectations of the owner of the winery.

The player performs approximately 150 decisions during the game. In most decisions, for example on how to conduct a strategy meeting with a team, the player must choose a response from 2 to 5 options presented. The selected answers will put each player in a proper and individual path of the story. There are many possible paths that result in different results at the end of the game. The experience of using the game is therefore personalized by the player according to the decisions made throughout the game.

The player will necessarily have to balance several goals to employees reach the Flow state. This can be done in two ways: By making management decisions that affect colleagues so that they can enter a state of Flow or by calibrating their decisions so as to promote a corporate atmosphere conducive to Flow overall. It is also the winery's goal to follow business practices that are friendly to the wider social and ecological environment, without compromising its profit potential.

The overall performance of the player as GM of Turul Winery will be indicated whether or not he wins the maximum prize of the game, the so-called "Spirit of Wine Trophy". The average playing time is 7–9 h. The software structure consists of the following three main parts:


In the implementation of FLIGBY were considered the four stages of the approaches for serious game design (i.e., analysis, design, development, and evaluation) synthesized by Ávila-Pesántez et al. [48]. FLIGBY offers the possibility of being used in different domains. This flexibility is achieved through the presentation of a wide variety of scenarios and situations, which allows the game to be explored in an academic and business environment with individuals in the field of management, leadership, or entrepreneurship. According to [49], this high flexibility is fundamental for the commercial success of a serious game in different industries. Additionally, FLIGBY was built with the objective of building learning in a constructivist and experiential way. In line with [50,51], this is a good practice that should be followed by serious games to remove any useless redundancy and engage the player throughout the various challenges that are posed by the game.

The game is available through a web-browser interface. The dynamics of the game occur in the central window where the various characters interact with each other. In the lower left corner, the game presents the consultant persona (Mr. FLIGBY), who is a key character that gives feedback to the player on his/her actions (see Figure 2). On the right side, the game offers several buttons to access the features of the game and also gives an indication regarding the time spent in each scenario. Marer et al. [52] describe many details about FLIGBY's conceptual, academic, and pedagogical aspects.

**Figure 2.** FLIGBY interface (authors' own illustration).

#### **4. Materials and Methods**

#### *4.1. Structure of the Study*

Mixed methods were employed in the development of this study. This approach allows us to simultaneously use quantitative and qualitative methods in the exploration and analysis of data. According to [53], the adoption of mixed methods enriches the findings and increases the depth and breadth of the study. The literature offers several ways to structure a mixed methods study, and the multilevel design was applied in this study. This approach assumes the existence of multiple layers in the analysis of a problem, in which quantitative and/or qualitative methods can be used throughout the process. In [54] it is stated that this approach allows us to explore the cross-level direct effects and

provide more accurate and robust results. Figure 3 represents schematically the adopted multilevel design in which it becomes possible to perceive the objective of each phase and the adopted method. The study was structured in three phases, with qualitative and quantitative techniques being adopted in the collection and analysis of data in each phase. The first phase followed from 15 October 2018 to 1 November 2018; the second phase from 1 November 2018 to 15 December 2018; and the third phase from 7 January 2019 to 25 January 2019.

**Figure 3.** Phases of the adopted methodology (author's own illustration).

The study began on 15 October 2018, in which a survey was distributed among students that aimed to characterize the students' profile considering their possible previous experience as entrepreneurs, the experience of working in groups, their individual attitudes, and the self-evaluation of their skills. This survey was available to students during the 2-week period. Among the various methodological options in the field of social sciences and humanities research, the survey is assumed to be one of the most frequently used. According to [55,56], the use of surveys allows direct knowledge of reality and rapid quantification of the findings. However, it is also an approach with low perception and comprehension of evolutionary phenomena. For these reasons, we adopted a research methodology consisting of three phases (Figure 3) in which qualitative and quantitative methodologies were used along the research process. Table 1 presents a brief descriptive analysis of the survey. A total of 51 students responded to this survey. The results of this survey show that a significant majority of students do not have experience as entrepreneurs and consider that they enjoy working in groups. A considerable majority of learners accepted failures in their learning process and considered that these situations offered opportunities for improvement. In relation to quality, timing, and perfectionism there was a greater dispersal of responses, and there was no clear conviction from students as to which of these elements may be most important.

This survey also contained some qualitative interpretation questions that aimed to explore the process of working in a group and self-evaluate students' skills, respectively:


In phase II which took place between 1 November 2018 and 15 December 2018 the FLIGBY was made available to students via the cloud. An individual access account was created for each student. The student's performance during the game was recorded. During these 6 weeks, a weekly session of 30 min was organized to discuss the main difficulties experienced by students in the game until that week. An informal approach was chosen through unstructured interviews with the students of the class. Only two types of problems were found: (i) the application did not work on IoS devices due

to technical problems in running a Flash application; and (ii) due to network breaks, some students had to play the same scenario several times. These problems were overcome with the support of the teacher. These sessions also proved to be essential to increase the students' level of motivation for the game and to ensure that all students received the access code for the game.


**Table 1.** Sample characteristics of the survey (phase I).

In phase III, the authors tried to evaluate the experience of using FLIGBY and the performance obtained by the students. This process was performed using a survey in which quantitative data on the experience of the game, its benefits, and limitations were obtained. Table 2 shows the data obtained. In the "evaluation of experience" dimension, the students gave a high score for the experience offered by FLIGBY, having also mentioned that it was an experience "worth the time". They also considered that the game is not too long or complex, despite a high dispersion of responses. In the "evaluation of game" dimension, the students pointed to interactivity as the main asset of the game. In the "benefits of the game" dimension, the students considered that FLIGBY allowed them essentially to increase the level of knowledge in the management field and to have a greater knowledge of their skills that will be essential to them along their career at university. On the other side, in the "limitations of the game" dimension, the students considered that FLIGBY needs previous training before starting to play. In this sense, the pedagogical session held by the teacher one week before the game was played by the students proved to be absolutely essential. This session lasted 90 min and two topics were covered: (i) presentation of the flow theory and how this theoretical concept is essential in the management of an organization and team, and (ii) explanation of the interface of the game.

In addition, a semi-structured study was performed in which students were asked to perform a self-assessment of their competences against the performance indicators provided by the game. This semi-structured interview was conducted in a group in which all students were present and three questions were asked: (i) the main difficulties experienced by students in using FLIGBY, (ii) the indicators in which they performed best and worst were in line with their expectations, and (iii) what is the influence of their performance in FLIGBY on their academic and professional career.


**Table 2.** Sample characteristics of the survey (phase III).

A thematic analysis was employed to analyze the findings of the semi-structured interview (Table 3). The thematic analysis is one of the most commonly used approaches in qualitative research methods and, according to Braun and Clarke [57], allows identifying and exploring patterns within the data. Furthermore, Creswell and Poth [58] point out that this approach allows exploring the data from two perspectives: From a data-driven perspective and from the research question perspective. This complementarity turns possible to assess the consistency of the data with the research questions and to explore the level of detail obtained in each of them.


**Table 3.** Identification of final themes in the thematic analysis process.

#### *4.2. Research Dimensions*

Player performance in FLIGBY is measured using 29 indicators. These indicators correspond to the skills identified by [47] from interviews conducted with CEOs and organizational leaders, which allowed the leader persons to personally reach the state of flow, in which there is total absorption and joy in the present moment, and also to create what Csikszentmihalyi calls "Flow-promoting organizational work environments and leadership culture". Therefore, in the business context, in order to obtain the best from each employee and to obtain business success, it becomes necessary to have harmony and positive results in each of the 29 indicators identified in Table 4. FLIGBY creates an individual final report for each player in which its performance is measured considering those 29 indicators. This report is made available to the teacher and to each student.

**Table 4.** MAP dimensions.



#### **Table 4.** *Cont.*

#### **5. Results and Discussion**

#### *5.1. RQ1—Is the Motivation to Attend Entrepreneurship Classes Increased?*

The discipline of entrepreneurship is attended by students from management and computer science courses. This heterogeneous profile allows the creation of a multidisciplinary learning environment as suggested by the Organization for Economic Cooperation and Development (OECD) when analyzing the state of entrepreneurship education in higher education institutions [59]. However, the process of integrating multidisciplinary students also presents some challenges, namely in terms of communication within the teams, in the division of tasks, and also in the motivation to work together. According to the data available in Table 2, the use of FLIGBY before the formation of these multidisciplinary teams allowed students to know their skills and develop management skills. Both dimensions present a mean equal to 4.105. The semi-structured interview held after the conclusion of the game by students allowed us to identify that the development of management skills was more valued by the students of the computer science course. For the students of the computer science course, the entrepreneurship course is the first curricular unit in which they address basic knowledge of management of an organization, while the management students already attended two disciplines (i.e., Management and Organization of Companies, and Marketing) that introduce fundamental management competencies. Therefore, FLIGBY emerges as a valuable tool in reducing asymmetries in the preparation of students in the creation of an entrepreneurship project.

Looking at Table 2, the authors found that the increase in motivation was positively evaluated by the students, despite being one of the least relevant dimensions within the tested benefits. Nevertheless, motivation cannot be seen in isolation. Frese and Gielnik [60] advocate that there is a slight but important difference between people who seek to perform activities to meet their strengths, which is natural because they feel competent, and the pleasure of facing challenges and testing new skills and which are typical for entrepreneurs. FLIGBY seeks to exactly pose new challenges to students throughout various scenarios. In third and fourth place as the main benefits offered by FLIGBY (see Table 2) stand the development of leadership competencies and the testing of new approaches. According to [61], entrepreneurship and leadership present conceptual similarity with a considerable area of overlap but are still different. While leadership is more directly associated with the conceptual components related to people, entrepreneurship tends to be more linked to the concepts of seeking independence through the exploration of market opportunities. Kouzes and Posner [61] state entrepreneurship is not a necessary part of successful leadership, but leadership is an element of entrepreneurial success. Despite the importance of leadership, there is no discipline in the management course and computer science course that addresses this theme. Students in general, and in particular those from the management course and with experience in the labor market, expressed as a positive point that FLIGBY allowed them to explore how different followed options have an impact on the cohesion of teams and in the individual performance of employees. Two of these students expressed that the knowledge acquired has a direct impact on their business activities as they hold middle management positions, one in a financial company and another in the logistics field.

#### *5.2. RQ2—Is the Perception of the Importance of Group Work Increased?*

The development of entrepreneurial projects requires the research, identification, and development of creative and innovative ideas. Group work emerges as a fundamental competence in the various phases of an entrepreneurship project, from the generation of the business idea, conceptualization, to the entry into the market. The first survey made available to students before they started the game allowed us to identify the students' perception of group work. The results confirmed the students' appreciation of this topic since more than 95% of the students said they liked group work and were able to identify some of the main advantages of collaborative work, such as the appreciation of each individual, the development of skills, or the exchange of experiences. The data in Table 1 also allowed us to identify that the majority of students (mean equal to 4137) stated that they felt comfortable in communicating with people different from them, although the number of experiences they had in this situation was relatively lower (mean equal to 3725). In fact, the discipline of entrepreneurship proved to be fundamental in increasing this indicator, since teamwork is absolutely essential and the 2018/19 school year was attended by four Erasmus students from partner institutions in Lithuania and Poland.

The results of the survey available in Table 2 and the semi-structured interview conducted after the conclusion of the game were inconclusive regarding the direct benefits offered by FLIGBY for the increase of group work skills. However, the results of the survey in Table 2, namely, "helps me to have new attitudes to people" with an average of 4.053, show that students after playing FLIGBY feel more receptive to working in groups. Furthermore, students indicated that in their role as CEO of Turul Winery in the FLIGBY game they needed to identify the key skills of each employee and get them to work as a team. FLIGBY's main challenge was to get all Turul Winery employees to achieve Flow by the end of the game. The way each student managed the conflicts within the team allowed them to be better prepared for the real challenges they will encounter in multidisciplinary group work in the context of the entrepreneurship course.

#### *5.3. RQ3—What Are the Skills in Which Students Performed Better or Worse?*

Table 4 allows us to explore the students' performance in FLIGBY considering the established 29 MAP dimensions. The students' performance in each dimension was analyzed considering the course in which they were enrolled, and it was possible to reach the following conclusions: (i) the three dimensions in which the students of the business course performed best were information gathering, involvement, and emotional intelligence; (ii) in the computer science course, a similar behavior was observed, plus the importance of social dynamics; (iii) the variables in which the students performed worst were time management, decision-making on pressure, delegation, and prioritization. The dimensions in which the students had the worst performance are globally transversal to both courses.

Table 5 also seeks to assess whether the behavior of students in any of the considered courses is different for each of the 29 MAP dimensions. For this purpose, the *p*-value was calculated, which represents the probability of having observed a distinct significant value under the null hypothesis. A cut-off value of 0.05 was defined to reject the null hypothesis. The results indicate the existence of different significant performance for the students of the computer science course in the following dimensions: (i) decision-making on pressure; (ii) feedback; (iii) future orientation; (iv) social dynamics; and (v) time management. In these five situations, it was found that the performance of students from the computer science course was significantly higher.


**Table 5.** Students' performance in FLIGBY considering MAP dimensions.

Shaded cells have a *p*-value less than 0.05.

The semi-structured interview conducted after the conclusion of the game made possible to explore the reasons that justify the obtained results. Difficulties in time management was a top difficulty pointed out transversally by all students, and which occurred right in the first scenarios of the game in which the player in the role of CEO had to schedule meetings with all employees of the company. In fact, most of the students showed interest in knowing in depth each employee but could not manage the time allocated to each meeting, having typically exceeded this time. This difficulty experienced in the game is also experienced by students in the discipline of entrepreneurship since this discipline works in parallel with other disciplines with a high taught time and workload on projects. In fact, time management is considered a very important determinant in the individual performance of students in higher education, but it is also important for entrepreneurs in order to complete the pre-startup phase successfully [62,63]. On the other hand, the students' performance in the emotional intelligence component was a surprise. Most students were able to identify, understand, recognize, and manage their emotions and the emotions of other characters in the game in a positive way. This positive performance also represented a broad growth of students in this field since most of them did not even know the concept of emotional intelligence. In [64], it is highlighted the importance of developing emotional intelligence in education and FLIGBY is a non-immersive solution that can promote the development of these skills.

Finally, the authors also explored the reasons why computer science students got better results in five MAP dimensions. At this level, three reasons emerged as relevant to explain this performance: (i) project management practices, namely the adoption of agile methodologies in Software Engineering and Decision Support Systems discipline were important elements for students to apply personal time management practices that proved to be important in the context of the game; (ii) the group work in large dimension groups (i.e., 6 to 8 students) carried out mainly in the Decision Support Systems discipline proved to be important in developing both time management skills and giving feedback to a project team; and (iii) around 20% of students in the Computer Science course already had an Erasmus + experience, which was important in developing social dynamics.

#### *5.4. RQ4—What Are the Summative and Formative Elements Used in the Assessment of Students' Performance?*

The semi-structured interview held at the end of the game was also relevant to systematically identify and expose the summative and formative elements used in the process of assessing student performance. The versatility of FLIGBY allowed the incorporation of both summative and formative assessment components. The main element of summative evaluation is FLIGBY personal report which is a relatively long report that evaluates the player's performance according to the 29 MAP dimensions and lists the key performance indicators (KPIs) of the player's performance. However, despite the unequivocal importance of the summative assessment elements, the main advantage offered by FLIGBY is the introduction of formative elements that give the student important real-time feedback on their performance. These elements are essential to the player's motivation and learning process. As main elements of formative assessment, FLIGBY offers: (i) Mr. FLIGBY, which is a virtual coach that gives advice to the players and praise them if they make good decisions; (ii) the FLOW Profile that allows visualizing the positioning of each player in the eight emotional states of the Flow theory proposed by [47]; (iii) the Flow Meter of the Turul Winery indicates the corporate atmosphere meter which shows the average mood inside the organization; and (iv) Profitability Index which shows how a player's decision impacts Turul Winer's revenue-generating ability. In Figure 4 the following elements are identified: (A) Flow Profile; (B) Flow Meter; and (C) Profitability Index.

**Figure 4.** FLIGBY formative feedback (authors' own illustration). (**A**) Flow Profile; (**B**) Flow Meter; and (**C**) Profitability Index.

The inclusion of these four formative elements is fundamental for students to adjust the development of their game processes and strategies as suggested by [44]. This adjustment is performed by the player himself without the intervention of the teacher. Another key aspect offered by the game is the level of challenge that is posed to each player that is adapted according to their actions in the game. As indicated by [45] it is essential that the feedback that is given to the player is personalized according to the different learning styles. Mr. FLIGBY's intervention is important to give indications to the player how he/she can improve his/her performance in the next scenario, and the feedback that was given by Mr. FLIGBY always seeks to be constructive by identifying first of all the correct actions taken by the player, but also indicating the areas in which he/she should improve.

#### **6. Conclusions**

Entrepreneurship is a standard of living that includes a set of behaviors and skills that can be developed and applied not only when launching a new business but also to enhance the performance of any job activity. The mission of universities, in addition to preparing students for the job market, is to form critical and aware citizens who can contribute as agents of change in society. In this sense, the inclusion of entrepreneurship in the formal university education system is relevant. FLIGBY presents itself as a technological tool that through a serious game allows the acquisition and development of competencies through a powerful, immersive, and personalized learning space.

The use of FLIGBY in the context of an entrepreneurship course involving students with multidisciplinary skills in management and computer science was received very positively by students. The students mentioned the usefulness of the game in the development of management, leadership and entrepreneurship skills that will be essential to them throughout their academic career and in the labor market. By using FLIGBY, students were able to train their skills in a wide range of domains like gathering information, motivating employees, training their emotional intelligence, and establishing social dynamics in a corporate environment. Furthermore, the use of FLIGBY allowed these students to better understand their skills and explore how they interact with individuals that have very distinct and often conflicting characters. These skills are fundamental in the labor market and are typically not included and addressed in the context of a higher education degree.

FLIGBY offers both summative and formative assessment components. For one side, the FLIGBY personal report is a fundamental element in the analysis of the player's performance in the game considering the 29 MAP dimensions. This report also allows a comparative analysis of the benchmarking obtained by players in each of these dimensions. Despite the importance of summative elements, the potential of FLIGBY is revealed in the inclusion of several formative elements that support and evaluate the player in a non-intrusive way throughout the game. These formative elements are essential in the development of student's skills while they are playing.

This study has unequivocal theoretical and practical potentialities and benefits. From the theoretical point of view, a new informal teaching method based on a serious game is presented that allows students to increase technical skills in the field of management and entrepreneurship and also allows them to develop essential soft-skills (e.g., leadership, group work, emotional intelligence, problem-solving skills, among others) that are crucial when launching a new venture. On the other hand, in practical terms, this study seeks to arouse in other higher education institutions the desire to include serious games as a complementary activity to formal teaching methods in an entrepreneurship course, highlighting these game's relevance. It is intended that this case study serve as a reference for other higher education institutions to adopt serious games in the context of an entrepreneurship course.

As future work, it would be relevant to increase the number of pupils involved in this initiative. Currently, only students attending the 2018/19 school year entrepreneurship discipline have been included in this study, but prospectively it is intended to include students from upcoming editions of this course. Furthermore, and entrepreneurship being a multidisciplinary theme, it is pertinent to involve other engineering and social sciences courses in this discipline. Another aspect to be assessed is the impact of the use of FLIGBY on students' academic performance. To this end, it is important to explore how performance in each of the 29 MAP dimensions has an impact on students' academic success through the use of non-parametric and parametric statistical techniques. Finally, it is also relevant to explore the use of FLIGBY as a coaching tool in which students can practice the development of skills in which they experience greater difficulty.

**Author Contributions:** Conceptualization: Z.B.; formal analysis: Z.B. and F.A.; investigation: Z.B. and F.A.; methodology: Z.B. and F.A.; project administration: Z.B.; resources: Z.B.; supervision: Z.B.; validation: Z.B.; visualization: F.A.; writing—original draft: F.A.; writing—review and editing: Z.B.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The first author is the leader of the "Leadership&Flow" Management Board and was involved in the design and development of FLIGBY. This manuscript presents a case study of a higher education institution that adopts FLIGBY as a serious game in an entrepreneurship course.

#### **References**


© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
