Digital Storytelling in a Flipped Classroom for Effective Learning

Abstract

Conclusive empirical evidence on whether virtual classrooms result in higher performance, satisfaction, or an improvement in problem solving skills when compared with traditional face-to-face lecturing does not exist. Various studies point in different directions. However, blended learning outperforms the traditional classroom in student performance and satisfaction. A flipped classroom is one type of blended learning. For more than 20 years, this approach has been used at a European executive MBA (EMBA) program delivering online content combined with six residential weeks where students collaborate and reflect upon their online learning. Our research examined the overall setup of this program, and assessed one course in depth. As part of the course—International Management—an intercultural negotiation project was chosen to highlight the integration of online and offline activities. The flipped classroom is a demonstration of the reform-based teaching approach. The power of reform-based learning in executive education is the engaging combination of practice and theory, which improves the performance of executives. The participants considered the flipped approach exciting, dynamic, and insightful. The emphasis on a negotiation process involving classmates from around the world increased their global understanding. Beginning with a negotiation experience in the digital story project gave them a better appreciation of the relevant theories, techniques, and applications. Focusing on the practice of international negotiation and a cross-cultural analysis with reflection on cultural intelligence improved the competencies of the participants both during the course and after it.

1. Introduction

The flipped classroom engages students by delivering content outside the classroom and focusing classroom time on collaborative exercises, question and answer sessions, and other interactive communication [1]. It is a reversed setting of a traditional lecture insofar as content is delivered online in the first stage, and followed by face-to-face sessions in the second stage. Some authors, therefore, call it an inverted or reversed classroom [2]. Blended learning combines face-to-face with computer mediated instruction [3]. A virtual classroom is an online learning environment which is often web-based [4].

Recent research indicates that a flipped classroom setting is well received by participants because it increases motivation, performance, and learning outcomes [5,6,7]. Since the actual course content has to be delivered outside the classroom, an e-learning component is an essential part of any flipped classroom. The way in which such content is delivered can take various formats, such as video clips, animations, ebooks, and even traditional print media. Whether a virtual classroom outperforms a traditional classroom setting is an open question. The main challenge is to determine differences by having a control group.

Table 1. Virtual vs traditional classroom.

Author	Affects	Effect
Williams, 1996 [8]	Study at own pace	yes: positive
Laurillard, 2002 [9]	Study at own pace	yes: positive
Chou and Liu, 2005 [10]	Performance	yes: positive
Piccoli et al., 2001 [11]	Performance	no
de Jong et al., 2013 [12]	Performance	no
Li et al., 2014 [13]	Performance	no
Stöhr et al., 2016 [15]	Performance	no
Piccoli et al., 2001 [12]	Satisfaction	yes: negative
Chou and Liu, 2005 [10]	Satisfaction	yes: negative
Cole et al., 2014 [15]	Satisfaction	no
Piccoli et al., 2001 [12]	Self-efficiency	yes: positive
Chou and Liu, 2005 [10]	Self-efficiency	yes: positive
Chou and Liu, 2005 [10]	Higher-level thinking	yes: positive
Li et al., 2014 [13]	Higher-level thinking	yes: positive

shows a brief summary of comparisons of the two types. Although various authors have been using different definitions of performance, it generally relates to achieving learning outcomes as specified in the curriculum.

Table 1 gives a mixed picture, which leads to speculation about the advantages of a virtual classroom. This paper will not give a comprehensive meta-analysis on the topic, because it would require a deeper analysis. For example, satisfaction should be differentiated further into convenience and interaction with peers and tutors. It might well be that e-learning is more convenient for participants but lacks interaction.

Table 2. Blended learning vs traditional classroom.

Author	Affects	Effect
Tayebinik and Puteh, 2013 [16]	Performance	yes: positive
Aljahni et al., 2014 [17]	Performance	yes: positive
O'Flaherty and Phillips, 2015 [7]	Satisfaction & Performance	yes: positive, depending on subject
Tayebinik and Puteh, 2013 [16]	Satisfaction	yes: positive
Aljahni et al., 2014 [17]	Satisfaction	yes: positive
Baba et al., 2014 [18]	Satisfaction	yes: positive
Cole et al., 2014 [15]	Satisfaction	yes: positive
Lam et al., 2015 [19]	Satisfaction	yes: positive
O'Flaherty and Phillips, 2015 [7]	Satisfaction	yes: positive

looks at the blended learning research, which shows a more positive picture. Blended learning seems to have a positive effect on both student academic performance and satisfaction. All nine studies reviewed were positive. However, O'Flaherty and Phillips [7] found in their scoping review of 28 flipped learning studies that there is limited evidence on learning outcomes, and results may depend on the actual subject.

During the face-to-face classroom phase, questions and exercises play an important role in engaging students [20]. Abdulrahman and Al-Zahrani [21] provided empirical evidence that a flipped classroom increases student creativity when compared to a control group in a traditional setting. However, the fact that a classroom is flipped does not make it automatically more engaging or creativity-enhancing. Technology is only a means to an end. The instructional approach determines learning outcomes, not the technology itself [8,12]. Stöhr et al. [14] found no empirical evidence that virtual learning environments are better or worse than the traditional type, but that this depends on the learning activities that should train students to develop higher-order thinking skills. For this, they need appropriate instructional approaches supported by modern technology and an originality of teaching methods that create the desired skills for the participants [20]. This teaching approach can be based on authentic events that students have to describe and later reflect upon in a group setting by applying academic models [23]. Vaughan [24] warns, however, not to overburden students by using too many forms of assessment to cover too much material. Students can quickly become overwhelmed with content, feel over-assessed, and forget about key concepts.

A critical success factor is the openness of the instructors. They are often unwilling to change [25]. Maarop and Embi [26] observed that instructors are often struggling to design engaging and balanced courses. One reason might be that any change will imply the redesign of a syllabus and a teaching style.

Snelling et al. [27] suggest a seven-step framework to design and evaluate flipped classes, starting with learning objectives/outcomes, pre-class activities, actual delivery online, and followed by face-to-face and final evaluation. Pre-class activities are important because at the initial online phase the lesser motivated participants may be passive if not supported enough. Green [28] stressed the importance of engaging in a dialogue with participants in the pre-course phase through blogs and discussion fora.

In summary, there is little evidence to support a conclusion that e-learning performs better than traditional classroom learning. However, a combination of both in the form of blended learning can outperform a single approach, which is supported by Table 2.

2. Materials and Methods

The Euro*MBA implemented the “flipped classroom” concept more than 20 years ago, before it became a popular term. The program is a postgraduate executive MBA (EMBA) program led by a consortium of six European universities. In 1996, they decided to have a joint blended learning EMBA program with the actual course content delivered online, plus six residential weeks (RW) held once at each campus that serves as a face-to-face meeting place. During the RWs, the students work on projects and collaborative exercises, which is combined with one company visit and some lectures. The whole program lasts two years. On average, students have 10.3 years work experience, and each class ranges between 50 and 70 students representing around 20 different nationalities. The program is organized in trimesters. There are two intakes per year. After two online courses—each lasting for seven weeks—one RW follows.

The content is delivered online using IBM Connections in its cloud version. The system differs from traditional LMS (Blackboard, Moodle etc.) in its specific design. It is a collaborative system similar to Facebook. A course is no longer a highly-structured set of topics, but a community with subcommunities who can communicate with other (sub)communities. The students were highly appreciative of the transition from a more traditional LMS to Connections. Each course, depending on the faculty, may include videos, a blog, a wiki, formative assessments, simulations (Excel), discussion fora, team and individual assignments, webinars, and guest speakers.

The organization of the RWs is done by the partner universities. The content is given as part of the curriculum. We follow the seven-step framework as suggested by Snelling et al. [27]. Step 1 defines learning outcomes and key concepts. Each course has learning objectives, which are shown in

Table 3. Learning Objectives for the International Management Course.

Dublin Descriptors (DD)	Associated General Learning Goals	Learning Objective Fined Tuned to the Course
DD1: Knowledge and Understanding	Students are able to identify and interpret global opportunities in the international business environment	Student identifies and recognizes the core concepts, methods, theories, and tools in intercultural management
DD2: Application of knowledge	Students are able to draw upon deep functional knowledge and perspectives to analyze and solve global business challenges	Students will be able to apply the correct tools to support global decision-making
DD3: Judgment	Students are able to assess and make responsible and inclusive decisions for all relevant stakeholders	Students will be able to analyze cultural distance and use this information in the decision-making process
		Students will be able to analyze actors in real-life negotiation stories critically by measuring cultural distance and cultural intelligence
DD4: Communication	Students are able to communicate effectively their conclusions to superiors, peers, subordinates, and external stakeholders within a multicultural context	Students will be able to write in a clear academic style
DD5: Self-Direction	Students are able to contribute both creative and sustainable solutions to real business problems	Students are able to collect their own information, data, and/or research papers to support their own argumentation

.

Step 2 refers to the Implementation Strategy. The setup of this European MBA program is unique in the world. Six universities from five different countries agreed on quality standards. They jointly issue a degree, share profit and loss, and commit faculty to teaching in the program. Teaching (mentoring) hours are part of the overall teaching load at the seconding university. Although all the classes are taught in a flipped manner, we analyze a specific course—International Management—in more detail to present exactly how the course is taught. International Management course introductions are always given at Maastricht University (NL) during the January RW. During an introduction, students will learn about the course’s objectives, assignments, available resources, webinar schedule, and assessment rubrics, which are shown in

Table 4. Assessment Rubrics for the Digital Storytelling Project (GA).

Associated General Learning Goals	Learning Objective Fined Tuned to the Course	Grading Criteria	Weight
Students are able to identify and interpret global opportunities in the international business environment	The student identifies and recognizes the core concepts, methods, theories, and tools in intercultural management	The student identifies the correct method or tool needed for the analysis	10%
Students are able to draw upon deep functional knowledge and perspectives to analyze and solve global business challenges	Students will be able to apply the correct tools to support global decision-making	The students are able to apply cross-cultural models, tools, or methods for the analysis of global business negotiations	20%
		The student is able to compare negotiation styles globally
Students are able to assess and make responsible and inclusive decisions for all relevant stakeholders	Students will be able to analyze cultural distance and use this information in the decision-making process	The recommendations are relevant	30%
	Students will be able to analyze actors in real-life negotiation stories critically by measuring cultural distance and cultural intelligence	The recommendations are consistent with the analysis of cultural distance and cultural intelligence
Students are able to communicate effectively their conclusions to superiors, peers, subordinates, and external stakeholders within a multicultural context	Students will be able to write in a clear academic style	Assignment is logically structured	10%
		The student makes use of clear referencing throughout the text
Students are able to contribute both creative and sustainable solutions to real business problems	Students are able to collect their own information, data, and/or research papers to support their own argumentation	The student is able to identify and collect relevant data for the analysis of intercultural negotiations	30%
		The student is in a position to learn from the analysis of intercultural negotiations and apply it to future negotiations
Total			100%

. The final grade is constituted of four parts: an Individual Assignment (IA), a Group Assignment (GA, digital storytelling project), forum contributions, and a final exam, each carrying a weight of 25%.

There is no Step 3 in the form of key concepts and related content that students will need to remember and understand before the commencement of the online class. Step 4, class activities, includes four assessment components. For the IA, the students can choose between a case study, writing their own case study, or a company project. Since they are executive students, they are encouraged to write their own case study accompanied by a teaching note in the form of a caselet between 4–5 pages with three components: the context, the task, and the guide to the task. Around 10 hyperlinks pointing to background information should be provided, including some visualization (pictures) of the problem/task. It is mandatory that all three options relate to at least one course topic. The best cases will be used for future classes, and some are even published. The final exam is also based on a case study, which will be given out during the course introduction. The exam itself will take place at the following RW right after the online course. The whole course consists of seven units. Each unit has one forum question. Students are expected to post four to five thoughtful contributions per unit onto the forum board. Webinars are held every Sunday at 3 p.m. CET to allow students from Asia as well as the Americas to participate. The weekly topic reflects the subject of the current unit and often includes a guest speaker. Students are encouraged to host webinars or at least contribute their own experiences. Besides graded assessments, there are also formative ones. Each day the students receive a MCQ (multiple choice question) that relates to the subject. It also serves as an indicator for what page of the textbook they should be at. Executive students tend to shift the main learning activity to weekends, and may become overwhelmed when seeing many posts on the forum, or having textbook and journal articles to catch up with. Joining a discussion at an early stage or late in the process is one criterion of forum assessment. MCQs should help them to pace themselves.

Step 5—delivering the class—happens online. Executive students cannot leave their jobs and meet face-to-face during the seven-week online course. Interactions happen via webinar, discussion forum, and email. Each forum thread will be summarized by the instructor at the end of the unit. Each unit contains at least one journal article that forms the basis for the forum discussion. The students receive special coaching when writing their own case studies or doing a project as an IA. For the GA, there are subcommunities where students can interact, including virtual meetings. Most communication between students and instructors happens through email. One reason is the time zone difference. However, virtual meetings come into play when there is a more complex issue to discuss, such as writing a case study.

Step 6—linking the post-class activities and the assessment—takes the form of a final exam at the RW and seminars on the topic. It also includes a company visit. RWs last for seven full days starting at 8 a.m. (after breakfast) and finish at 7.30 p.m., followed by dinner, and with a 1.5 h lunch break. During the RW, the concepts taught will be discussed and questions answered if this has not been done already during the online course. This is also the time when peers interact by asking each other questions. For example, when they see each others‘ GAs, they can ask for the rationale, the outcome, or latest updates on what has happened in the meantime. A holistic summary of all GAs has been done and posted online by the instructor. Step 7—evaluation of the class—will be discussed in Section 3.4, Student Evaluation.

Five classes and a total of 276 (N) students were the sample for this research. Each student was given the textbook called International Management, by Fred Luthans and Jonathan Doh. In addition, each week, they received a scholarly article that fit the unit’s topic. The discussion forum played a major part in the course, and was weighted at 25% of the final grade. As a group assignment, also weighted at 25%, they had to create a real-life intercultural negotiation. There were three parts to this assignment: (1) the realistic story itself narrated as it happened; (2) an interpretation of the story based on cross-cultural and negotiation frameworks; and (3) an assessment (survey) of the actors in the story.

To narrate the story, they had to develop a script, and then make a video or use animation software to bring the actors to life. Not all teams managed to handle the video and animation software, so the option was given to use powerpoint, prezi, a blog, a website and so forth, as long as it could elaborate the story. Stories had to be based on real events that happened to the participants. With an average of more than 10 years work experience, the students had no problem coming up with such authentic negotiations. Figure 1 illustrates one sample story: in it, a woman accuses an expatriate working in Equatorial Guinea of having promised to marry her. Since he is unwilling to do so, her family kidnaps him and asks for US $200,000 for his release. When his friends involve the police, the negotiation becomes even trickier because the policeman wants to have his cut.

As a part of the flipped classroom, there were three phases of the course: (1) a pre-course introduction during the RW; (2) the online delivery of content; and (3) another RW. The introduction of the course took place at the RW prior to the seven-week course. It generally lasts half a day. The main idea is to see the lecturer in person, and get a course overview which includes a briefing on the group assignment, which is in this case an intercultural negotiation. The introduction takes place on Monday, so students have the whole week to discuss it face-to-face. A team consists of six students, some of whom shoot videos during the week, while others just draft the rough script for an animation. Each team had to develop six stories i.e., one story per team member. To prevent bias before the group assignment, students were not given any theoretical input. The story was instructed to be told as it happened. Had the students been instructed in cross-cultural management, they might have added some details which did not happen to support an interpretation later. In this respect, it is better that they did not know anything about the theory. During the online phase, they had to interpret the stories in the light of cross-cultural frameworks (especially cultural distance).

Table 5. Cultural distance.

Dimension/Author	GLOBE	Hall	Hofstede	Schwartz	Triandis	Trompenaars
Achievement				X		X
Ascription						X
Assertiveness	X
Benevolence				X
Collectivism	X		X		X	X
Communitarism						X
Conformity				X
Confucian			X
Diffuse						X
Emotional						X
Femininity			X
Future orientation	X
Gender differentiation	X
Hedonism				X
High context		X
Humane orientation	X
Individualism	X		X		X	X
Indulgence			X
In-group	X
Low context		X
Masculinity			X
Neutral						X
Particularism						X
Performance orientation	X
Power				X
Power distance	X		X
Restraint			X
Security				X
Self-Direction				X
Societal	X
Specific						X
Stimulation				X
Time Orientation			X			X
Tradition				X
Uncertainty avoidance	X		X
Universalism				X		X

shows the main indicators that are used to measure cultural distance, and the corresponding authors.

These six frameworks represent the mainstream models to measure cultural distance. Hofstede’s [29] Cultural Dimensions explain why people from different cultures behave as they do. In its original form, it consisted of four dimensions, but was later expanded to six [30]. One frequently used dimension to measure cultural distance is ‘Power Distance’, which measures the extent to which less powerful people (for example, an employee) accept more powerful people (for example, his/her superior) [29]. Trompenaars and Hampden-Turner [31] built upon Hofstede’s [29] findings, and based their research on 15,000 managers from 28 countries. They derived five relationship orientations—how people deal with each other—that can be considered Cultural Dimensions. Although having used different terms, the findings are similar. For example, whereas Hofstede [29] used the term ‘Collectivism’, Trompenaars and Hampden-Turner [31] used the term ‘Communitarism’ to measure whether someone is part of a group rather than being individualistic. In the project GLOBE, a team of 170 scholars collaborated to analyze 17,000 managers from three industries [31]. Although they used a different approach, they came up with similar categories to Hofstede [27]. Hall [31] divided cultures into low and high context cultures. Higher context cultures generally have a stronger sense of group orientation, seniority, unspoken rules, and tradition [31]. For example, Japan would represent a typically high context culture, whereas the United States would represent a low context culture. Lesser known are Schwartz’s [34] Value System, and Triandis’ [35] Individualism-Collectivism model. All six frameworks are compatible despite using different research approaches and different terms.

Analyzing the behavior of the actors in a story was the third part of the assignment, and it was not graded. Students had to fill in a character description form. Besides Cultural Distance (CD), the students were taught about the concept of Cultural Intelligence (CQ), which is defined as an individual’s capability to adapt effectively to situations of cultural diversity [36], which was also part of the character description form. The students could also add their own criteria on how to measure CQ after forum discussions. After a broader consensus was achieved, the question was then added to the form.

3. Results

3.1. Digital Negotiation Story

3.1.1. Gender and Region

Analyzing the nationality of the Negotiator and the nationality of the counterpart where the negotiation took place (Host) can be a starting point for class discussion, as shown in

Table 6. Negotiation partner matrix.

Gender	Negotiator	Male				Female
Host		America	Asia	Europe	Africa	America	Asia	Europe	Africa	Pacific
Male	Asia	4.1	4.0	4.4		2.0		5.2
	Europe	5.1	4.2	4.8		3.5		3.2	9.0
	America	3.5	4.4	4.4		3.0		5.7
	Pacific							1.5
	Africa	6.5	3.5	5.0	2.0			4.3		9.0
Female	Asia		8.5	10.0				1.0
	Europe		5.3	2.3	1.0		3.0
	America	8.0		8.0				10.0
	Africa	4.0		1.0

. For example, a male American negotiating with an African male (in Africa) achieved a higher negotiation win-win outcome (6.5) than with an African woman (4.0); the scale ranged from 1 (lose-lose) to 10 (win-win). To determine the statistical significance, a t-test should be used, but at this initial stage of class discussion this would not be critical.

In the form of a Grounded Theory approach, the students were encouraged to come up with their own determinants to explain why certain combinations might work better than others. In a second step, these determinants were amalgamated and related to cultural distance.

3.1.2. Cultural Determinants

The most frequently used term in the stories (excluding filler words) was ‘time’, as shown in

Table 7. Terms used in stories.

Term	Count
Time	323
Agreement	72
Relationship	51
Dinner	61
Lunch	41
Respect	25
Risk	17
Trust	20
God	31
Power	26

.

The term ‘time’ was used in different contexts. Most often, it referred to a negative situation with one negotiator coming late to the meeting. Typically, the German negotiations were ‘on time’, whereas for example Arabs were not always ‘on time’, with the excuse that ‘time’ belongs to Allah. ‘Dinner’ and ‘lunch’ ranked high because in certain parts of the world, one negotiates contracts over or after a meal. It is a means to build trust and relationship. Power (distance) is a cultural dimension used by Hofstede [29].

3.2. Theories Related to Negotiation

Hofstede [29] was by far the most frequently used model for the interpretation of the stories, as shown in

Table 8. Models used in interpretations.

Author/Model	Cited	In %
Hofstede	428	70.05%
GLOBE	61	9.98%
Trompenaars	60	9.82%
Hall	34	5.56%
Schwartz	28	4.58%
Triandis	0	0.00%

. This is consistent with the number of citations in the field of cross-cultural management, as well as critics [37].

Although heavily criticized because of its research methodology, simplification of the complexity of culture, and disregard for cultural convergence, Hofstede [29] is still the cross-cultural model that is most widely applied in academia, as well as in the interpretations of our negotiation stories.

3.3. Character Description

Students had to fill in a character description form. A linear stepwise regression with negotiation outcome (defined as win-win to lose-lose) as the dependent variable; the result is shown in Equation (1). R² was 0.73.

Negotiation Outcome = 0.598 CD Performance + 0.424 CD Humane Orientation – 0.335 Time Orientation + 0.183 CQ Adjustment.

(1)

Performance and humane orientation (relationship) are CD factors with the highest impact on a successful (win-win) outcome. Time orientation is a concept developed by Hofstede and Bond [36], which is also referred to as Confucian Dynamism. Interestingly, ‘time’ was the most frequently used term in the stories. The minus sign in Equation (1) represents short-term orientation with a focus on achieving quick results. Adjustment, the fourth predictor, is a CQ attribute that describes a person who adjusts cultural knowledge when interacting with people from another culture.

All three parts of the assignment demonstrate an example of the reform-based teaching practice [39] that encourages student ownership of the learning process. Deslauriers et al. [40] found that the reform-based teaching practice resulted in an increase in student attendance, and twice the engagement—based on two sections (N = 267 and N = 271) of the same course—for the flipped section when compared with the traditional section. In our case, the students created their own multimedia learning materials (the digital stories), and applied the course content as an interpretation.

3.4. Student Evaluation

3.4.1. Residential Weeks

When the students were asked whether RWs were relevant for their professional development, they ranked it a 4.16 on a 1–5 Likert Scale. The top three reasons were: (1) productive team work; (2) an opportunity to show leadership in a team; and (3) constructive conflicts. The course introductions for International Management, given during RWs, received an average of 4.29 for providing a clear rationale for the course. The lowest ranked features were non-academic RW features such as food/drink/dinners with an average of 2.79, as shown in

Table 9. Residential weeks (RW) Evaluation.

Statement	Mean	Std. Deviation
I was satisfied with the organization of the week	4.53	0.718
International Management: the course introduction made clear what the rationale was	4.29	0.672
The objectives of the residential week were met	4.34	0.701
I found the integration activity an effective way of getting to know my team members	4.31	0.896
I found the theme of the residential week relevant for my professional development	4.16	0.907
I was satisfied with the lunch and dinners	2.79	1.092

.

Students were also asked to provide open-ended responses. Around 90% of students provided open-ended responses. These include:

-

I definitely recommend keeping <the RW> as these soft factor learnings are becoming more and more relevant;

-

good solid RW; great experience in terms of learning;

-

some sessions were a bit theoretical but the overall program was well organized and a nice blend between business and theory;

-

the lectures were quite fun, although this is one of those topics where there is far too much philosophy created by academics who have nothing better to do;

-

the week is perfectly linked to the previous course;

-

the approach to teach the theory after working on the problem works great in that environment;

-

group work is nice for group development under pressure;

-

finally those weeks have been the perfect meeting between the two worlds: experience and knowledge.

Giving a course introduction face-to-face is appreciated by students. The classic design of a flipped classroom where students jump without any preparation into online learning should be adapted. The lesser motivated students especially might need this type of initial motivation. They interact with peers live, and have a chance to talk to the instructor. They can put a face to the name before the online course starts. Besides the official course introduction, there are opportunities to meet, for example, over lunch/dinner or a beer at the hotel bar. Such informal contacts can bring communication barriers down. Logistically, it is impossible to host more than six RWs. Executive participants are not in a position to leave their jobs for a week to attend physical classes more than three times per year.

3.4.2. Online Course Content

Some items in the evaluation were identical with the RW. For example, the question regarding the relevance for professional development was rated as 4.47; i.e., higher than the 4.16 of the RWs, as shown in

Table 10. Course Evaluation.

Statement	Mean	Std. Deviation
The course was well organized	4.66	0.55
The tutor provided adequate feedback on assignments	4.60	0.62
The tutor played an active role in the course	4.57	0.73
The course met its learning objectives	4.57	0.68
I found the subject relevant for my professional development	4.47	0.86
The course materials provided were sufficient for my learning needs	4.45	0.74
The tutor created an environment conducive to learning	4.45	0.69
The grading was fair and the assessment criteria were clear	4.37	0.76
Please provide a grade between 1–5 for the course	4.30	0.54
The group assignment(s) contributed to my learning	4.27	0.87
Working in a team contributed to my learning	4.23	0.57
The discussion forum(s) contributed to my learning	4.10	0.98
The individual assignment(s) contributed to my learning	4.07	0.75
The webinars contributed to my learning	3.60	0.86
Email contributed to my learning	3.40	0.81
Skype contributed to my learning	3.31	0.85
Instant messenger/chat contributed to my learning	2.79	0.98

. This is not surprising, because the actual content is delivered online, and the RWs serve more as an interaction platform.

Overall, the International Management courses received on average a student evaluation of 4.3. The expectation that webinars would be well received because they are the main interactive component during an online course was not confirmed, as it was just rated at 3.6. Skype sessions, which students used for team discussions, were regarded as even less useful, with a rating of 3.31. Chat, which is mainly used for one-to-one communication, received a 2.79. Students are enthused by relevant content, feedback, and team projects. The apparent lack of enthusiasm for online communication is a weakness of e-learning that can be compensated for by personal face-to-face interactions during RWs.

At the end of the course, students were asked what they had learned. Qualitative feedback (95% answered) was very positive (student quotes):

-

To be honest, I think this course gives me a lot and shows me where I can improve my leadership style resp. global mindset.

-

I have already put into practice the specific Korean negotiation and behavior tips out of the ZIP file, the book and the forum in the last months.

-

What this course has taught me is to become more aware of certain cultural behavior and specific managerial styles and it made me reflect on certain situations that happened in the past.

-

I feel like I am better equipped to analyze and respond to similar situations in the future based on the knowledge gained here and experiences that are shared by others. I also realized that while I often consider the cultural differences between the Americans (and the corporate culture in our company) and ‘us Europeans’, I rarely consider the many differences closer to home—across Europe which in many ways are even more important.

-

Increased cultural awareness and more thoughtfulness in my work are certainly key items that I have gained.

-

I really had lost appreciation for this situation over time, just adopting a ‘plain Swiss’ style when working with Indians in Switzerland, but a pretty much Indian-tailored style when talking to colleagues offshore. It is really not that simple, and the course helped me reflect on that again; this <course> got me thinking that leadership goes beyond the leader/manager relation with followers and I will try to apply it from now on.

-

My Latin/South American coworkes have already asked me why, all of a sudden, I have a deeper understanding of why they miss deadlines. Well, they do not know yet that they have different (earlier) deadlines than others with the same task.

Table 11. Personal learning outcomes 2016 and 2017 for a flipped classroom.

Level	2016	2017
Awareness (DD1)	20%	34%
Application (DD2)	26%	21%
Judgment (DD3, DD5)	42%	32%
Practice	12%	13%
N	50	38

summarizes the qualitative course feedback by categorizing the responses into four broad categories.

The above four categories are in line with the Dublin Descriptors (DD). ’Practice means that the students implemented something in practice that they had learned during the course. Practical application is not a Dublin Descriptor. It requires a managerial position or at least exposure to foreign cultures to have the chance to implement something based on the course. This differentiates it from a theoretical application of knowledge (DD2). The above quoted statement mentioning a participant giving South-American workers an earlier deadline is such a practical application. There was not a single negative comment; i.e., nobody said that s/he did not learn anything.

During a RW in May 2017, one focus group and five in-depth interviews with students were conducted. The interviews showed that the main reason for choosing the program was flexibility. Even part-time EMBA programs with lectures during evenings and weekends were considered too rigid. Students want to study at their own pace, and therefore chose blended/flipped learning. A large student focus group (N = 28) discussed the flipped course International Management. The digital project was considered time consuming but insightful. Even the geographical spread around the globe was seen as positive. For example, one group had a student living in Chile, one in Germany, one in Vietnam, and one in Oman. It was a challenge to find a suitable time slot and day for teamwork due to time zone differences, and the fact that Sundays are working days in the Middle East. Other groups faced similar challenges, but it was considered a learning experience and culturally enriching. Suggestions for improvement were not related to the flipped classroom design; instead, assessment criteria were the main focus (for example, dropping the final exam in favor of an individual assignment or project work). Students showed high satisfaction with the flipped approach because it made the course more exciting. The company visits that related to International Management topics resulted in deeper learning. The topics exemplified during three company visits were: (1) global expansion strategy, and exports and logistics (company A—chemicals); (2) the adaptation of products catering for different cultures (company B—ice cream manufacturing); and (3) the integration of different organizational cultures, and strategic alliances (company C—telecom). Such integration of theory and practice was highly appreciated by the students. Some students mentioned that the course was too short because they enjoyed it so much. These findings underline the merits of a flipped classroom from a more qualitative perspective.

4. Discussion

Flipped classrooms that foster higher-order thinking skills can come in many shapes and sizes. Letting executive students narrate their own experiences as objectively as possible, and in a second step putting them into a theoretical framework, follows a reform-based teaching approach. Reform-based learning is an extension of the constructivist learning paradigm, which emphasizes critical thinking, problem solving, action learning, understanding, and applications. In this case, we assessed intercultural negotiations. In executive education, the emphasis is on competencies, which are a combination of knowledge, skills, applications, and results. The traditional approach to learning concentrates on knowledge. Most classrooms begin with concepts. Our flipped approach begins with practice. In this example, it is a reflection on the negotiation experience of the participant. This practice is analyzed using cultural analysis and related negotiation approaches. This will lead to competencies in international negotiations. It is a more effective pedagogy for executive participants. As it is designed in this flipped International Management course, it represents a better learning approach for executives, moving from the practical to the theoretical to improve practice.

To let students work on a project without prior knowledge of the subject is advantageous because they focus on the reality of their experience. If students had known cross-cultural theories before, their stories might not have been authentic.

The flip side of the online class can be in form of RWs. Evaluations shows that six RWs are highly appreciated. They can take place at one location or at different ones. Although not academically relevant, appropriate logistics should be in place. The findings show low satisfaction with food. Such small things may add to the overall impression and lower or increase the motivation of the participants.

The flipped classroom is a demonstration of the reform-based approach. Is this paper, which considers a course in international management in an executive MBA, the flipped classroom is based on the experience of a wide range of management professionals in international business negotiations. Based on this experience, they developed digital stories which distill their real understanding of what took place in the actual negotiation.

Using the digital stories, executives apply cross-cultural frameworks to specify the cultural distance of the partners involved in and related to the negotiation process. They recognize the importance of cultural intelligence to the negotiation’s success. Most importantly, these executives develop competence in the negotiation process, which relates to successful outcomes.

The informal class dimension of the flipped classroom takes place in RWs. In this phase, the emphasis is on the discussions of frameworks, theories, methodologies, and negotiation approaches that enhance the understanding of the executives’ past experience in international management. This learning process prepares them to be better international executives in the future.

This is the power of reform-based learning and the flipped classroom implemented in executive education. It is a combination of practice and theory, which improves the performance of executives in the accomplishment of learning objectives, and the assessment of those objectives for which executives demonstrated very positive results. In the open-ended comments in the course evaluations, they indicated very favorable responses to the reform-based approach to learning practice and related theories. They appreciated the impact of the course on their professional development.

The methodological limitations of this paper are that it is an actual course in an accredited MBA program. As such, there are requirements in the design and evaluation of the course which limit research potential. Analytically, our paper emphasizes more descriptive approaches because of the journal’s perspective. In another study using the same data base, a sophisticated data mining and content analysis has been applied.

In summary, the reform-based learning approach, as implemented in the flipped classroom, is extremely effective for executive education. The future challenge will be the customization of reform-based learning to cater for the needs of each individual executive without compromising learning outcomes [41].