Creativity, Intelligence, and Collaboration in 21st Century Education

An Interdisciplinary Challenge

Edited by Frédéric Darbellay, Zoe Moody and Todd Lubart

mdpi.com/journal/jintelligence

## **Creativity, Intelligence, and Collaboration in 21st Century Education: An Interdisciplinary Challenge**

## **Creativity, Intelligence, and Collaboration in 21st Century Education: An Interdisciplinary Challenge**

Editors

**Fr ´ed ´eric Darbellay Zoe Moody Todd Lubart**

*Editors* Fred´ eric Darbellay ´ University of Geneva Sion, Switzerland

Zoe Moody University of Teacher Education Valais (HEP-VS) St-Maurice, Switzerland

Todd Lubart University Paris Cite´ Paris, France

*Editorial Office* MDPI St. Alban-Anlage 66 4052 Basel, Switzerland

This is a reprint of articles from the Special Issue published online in the open access journal *Journal of Intelligence* (ISSN 2079-3200) (available at: https://www.mdpi.com/journal/jintelligence/ special issues/creativity intelligence collaboration).

For citation purposes, cite each article independently as indicated on the article page online and as indicated below:

Lastname, Firstname, Firstname Lastname, and Firstname Lastname. Article Title. *Journal Name* **Year**, *Volume Number*, Page Range.

**ISBN 978-3-0365-9473-6 (Hbk) ISBN 978-3-0365-9472-9 (PDF) doi.org/10.3390/books978-3-0365-9472-9**

© 2023 by the authors. Articles in this book are Open Access and distributed under the Creative Commons Attribution (CC BY) license. The book as a whole is distributed by MDPI under the terms and conditions of the Creative Commons Attribution-NonCommercial-NoDerivs (CC BY-NC-ND) license.

## **Contents**



## **About the Editors**

#### **Fr ´ed ´eric Darbellay**

Fred´ eric Darbellay is an associate professor of inter- and transdisciplinary studies at the ´ University of Geneva (Switzerland); head of the Inter- and Transdisciplinarity Unit at the Centre for Children's Rights Studies (CIDE); and the CIDE deputy director. His research and teaching focus on the study of inter- and transdisciplinarity as a creative process of knowledge production between and beyond disciplines. He authored multiple publications on the theory and practice of inter- and transdisciplinarity through various scientific fields across the arts, humanities, social, natural, life, and technical sciences. Fred´ eric Darbellay is a member of the Td-net (Network for Transdisciplinary ´ Research) Scientific Advisory Board of the Swiss Academies of Arts and Sciences, the Association for Interdisciplinary Studies (AIS) Board of Directors, and the Scientific Council of the Global Research Institute of Paris (GRIP).

#### **Zoe Moody**

Zoe Moody (PhD) is a professor at the University of Teacher Education Valais (HEP-VS) and senior research associate at the Centre for Children's Rights Studies, University of Geneva. Her research and teaching activities draw on disciplinary contributions mainly from social psychology, educational sciences, and childhood studies and are rooted in the interdisciplinary field of children's rights studies. Her research focuses on children, their social interactions, developmental trajectories, and learning to live together. Zoe Moody also carries out epistemological work. Ethical issues relating to the position of children as actors and subjects rather than simply objects of research are at the heart of her reflections and methodological developments. Zoe Moody is currently a member of the Board of Directors of the Exploration Collection (Peter Lang), as well as of the editorial board of the *Journal of Participatory Research Methods* and the *Swiss Journal of Educational Sciences*.

#### **Todd Lubart**

Todd Lubart obtained his PhD from Yale University. He is a professor of psychology at the University of Paris Cite. Former director of an applied psychology research laboratory and ´ coordinator of several grants and contracts, he has numerous publications in articles, books, and psychological tests. His current lines of research involve measures of creative potential, environmental support for creativity using virtual reality, the development of creativity through game play, and the use of generative AI for creativity. Todd Lubart serves on the editorial board of several journals concerning creativity and innovation, received the Berlyne award from the American Psychological Association, the NAGC Torrance Award, and was a junior member of the Institut Universitaire de France. He is president of the non-profit scientific society ISSCI (the International Society for the Study of Creativity and Innovation).

## **Preface**

In the intricate fabric of the 21st century, humanity grapples with a plethora of interconnected challenges, ranging from climate change and health crises to profound social, political, and cultural transformations. These issues, characterized by their multifaceted and interdependent nature (i.e., natural, biological, social, and psychological dimensions), demand a global, interdisciplinary, and collaborative approach. Education occupies a pivotal position in addressing these pressing issues. Spanning from early childhood education to university-level learning, it serves as the crucible wherein future generations are shaped. Beyond instilling fundamental disciplinary knowledge, education equips individuals with the essential skills to articulate disciplines and navigate the complexities of our dynamic and uncertain world. This volume examines the intricate interplay between education and the exigencies of the 21st century. The subsequent pages provide a diverse tapestry of perspectives, delving into the assimilation of transversal competencies and creative practices across diverse institutional settings and training programs. It articulates theoretical reflections, research findings, and practical experiences, prompting profound inquiries into the adaptability of educational systems, the reconceptualization of intelligence, and the cultivation of indispensable 21st-century skills such as critical thinking, collaboration, creativity, and digital literacy. Within this compilation, readers will discover an array of contributions spanning various disciplines, including psychology, creativity studies, and educational sciences. The contributors delve into multifaceted realms, conducting literature reviews, engaging in epistemological reflections, and presenting research findings. Notably, they emphasize the intricate interplay between teacher and student creativity, as well as the broader dynamics of learning and teaching, with a specific focus on STEM education. Furthermore, the exploration extends to the transformative influence of digital technology and artificial intelligence on the overall landscape of learning experiences. This collection serves as a compass, guiding educators, policymakers, researchers, and curious minds alike through the labyrinth of educational challenges in the 21st century. It invites readers to contemplate the future of education, to question assumptions, and to envision a landscape where creativity, collaboration, and critical thinking form the bedrock of a resilient and adaptable society. Embark on this intellectual journey, traverse the diverse terrains of knowledge, and emerge with a deeper understanding of the intricate interplay between education and the challenges that define our era.

> **Fr ´ed ´eric Darbellay, Zoe Moody, and Todd Lubart** *Editors*

## *Concept Paper* **Accepting the Challenge: Helping Schools Get Smarter about Supporting Students' Creative Collaboration and Communication in a Changing World**

**Ronald A. Beghetto 1,\* and Ed Madison <sup>2</sup>**


**Abstract:** Although the purpose of schools can be (and has been) debated, one common goal that most people agree upon is that schools can and should play a role in preparing young people for the complexities of the future. This goal is somewhat paradoxical in that the future is unknown. So how might schools prepare young people for the unknowable? The prototypical response has been to design learning experiences based on what is already known in the hope that the knowledge, skills, and experiences in school will be durable enough to equip students for navigating the complexities of the problems they will encounter in the future. Consequently, most of what students learn in schools is predetermined. Although we recognize that some of these experiences can be beneficial for helping students in the future, we assert in this concept paper that schools can (and need to) get smarter about the kinds of educational experiences that students engage with if we are to prepare them for addressing the uncertainty of complex problems that they face now and into the future. More specifically, we open this concept paper by briefly discussing the prototypical curricular experience that schools provide young people and how these experiences sometimes fall short in providing students with the opportunities, experience, and confidence necessary to creatively engage with, resolve, and communicate about their experiences addressing complex problems. We then introduce a collaborative creative curricular experience called *Journalistic Legacy Challenges* (JLC). JLCs can support students in learning how to identify, address, document and communicate about complex problems that can make a difference in their communities and in their own and others' lives. The experiences offered by JLCs differ from prototypical learning experiences because they require young people to identify problems that matter to them, collaborate with skilled others to address those problems, develop their creative confidence, and learn how to use *journalistic learning* to document and communicate about their work to broader audiences.

**Keywords:** 21st century learning; creativity; creative collaboration; creative communication; creative confidence; creative curricular experiences; journalistic legacy challenges; journalistic learning

### **1. Introduction**

We live in increasingly challenging and complex times. Scholars have described life in the current epoch as posing new and pressing concerns that present existential threats to life as we know it (Barr et al. 2022; Leahy et al. 2022). Rapid social, ecological, technological, and geopolitical transformations give rise to critically important questions for how we might better prepare ourselves and future generations for addressing current and impending uncertainties. Schools have long been recognized as important socio-cultural institutions with a central goal of preparing young people to develop into productive local and global citizens.

Although the specific purpose of schools can be (and has been) debated (Hannon and Peterson 2021), most people agree that schools can and should play a role in preparing

**Citation:** Beghetto, Ronald A., and Ed Madison. 2022. Accepting the Challenge: Helping Schools Get Smarter about Supporting Students' Creative Collaboration and Communication in a Changing World. *Journal of Intelligence* 10: 80. https://doi.org/10.3390/ jintelligence10040080

Received: 15 July 2022 Accepted: 28 September 2022 Published: 8 October 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

1

young people for the future. This goal is somewhat paradoxical in that the future is unknown. So how might schools prepare young people for the unknowable? The prototypical response has been to design learning experiences based on what is already known in the hope that the knowledge, skills, and experiences in school will be durable enough to equip students for navigating the complexities of the problems they will encounter in the future. When students question the relevance of curricular content, a common assurance provided by educators is that they should learn what is being taught because "they will need and use it *someday*."

Equipped with smartphones offering instant access to unlimited information, the "someday" promise can, however, seem ambiguous, intangible, and disconnected from students' present reality and lived experiences. Consequently, most of what students learn in schools is predetermined and sometimes perceived as irrelevant (at least with respect to students' lived reality). Although we recognize that the curricular experiences students typically have in schools can be beneficial for students now and into the future, we assert in this concept paper that schools can (and need to) get smarter about the kinds of educational experiences that students engage with if we are to better prepare them for uncertain futures.

More specifically, we open by briefly discussing the prototypical curricular experience that schools provide young people and how these experiences can fall short in providing students with the opportunities, experience, and confidence necessary to engage with, resolve, and communicate about their experiences addressing complex problems. We then introduce a collaborative creative curricular experience called *Journalistic Legacy Challenges* (JLC). JLCs can support students in learning how to identify, address, document and communicate about complex problems that can make a difference in their own and others' lives. The experiences offered by JLCs differ from prototypical learning experiences because they require young people to identify complex problems that matter to them (Getzels 1964; Pretz et al. 2003), creatively collaborate with skilled others to address the problems (Hämäläinen and Vähäsantanen 2011; Moran and John-Steiner 2004), develop their creative confidence (Karwowski et al. 2019) and learn how to use journalistic learning to document and communicate about their work to broader audiences.

#### **2. Prototypical Curricular Experiences**

One way to understand the prototypical curricular experience in schools and classrooms is to recognize that it tends to focus on sameness and certainty (Glăveanu and Beghetto 2016). More specifically, the same group of students typically are required to work through routine problems in the same way, at the same time, and produce the same result. Indeed, much of what students experience in school is predetermined and focused on individual accomplishments. Moreover, the student experience and products of the work tend to be ephemeral and discarded. Other than what students carry in their memories and document in their class notes, the typical documentation and curation of "what students did and learned" tends to be reduced to a numerical score or letter grade in the teachers' gradebook and on student's report cards and transcripts. We are not suggesting that this is the fault of teachers or that there are not individual teachers who have broken the prototypical mold by teaching for and with creativity. Rather, we are highlighting systematic and long-lasting features of the typical curricular experience (see Claxton 2008; Sirotnik 1983; Beghetto and Zhao 2022).

Moreover, when it comes to preparing young people for uncertain futures, we see at least three areas where the typical curricular approach seems to fall short in providing opportunities for students to develop their creative confidence, engage in creative collaboration, and learn how to communicate creatively. *Creative confidence* refers to a belief in one's ability to think or act creatively in and across performance domains (Karwowski et al. 2019). *Creative collaboration* involves students working and problem solving together by actively seeking out alternative perspectives, working with skilled others, and building on each other's strengths to produce new ideas (Baruah and Paulus 2019; Etelapelto and Lahti 2008; Lubart 2018; Moran and John-Steiner 2004). Additionally, *creative communication* in-

volves learning how to articulate the merit and value of creative work to various audiences (Glăveanu 2013; Plucker 2022). In what follows, we briefly discuss how the prototypical curricular approach can limit opportunities for developing these components of creativity development.

First, with respect to creative confidence, the typical approach in schools is to have students learn how to efficiently and effectively do what is expected and how it is expected (Beghetto 2018a) by working through routine problems (Pólya 1966), rather than develop confidence in identifying their own problems to solve and their own ways of solving them. Routine problems are "pseudo-problems" (Getzels 1964) because they have already been solved. In a school context, routine problems represent learning "exercises" rather than actual problems (Robertson 2017). This is not to say that routine problems lack educational value. Indeed, routine problems play a prominent and important role in teaching and learning because they help students learn how to address existing problems using standard procedures (Lee and Anderson 2013).

The issue with routine problems arises when they become the predominant type of problem students encounter. In order for students to develop their creative confidence necessary for navigating the complexities of the future, they need to also have opportunities to 'find' (Runco and Chand 1994) and work through 'ill-defined' problems (Pretz et al. 2003). Indeed, as Pólya (1966) has argued, unlike routine problems, ill-defined or non-routine problems "demand some degree of creativity and originality" (p. 127), because these kinds of problems have some level of uncertainty in how they can and should be addressed. Consequently, for students to develop the self-efficacy necessary to engage, persist, and creatively work through ill-defined problems that they face now and into the future, they need opportunities and experiences that require them to identify and tackle ill-defined problems in otherwise structured and supportive learning environments (Bandura 1997; Beghetto 2018b). Indeed, engaging with ill-defined problems require students to seek out new experiences and accept unpredictability, which have been found to be associated with students who have higher levels of creative-self efficacy (Karwowski 2012). Such experiences are also curiosity driven and curiosity is a trait that has been found in prior research to be positively associated with creative self-efficacy in students (see Karwowski 2012; Puente-Diaz and Cavazos-Arroyo 2018).

Next, with respect to creative collaboration (Baruah and Paulus 2019; Lubart 2018; Moran and John-Steiner 2004), much of the curricula in school tends to be focused on individual learning and individual students demonstrating what they know and are capable of doing on their own. Working with others is often forbidden and sometimes even considered cheating (e.g., "do your own work," "no talking when working on this assignment," and so on). Conversely, when students have the opportunity to learn how to creatively work together, they can learn how to build on their own and others' strengths in order to do more than what they might otherwise be able to do individually (Moran and John-Steiner 2004). Even primary school children can learn how to creatively collaborate to support their learning and creativity development. Rojas-Drummond and colleagues (Rojas-Drummond et al. 2008), for instance, report on a study of primary school students in Mexico City who successfully "learned how to collaborate" and "collaborated to learn" and, in turn, were able to collaboratively produce creative writing and multimedia projects.

This is not to say there is no collaboration in school. Indeed, group work is often a common feature of many classrooms throughout young people's schooling experience. Collaboration in the form of group work, however, often serves as a very narrow representation of collaborative effort. Indeed, "group work" in schools is often not an experience of creative collaboration, but rather students tend to be assigned to groups by the teacher and typically one or two students do most of the work (Chiriac and Granström 2012). This inequitable distribution of who does the work can even occur in situations where students form their own groups, because the goal is often to meet the criteria as quickly as possible. Consequently, students sometimes get upset when working in groups, because they have concerns about equitable distribution of the work and worry about how the

group's efforts (or lack thereof) will impact their individual grade (and, ultimately, their grade point average). Moreover, once groups are formed, students tend not to have an opportunity to disband and join other groups or collaborate with people outside the classroom, which is often required when identifying and working creatively to solve complex problems (Beghetto 2018a; Moran and John-Steiner 2004). In sum, students have limited opportunities to learn how to creatively collaborate in school, particularly when it comes to working together to identify problems to creatively solve that are important to them and that can make a positive impact on others.

Finally, with respect to creative communication, students also tend to have limited opportunities to learn how to effectively document and communicate their creative work to others (Madison 2015; Plucker 2022). If students have opportunities to learn how to communicate their creative work to diverse audiences and receive feedback on that work, then they can develop important creative self-regulation skills necessary for planning, realistic goal setting, managing emotions, and overall improvement of their creative efforts (Ivcevic and Nusbaum 2017). Again, this is not to say that students do not receive helpful feedback from teachers and peers in school. The focus of that feedback, however, tends to be aimed at making sure students are "on track" to meet expectations in expected ways, rather than supporting students' creative self-regulation and creative communication skills.

Along similar lines, when students have opportunities to present their final work or projects to others (in the form of class presentations), the focus is often on the finished product and not the creative process that resulted in those products including the setbacks, failures, and learning that they experienced throughout the process. Given that experiencing setbacks and failure can play an important role in promoting thinking skills and creativity (Manalo and Kapur 2018; Kapur 2016; von Thienen et al. 2017), it is important that schools provide students and teachers with opportunities to communicate about both their successes and their failures so that young people can learn how to anticipate and grow from setbacks as well as their accomplishments (Beghetto and McBain 2022).

#### **3. Creative Curricular Experiences**

Creative curricular experiences (CCEs) represent a shift away from prototypical curricular experiences (PCEs), which tend to have a more *transactional* logic, and toward more *transformative* educational experiences. Transactional educational experiences are based on the logic of: "If you do this work, in this way, then you can expect to receive this grade." Conversely, *transformative* educational experiences focus on doing creative work that can result in positive changes and contributions to others (Beghetto and Glăveanu 2022; Lubart 2018; Sternberg 2021). CCEs, like many creative experiences, include the following features (adapted from Glăveanu and Beghetto 2020):


Unlike the transactional focus of PCEs whereby academic learning often serves as a means to its own end, CCEs provide a means for young people to put their existing and developing academic learning to creative work. Importantly, CCEs can be infused in the everyday curriculum and thereby can serve to democratize creative learning experiences for students who have traditionally not had the access or opportunities to participate in creative learning. Indeed, CCEs can be designed to include *all* students rather than be restricted to only a tiny proportion of students identified and selected for gifted and talented programs or who otherwise have the privilege, access, and opportunity to participate in extra-curricular creative learning experiences.

Fortunately, efforts are underway to broaden CCEs for all students (see Beghetto and Zhao 2022; Boss 2017; Zhao 2021). There are even examples of schools and entire school districts that have made important strides in providing students with opportunities to engage in creative learning by tackling authentic community challenges (Boss 2017). Iowa BIG (https://iowabig.org/, accessed on 27 September 2022), for instance, is an initiative that was developed to partner students with local businesses and organizations to engage young people in solving real-world problems. Example projects include everything from efforts aimed at reducing stigmas about mental illness to using hydroponics to grow food (Cedar Rapids Economic Alliance 2019).

If we are serious about supporting students in their development of creative confidence, collaboration, and communication necessary for navigating increasingly complex problems in a rapidly changing world, then we would argue that *all* schools need to get smarter about the kinds of curricular experiences they provide students. We further assert that CCEs should be available to *all* students and not limited to students in specialized, gifted or extra-curricular programs.

In arguing for the inclusion of CCEs in schools and classrooms, we are not suggesting that existing curricular experiences lack merit and should be abandoned. We are also not arguing for "add-on" curricular experiences as we recognize that educators already operate under severe time constraints and may feel that they are already being asked to do more than what they have time to do. Rather, we assert that schools can get smarter about how curricular time is used and the kinds of curricular experiences offered to students. Educators can infuse CCE's in their curriculum (see Renzulli 2016) by replacing fully predetermined learning experiences with more creative learning experiences (see also Beghetto 2018b for a more in-depth discussion of how this can be accomplished).

In what follows we introduce a particular type of CCE, called *Journalistic Legacy Challenges* (JLCs), that blend opportunities for students to work together identifying, documenting, and creatively solving ill-defined problems in an effort to make a positive and lasting contribution in and beyond their schools and classrooms.

#### **4. Introducing Journalistic Legacy Challenges**

Journalistic legacy challenges (JLCs) represent one-way schools can get smarter about supporting students' development of their creative confidence, work collaboratively with others, communicate about their creative efforts, and ultimately contribute to others. Figure 1 provides an overview of the various components of JLCs. The two major curricular components of JLCs are the *legacy challenge* framework (Beghetto 2018b) and *journalistic learning* (Madison 2012, 2015). These two major curricular components are driven by creative collaboration, creative communication, and creative confidence. Each of these elements will be discussed in the sections that follow.

**Figure 1.** Journalistic Legacy Challenges.

#### **5. Legacy Challenges**

*Legacy challenges* refer to student interest driven projects that provide students with opportunities to put their learning to creative work by allowing them to identify and address complex problems that can make a positive impact in their schools and the world around them (Beghetto 2018b). More specifically, legacy challenges represent a particular type of creative learning experience that provides structured and supportive opportunities for students to:


of making a beneficial and long-term impact on others. Unlike typical school-based projects that focus on individual student learning in the short term, legacy challenges require students to actively plan for and monitor the sustainability and positive impact of their work (Beghetto 2018b).

The four features listed above serve as the defining elements of legacy challenges and the last feature refers to the *legacy* component of these kinds of projects because students are expected to take a long-view approach to the work by developing sustainable solutions to problems. A group of graduating high school seniors who develop a food bank for families in their community, for instance, would need to establish a legacy plan to ensure that community partners and younger students (e.g., juniors and sophomores) are involved in the project so that the work carries on after they leave high school.

One way to think about legacy challenges is that they represent a specific category of creative learning experiences, which can include various existing approaches, such as: design challenges (Brophy et al. 2008; Brown 2009); service learning (Stanton et al. 1999), enrichment activities (Renzulli et al. 2004), playful learning environments (Kangas 2010), and other real-world projects (Boss 2015, 2017). These related approaches can be classified as legacy challenges if they meet the four features of: student identified problems, creative communication about the problems, creative collaboration with skilled others, and making a positive and lasting contribution.

Legacy challenges can range from the work of individual students who identify a social-contextual problem and work with outside partners to solve it to an entire classroom of students addressing a more academic question and contributing their own unique insights. Consider, for instance, Natalie Hampton a teenager in California who, based on her own experience, recognized a problem of social isolation and bullying that often manifests in kids having to sit alone at lunch. In response, Hampton developed an app called "Sit With Us" that provides teenagers who experience bullying and isolation to discreetly find other kids to sit with in the lunchroom (Drake 2016). The legacy of this work is evident in the number of people (100,000+ across seven countries) who have downloaded the app since its launch (TYA 2021). Another example of an impactful legacy project involved a group of students at Blackawton Primary School in Devon, England who collaborated with their teacher and a visiting neuroscientist to publish an article based on their questions about and observations of bee behavior in *Biology Letters*, one of the Royal Society's top academic journals (Yong 2010). Collaborating with supportive and skilled adults, the students were able to make a positive and lasting scientific contribution.

#### **6. Journalistic Learning**

A unique aspect of what we are proposing in this concept paper is that the creative learning of legacy challenges can be further enhanced by infusing aspects of *journalistic learning* (Madison 2012, 2015) into legacy projects. Journalistic learning involves granting students permission to explore topics aligned with their intrinsic interests, thereby fostering authentic student-driven experiences. In turn, students learn how to apply principles and strategies from journalism practice to work with multiple sources of information and communicate their ideas and insights to broader audiences. Part of this learning includes students developing a more active and critical understanding about information and that all sources of information are not equal. More specifically, students learn that claims must be evidence-based and verified. They also learn that one of journalism's highest aims is to serve the public good. This civic imperative extends beyond simply reporting the day's news to informing the public to affect positive, societal change.

In the context of journalistic legacy challenges (JLCs), students apply journalistic principles to the ways in which they communicate about the importance of the problems they are attempting to address and in reporting on the work and impact of their projects. In this way, we assert that JLCs can enable students to more effectively and responsibly collaborate and communicate with others using various forms of media. Such efforts align with recommendations of professional educational associations (e.g., National Council

of Teachers of English, NCTE 2022), which have called for greater curricular emphasis on media education and the importance of learning how to use information to inform public discourse and societal sense making. This is particularly important given that in the internet age the level of "noise," misinformation, and even malicious efforts by some to seed discontent requires that young people learn how to be more critical consumers and ethical producers of information (Madison and DeJarnette 2018).

Moreover, with respect to academic accomplishments, there is also correlational evidence that students, particularly students of color, who have had an opportunity to develop and apply principles of journalistic learning in school, tend to also have more successful academic and writing experiences (Dvorak 1990; Dvorak and Choi 2009). Recent research, however, has indicated that journalism classes tend to attract students who are already more academically accomplished (Bobkowski et al. 2017). It is therefore worth exploring in future research whether systematically incorporating aspects of journalistic learning in legacy projects will benefit the academic confidence and competence of more students (beyond those who typically self-select into journalism courses).

Indeed, only a small fraction of students typically have opportunities to learn and apply principles of journalism in school (Madison 2012, 2015). JLCs, however, can provide many more students with opportunities to learn and apply key principles of journalistic learning in and across various subject areas. The problems that students address in JLCs can and often are informed by multiple academic topics, including math, the sciences, history, literature, and the arts. We recognize that educators, particularly those who have little to no experience in journalism, may need to develop their own journalistic skills or partner with colleagues who have the requisite knowledge or even professional journalists to support students' journalistic learning. Fortunately, educators who are interested in developing their own and their students' journalistic knowledge can turn to promising self-guided professional development resources.

The *journalistic learning initiative* (JLI), for instance, offers a series of self-guided online professional development modules that can help educators and students develop their capacity to learn and apply journalistic principles (see https://journalisticlearning.com, accessed on 27 September 2022). Since 2015, JLI has been adopted by more than fifty middle and high schools in Oregon, California, Arizona, Idaho, and Washington state. Evaluative data collected from students involved in the program indicates a variety of positive outcomes including, positive perceptions and attitudes, high levels of individual interest, engagement, and persistence in journalistic research and writing, flow experiences in the writing process, high levels of autonomy, competency for critical thinking, and relational support from their teachers and peers as a result of their participation in journalistic learning (Madison et al. 2019).

Journalistic learning, when combined with legacy challenges, blends the unique features of the journalistic approach with students' engagement in creative problem solving to make a difference in the world around them. Telling the story of one's creative work is often viewed as a completely subjective and retrospective activity (i.e., people tell their version of the story after they have had the experience). When it comes to JLCs, however, students can learn how to apply professional journalistic standards and skills necessary to document and start telling the story of their creative process from the outset of their work.

#### **7. Creative Communication in JLCs**

JLCs can help students more systematically develop their *creative communication* skills by learning how to effectively communicate about and receive feedback on their own unique and meaningful perspectives and ideas. Learning how to be a more effective communicator in the context of JLCs can also help students learn how to document and articulate the creative impact of their work (Plucker 2022). In this way JLCs serve as a vehicle for students to develop their mini-c or subjective creative ideas into creative contributions that benefit others (Kaufman and Beghetto 2009).

As mentioned, JLCs provide opportunities for creative communication from start to finish. Indeed, prior to students embarking on a JLC, students can learn how to communicate about and learn from their own and others concerns about failure and setbacks. Given that JLCs require young people to creatively engage with uncertainty, it can be expected that they will encounter setbacks and failures along the way (von Thienen et al. 2017). One way to anticipate and prepare young people for setbacks and how they might productively respond to failures (Kapur 2016)—even emotionally painful ones—is to share stories of "favorite failures" (Beghetto and McBain 2022). Sharing stories of favorite failures represents a form of creative communication because doing so not only describes examples of failure, but also describes what that failure felt like, and what people who experienced that failure learned from the experience and about themselves. Sharing favorite failures at the outset of JLCs can thereby help young people anticipate potential setbacks, establish strategies for supporting each other when encountering setbacks, and recognize the importance of documenting and creatively communicating their own experiences to others.

Creative communication in JLCs continues as students work on the identification of the ill-defined problem they want to address. Indeed, students' need to be able to communicate about the problem they are working on so that other people understand what the problem is and why it is important to address. Students also need to be able to creatively communicate about their projects with outside audiences to develop partnerships, receive helpful feedback, and obtain material support and resources to help them address the problem they identified. Creative communication also plays an important role after students have developed and implemented their creative solutions. When students learn how to document and share the stories of their projects, they can more effectively articulate whether and how their efforts have made a positive impact on others and what they learned from the process.

Taken together JLCs can allow students to creatively communicate their efforts to multiple audiences across time and contexts (Plucker 2022). The creative communication that occurs in JLCs serves as a constant feedback loop—the story students develop and tell others helps external audiences and partners understand the project. As students continually communicate about their stories with others, they put themselves in a position to receive timely feedback that can help them to develop and improve their ideas and efforts, which can support them in the early stages of their work by requiring them to clarify and refine the problems they have identified and plan on addressing.

Constant creative communication can also help students identify new and unique facets of the project that might result in changes in how they think about and act on the problems they have identified. As they continue through the project, constant communication with others can help students reflect on the value and merit of their work and ultimately consider whether and how their creative actions are making a positive and sustainable difference to others, including what aspects of their work they might need to adjust or modify. When students learn how to document and creatively communicate with others about the twists, turns, zigs, zags, pivots and persistence in their projects (Sawyer 2013), the story of their work becomes a creative contribution in itself, because it can contribute new and meaningful insights and understandings to others and even inspire others to engage in their own JLCs.

#### **8. Creative Collaboration in JLCs**

JLCs also represent *creative collaborations* (Baruah and Paulus 2019; Lubart 2018; Moran and John-Steiner 2004) from start to finish. Given that JLCs represent opportunities to productively engage with ill-defined problems, students will need to partner with people in and beyond the classroom to receive support, feedback, and guidance on their work. When identifying a problem to solve, creative collaboration can be helpful to students in generating possibilities of issues and problems that they might address. Similarly, creative collaborations can help students gain useful and different perspectives on problems they

have identified, helping them to clarify why the problem matters, who it is impacting, and why it needs to be addressed.

Creative collaboration is perhaps most clearly needed when students start to consider how they might address the problem they identified. Although young people may be able to identify a complex problem on their own and even develop a compelling rationale, when they move towards solving it they likely will need to partner with people who have more experience, expertise, and resources to actually address the problem. Indeed, having sufficient domain knowledge is critically important when it comes to successful creative problem solving and producing creative contributions (Baer 2020). In this way, creative collaboration helps broaden the possibilities of what kinds of problems young people can address, because even though they may not yet have the experience, domain knowledge and resources necessary to solve an ill-defined problem themselves, they can collaborate with outside experts who can assist them.

#### **9. Creative Confidence in JLCs**

Finally, JLCs can help young people develop their *creative confidence*, which compliments their development of creative competence. Prior theory and research (see Karwowski and Kaufman 2017) has demonstrated that transforming creative potential into creative action requires creative confidence (and related self-beliefs). Indeed, just because someone can take creative action on a problem, does not mean that they will. Creative action is risky and is often marked with setbacks and, even, emotionally painful failures. The findings of a set of recent studies, for instance, suggests that unless people see the value in doing creative work, have confidence in their creativity, and are willing to take the creative risks necessary to persist in the face of setbacks, then it is unlikely that they will convert their creative potential into creative action (Beghetto et al. 2021; Karwowski and Beghetto 2019).

As discussed, students typically have limited opportunities in school to develop their creative confidence by identifying and creatively solving ill-defined problems. In order for students to develop their creative confidence they therefore need to have opportunities to both observe others (i.e., relatable models, Bandura 1997) and, most importantly, participate themselves in experiences that require creative thought and action. JLCs provide students with structured and supportive opportunities to identify and work through the kinds of problems and issues that require creative solutions and sustained creative action. Moreover, because JLCs provide opportunities for students to creatively engage with uncertainty in an otherwise supportive and structured environment, students have the opportunity to take sensible, creative risks.

When creative risk taking and setbacks are normalized and expected, students likely will be more willing to trust themselves and others (Grant and Coyle 2018) to take on the uncertainties inherent in JLCs. The small successes they have in addressing and working through the challenges can then accrue in the form of creative confidence (see Amabile and Kramer 2011), which in turn can support them in developing their competence in identifying and addressing complex and ill-defined problems they face now and into the future.

#### **10. Conclusions**

In this concept paper we have asserted that students do not need to wait until they develop domain expertise before they can do meaningful and impactful work. Rather, when schools provide students with creative curricular experiences, such as Journalistic Legacy Challenges, they offer young people opportunities to work with more skilled others and outside experts to put their academic learning to creative use by identifying and addressing complex, real world problems.

As we have discussed, the kinds of experiences offered by JLCs require students to creatively collaborate and communicate with others as well as develop their creative confidence and competence in making a difference in the world around them. We encourage educators and researchers to work together to explore and systematically test-out whether and how the infusion of JLCs can provide students with opportunities to identify complex problems, take creative action, and make an impact on others.

As discussed in the outset of this concept paper, incorporating JLCs in the curriculum does not require completely abandoning existing curricular experiences, rather JLCs can be infused in the everyday curricula by replacing existing assignments and activities with JLC projects. Even spending five minutes a day, over the course of the school year, may provide meaningful opportunities for *all* students to develop their creative confidence, communication, and collaboration skills. Moving forward, researchers and teachers can work together to assess what kinds of opportunities already exist and possible areas that can be opened in the curriculum for JLCs and evaluate progress. Formative evaluation approaches such as those described in Renzulli et al. (2021) can be helpful in supporting these efforts.

In conclusion, we stress that it is not students who need to get smarter before they address complex challenges, rather, it is schools that need to get smarter in the kinds of curricular opportunities they provide young people. If this is the case, then we would argue that schools have a responsibility to provide and support opportunities, like JLCs, for students so that they can learn how to identify and creatively address complex issues and problems they face now and into the future.

**Author Contributions:** Both authors contributed to the conceptualization and writing of this concept article. All authors have read and agreed to the published version of the manuscript.

**Funding:** This concept paper received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


Beghetto, Ronald A. 2018a. Taking beautiful risks in education. *Educational Leadership* 76: 18–24.


Beghetto, Ronald A., and Laura McBain. 2022. *My Favorite Failure*. Lanham: Rowman & Littlefield.


blog/partnership-strategies-real-world-projects-suzie-boss (accessed on 27 September 2022).


*Journal* 33: 75–80. [CrossRef]


Renzulli, Joseph S., Marcia Gentry, and Sally M. Reis. 2004. A time and a place for authentic learning. *Educational Leadership* 62: 73–7. Robertson, S. Ian. 2017. *Problem Solving: Perspectives from Cognition and Neuroscience*, 2nd ed. Abingdon: Routledge.


## *Review* **Creativity, Critical Thinking, Communication, and Collaboration: Assessment, Certification, and Promotion of 21st Century Skills for the Future of Work and Education**

**Branden Thornhill-Miller 1,2,\*, Anaëlle Camarda 3,4, Maxence Mercier 3, Jean-Marie Burkhardt 5, Tiffany Morisseau 3,6, Samira Bourgeois-Bougrine 3, Florent Vinchon 3, Stephanie El Hayek 7, Myriam Augereau-Landais 7, Florence Mourey 3, Cyrille Feybesse 8, Daniel Sundquist <sup>3</sup> and Todd Lubart <sup>3</sup>**


**Abstract:** This article addresses educational challenges posed by the future of work, examining "21st century skills", their conception, assessment, and valorization. It focuses in particular on key soft skill competencies known as the "4Cs": creativity, critical thinking, collaboration, and communication. In a section on each C, we provide an overview of assessment at the level of individual performance, before focusing on the less common assessment of *systemic support* for the development of the 4Cs that can be measured at the institutional level (i.e., in schools, universities, professional training programs, etc.). We then present the process of official assessment and certification known as "labelization", suggesting it as a solution both for establishing a publicly trusted assessment of the 4Cs and for promoting their cultural valorization. Next, two variations of the "International Institute for Competency Development's 21st Century Skills Framework" are presented. The first of these comprehensive systems allows for the assessment and labelization of the extent to which development of the 4Cs is supported by a formal educational program or institution. The second assesses informal educational or training experiences, such as playing a game. We discuss the overlap between the 4Cs and the challenges of teaching and institutionalizing them, both of which may be assisted by adopting a dynamic interactionist model of the 4Cs—playfully entitled "Crea-Critical-Collab-ication"—for pedagogical and policy-promotion purposes. We conclude by briefly discussing opportunities presented by future research and new technologies such as artificial intelligence and virtual reality.

**Keywords:** 21st century skills; 4Cs; assessment; certification; collaboration; communication; creativity; critical thinking; education; future of work; games; labelization; soft skills; training

#### **1. Introduction**

There are many ways of describing the massive educational challenges faced in the 21st century. With the appearance of computers and digital technologies, new means of interacting between people, and a growing competitiveness on the international level, organizations are now requiring new skills from their employees, leaving educational systems struggling to provide appropriate ongoing training. Indeed, according to the World Economic Forum's 2020 "Future of Jobs Report", studying 15 industries in 26 advanced and emerging countries, up to 50% of employees will need some degree of "reskilling" by 2025

**Citation:** Thornhill-Miller, Branden, Anaëlle Camarda, Maxence Mercier, Jean-Marie Burkhardt, Tiffany Morisseau, Samira Bourgeois-Bougrine, Florent Vinchon, Stephanie El Hayek, Myriam Augereau-Landais, Florence Mourey, and et al. 2023. Creativity, Critical Thinking, Communication, and Collaboration: Assessment, Certification, and Promotion of 21st Century Skills for the Future of Work and Education. *Journal of Intelligence* 11: 54. https://doi.org/10.3390/ jintelligence11030054

Received: 17 December 2022 Revised: 16 February 2023 Accepted: 23 February 2023 Published: 15 March 2023

**Copyright:** © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

(World Economic Forum 2020). Although many national and international educational efforts and institutions now explicitly put the cultivation of new kinds of skills on their educational agendas, practical means of assessing such skills remains underdeveloped, thus hampering the valorization of these skills and the development of guidance for relevant pedagogy (Care et al. 2018; Vincent-Lancrin et al. 2019; for overviews and discussion of higher education in global developmental context, see Blessinger and Anchan 2015; Salmi 2017).

This article addresses some of these challenges and related issues for the future of education and work, by focusing on so-called "21st Century Skills" and key "soft skills" known as the "4Cs" (creativity, critical thinking, communication, and collaboration), more particularly. It begins with a brief discussion of these skills, outlining their conceptual locations and potential roles in the modern educational context. A section on each "C" then follows, defining the C, summarizing research and methods for its scientific assessment at the individual level, and then outlining some means and avenues at the systemic level for fostering its development (e.g., important aspects of curriculum, institutional structure, or of the general environment, as well as pedagogical methods) that might be leveraged by an institution or program in order to promote the development of that C among its students/trainees. In the next section, the certification-like process of "labelization" is outlined and proposed as one of the best available solutions both for valorizing the 4Cs and moving them towards the center of the modern educational enterprise, as well as for benchmarking and monitoring institutions' progress in fostering their development. The International Institute for Competency Development's 4Cs Framework is then outlined as an example of such a comprehensive system for assessing and labelizing the extent to which educational institutions and programs support the development of the 4Cs. We further demonstrate the possibility of labelizing and promoting support for the development of the 4Cs by activities or within less formal educational settings, presenting a second framework for assessment of the 4Cs in games and similar training activities. Our discussion section begins with the challenges to implementing educational change in the direction of 21st century skills, focusing on the complex and overlapping nature of the 4Cs. Here, we propose that promoting a "Dynamic Interactionist Model of the 4Cs" not only justifies grouping them together, but it might also assist more directly with some of the challenges of pedagogy, assessment, policy promotion, and ultimately, institutionalization, faced by the 4Cs and related efforts to modernize education. We conclude by suggesting some important future work for the 4Cs individually and also as an interrelated collective of vital skills for the future of education and work.

#### *"21st Century Skills", "Soft Skills", and the "4Cs"*

For 40 years, so-called "21st century skills" have been promoted as those necessary for success in a modern work environment that the US Army War College (Barber 1992) has accurately described as increasingly "VUCA"—"volatile, uncertain, complex and ambiguous". Various lists of skills and competencies have been formulated on their own or as part of comprehensive overarching educational frameworks. Although a detailed overview of this background material is outside the scope of this article (see Lamri et al. 2022; Lucas 2022 for summaries), one of the first prominent examples of this trend was the Partnership for 21st Century Skills (P21), whose comprehensive "Framework for 21st Century Learning" is presented in Figure 1 (Battelle for Kids 2022). This framework for future-oriented education originated the idea of the "4Cs", placing them at its center and apex as "Learning and Innovation Skills" that are in need of much broader institutional support at the foundational level in the form of new standards and assessments, curriculum and instructional development, ongoing professional development, and appropriately improved learning environments (Partnership for 21st Century Skills 2008). These points are also consistent with the approach and assessment frameworks presented later in this article.

**Figure 1.** The P21 Framework for 21st Century Learning. (© 2019, Battelle for Kids. All Rights Reserved. https://www.battelleforkids.org/; accessed on 17 January 2023).

Other important organizations such as the World Economic Forum (2015) have produced similar overarching models of "21st century skills" with the 4Cs at their center, but the term "21st century skills" has been rightly criticized for a several reasons: the skills referred to are not actually all unique to, or uniquely important to, the 21st century, and it is a term that is often used more as an advertising or promotional label for systems that sometimes conflate and confuse different kinds of skills with other concepts that users lump together (Lucas 2019). Indeed, though there is no absolute consensus on the definition of a "skill", they are often described as being multidimensional and involve the ability to solve problems in context and to perform tasks using appropriate resources at the right time and in the right combination (Lamri and Lubart 2021). At its simplest, a skill is a "learned capacity to do something useful" (Lucas and Claxton 2009), or an ability to perform a given task at a specified performance level, which develops through practice, experience. and training (Lamri et al. 2022).

The idea of what skills "are", however, has also evolved to some extent over time in parallel to the nature of the abilities required to make valued contributions to society. The digital and information age, in particular, has seen the replacement by machines of much traditional work sometimes referred to as "hard skills"—skills such as numerical calculation or driving, budget-formulating, or copyediting abilities, which entail mastery of fixed sets of knowledge and know-how of standard procedures, and which are often learned on the job. Such skills are more routine, machine-related, or technically oriented and not as likely to be centered on human interaction. In contrast, the work that has been increasingly valued in the 21st century involves the more complex, human interactive, and/or non-routine skills that Whitmore (1972) first referred to as "soft skills".

Unfortunately, researchers, educators, and consultants have defined, redefined, regrouped, and expanded soft skills—sometimes labeling them "transversal competencies", "generic competencies", or even "life skills" in addition to "21st century skills"—in so many different ways within and across different domains of research and education (as well as languages and national educational systems) that much progress towards these goals has literally been "lost in translation" (Cinque 2016).

Indeed, there is also a long-standing ambiguity and confusion between the terms "competency" (also competence) and "skill" due to their use across different domains (e.g., learning research, education, vocational training, personnel selection) as well as different epistemological backgrounds and cultural specificities (Drisko 2014; Winterton et al. 2006; van Klink and Boon 2003). The term "competency" is, however, often used as a broader

concept that encompasses skills, abilities, and attitudes, whereas, in a narrower sense, the term "skill" has been defined as "goal-directed, well-organized behavior that is acquired through practice and performed with economy of effort" (Proctor and Dutta 1995, p. 18). For example, whereas the command of a spoken language or the ability to write are skills (hard skills, to be precise), the ability to communicate effectively is a competence that may draw on an individual's knowledge of language, writing skills, practical IT skills, and emotional intelligence, as well as attitudes towards those with whom one is communicating (Rychen and Hersch 2003). Providing high-quality customer service is a competency that relies on listening skills, social perception skills, and contextual knowledge of products. Beyond these potential distinctions, the term "competency" is predominant in Europe, whereas "skill" is more commonly used in the US. Yet it also frequently occurs that both are used as rough synonyms. For example, Voogt and Roblin (2012, p. 299) examine the "21st century competences and the recommended strategies for the implementation of these skills", and Graesser et al. (2022, p. 568) state that twenty-first-century skills "include self-regulated learning, collaborative problem solving, communication (... ) and other competencies". In conclusion, the term "competencies" is often used interchangeably with "skills" (and can have a particularly large overlap with "soft skills"), but it is also often considered in a broader sense as a set of skills, knowledge, and attitudes that, together, meet a complex demand (Ananiadoui and Claro 2009). From this perspective, one could argue that the 4Cs, as complex, "higher-order" soft skills, might best be labeled competencies. For ease and convenience, however, in this text, we consider the two terms interchangeable but favor the term "skills", only using "competency" in some instances to avoid cumbersome repetition.

Even having defined soft skills as a potentially more narrow and manageable focus, we are still aware of no large-scale study that has employed a comprehensive enough range of actual psychometric measures of soft skills in a manner that might help produce a definitive empirical taxonomy. Some more recent taxonomic efforts have, however, attempted to provide additional empirical grounding for the accurate identification of key soft skills (see e.g., Joie-La Marle et al. 2022). Further, recent research by JobTeaser (see Lamri et al. 2022) surveying a large, diverse sample of young workers about a comprehensive, systematic list of soft skills *as actually used in their professional roles* represents a good step towards some clarification and mapping of this domain on an empirical basis. Despite the fact that both these studies necessarily involved assumptions and interpretive grouping of variables, the presence and importance of the 4Cs as higher-order skills is evident in both sets of empirical results.

Various comprehensive "21st century skills" systems proposed in the past without much empirical verification also seem to have been found too complex and cumbersome for implementation. The 4Cs, on the other hand, seem to provide a relatively simple, persuasive, targetable core that has been found to constitute a pedagogically and policy-friendly model by major organizations, and that also now seems to be gaining some additional empirical validity. Gathering support from researchers and industry alike, we suggest that the 4Cs can be seen as highest-level transversal skills—or "meta-competencies"—that allow individuals to remain competent and to develop their potential in a rapidly changing professional world. Thus, in the end, they may also be one of the most useful ways of summarizing and addressing the critical challenges faced by the future of work and education (National Education Association 2011).

Taking them as our focus, we note, however, that the teaching and development of the 4Cs will require a complex intervention and mobilization of educational and socio-economic resources—both a major shift in pedagogical techniques and even more fundamental changes in institutional structures (Ananiadoui and Claro 2009). One very important issue for understanding the 4Cs and their educational implementation related to this, which can simultaneously facilitate their teaching but be a challenge for their assessment, is the multidimensionality, interrelatedness, and transdisciplinary relevance of the 4Cs. Thus, we address the relationships between the Cs in the different C sections and later in our Discussion, we present a "Dynamic Interactionist Model of the 4Cs" that we hope will

assist in their understanding, in the further development of pedagogical processes related to them, and in their public promotion and related policy. Ultimately, it is partly due to their complexity and interrelationships, we argue, that it is important and expedient that the 4Cs are taught, assessed, and promoted together.

#### **2. The 4Cs, Assessment, and Support for Development**

#### *2.1. Creativity*

In psychology, creativity is usually defined as the capacity to produce novel, original work that fits with task constraints and has value in its context (for a recent overview, see Lubart and Thornhill-Miller 2019). This basic definition, though useful for testing and measurement, is largely incomplete, as it does not contain any information about the individual or groups doing the creating or the nature of physical and social contexts (Glăveanu 2014). Moreover, Corazza (2016) challenged this standard definition of creativity, arguing that as it focuses solely on the existence of an original and effective outcome, it misses the dynamics of the creative process, which is frequently associated with periods of creative inconclusiveness and limited occasions of creative achievements. To move away from the limitations of the standard definition of creativity, we can consider Bruner's description of creativity as "figuring out how to use what you already know in order to go beyond what you currently think" (p. 183 in Weick 1993). This description echoes the notion of potential, which refers to a latent state that may be put to use if a person has the opportunity.

Creativity is a multifaceted phenomenon that can be approached from many different angles. There are three main frameworks for creativity studies: the 4Ps (Rhodes 1961), the 5As (Glăveanu 2013), and the 7Cs model (Lubart 2017). These frameworks share at least four fundamental and measurable dimensions: the act of creating (process), the outcome of the creative process (product), the characteristics of creative actor(s) enacting the process (person), and the social and physical environment that enable or hinder the creative process (press). Contrary to many traditional beliefs, however, creativity can be trained and taught in a variety of different ways, both through direct, active teaching of creativity concepts and techniques and through more passive and indirect means such as the development of creativity-supporting contexts (Chiu 2015; Thornhill-Miller and Dupont 2016). Alongside intelligence, with which it shares some common mechanisms, creativity is now recognized as an indispensable element for the flexibility and adaptation of individuals in challenging situations (Sternberg 1986).

#### 2.1.1. Individual Assessment of Creativity

Drawing upon previous efforts to structure creativity research, Batey (2012) proposed a taxonomic framework for creativity measurement that takes the form of a three-dimensional matrix: (a) the level at which creativity may be measured (the individual, the team, the organization, and the culture), (b) the facets of creativity that may be assessed (person/trait, process, press, and product), and (c) the measurement approach (objective, self-rating, other ratings). It is beyond the scope of this article to offer a literature review of all these dimensions, but for the purposes of this paper, we address some important aspects of individual-level and institutional-level assessment here.

Assessing creativity at an individual level encompasses two major approaches: (1) creative accomplishment based on production and (2) creative potential. Regarding the first approach focusing on *creative accomplishment*, there are at least four main assessment techniques (or tools representing variations of assessment techniques): (a) the historiometric approach, which applies quantitative analysis to historically available data (such as the number of prizes won or times cited) in an effort to understand eminent, field-changing creativity (Simonton 1999); (b) the Consensual Assessment Technique (CAT) (Amabile 1982), which offers a method for combining and validating judges' subjective evaluations of a set of (potentially) creative productions or ideas; (c) the Creative Achievement Questionnaire (Carson et al. 2005), which asks individuals to supply a self-reported assessment of their

publicly recognizable achievement in ten different creative domains; and (d) the Inventory of Creative Activities and Achievements (ICAA) (Jauk et al. 2014; Diedrich et al. 2018), which includes self-report scales assessing the frequency of engagement in creative activity and also levels of achievement in eight different domains.

The second major approach to individual assessment is based on *creative potential,* which measures the cognitive abilities and/or personality traits that are important for creative work. The two most popular assessments of creative potential are the Remote Associations Test (RAT) and the Alternative Uses Task (AUT). The RAT, which involves identifying the fourth word that is somehow associated with each of three given words, underscores the role that the ability to convergently associate disparate ideas plays as a key capacity for creativity. In contrast, the AUT, which requires individuals to generate a maximum number of ideas based on a prompt (e.g., different uses for a paperclip), is used to assess divergent thinking capacity. According to multivariate models of creative potential (Lubart et al. 2013), there are cognitive factors (e.g., divergent thinking, mental flexibility, convergent thinking, associative thinking, selective combination), conative factors (openness, tolerance of ambiguity, intuitive thinking, risk taking, motivation to create), and environmental factors that all support creativity. Higher creative potential is predicted by having more of the ingredients for creativity. However, multiple different profiles among a similar set of these important ingredients exist, and their weighting for optimal creative potential varies according to the profession, the domain, and the task under consideration. For example, Lubart and Thornhill-Miller (2021) and Lubin et al. (Forthcoming) have taken this creativity profiling approach, exploring the identification and training of the components of creative potential among lawyers and clinical psychologists, respectively. For a current example of this sort of comprehensive, differentiated measurement of creative potential in adults in different domains and professions, see CreativityProfiling.org. For a recent battery of tests that are relevant for children, including domain-relevant divergent-exploratory and convergent-integrative tasks, see Lubart et al. (2019). Underscoring the growing recognition of the importance of creativity assessment, measures of creative potential for students were introduced internationally for the first time in the PISA 2022 assessment (OECD 2019a).

#### 2.1.2. Institutional and Environmental Support for Development of Creativity

The structural support that institutions and programs can provide to promote the development of creativity can be described as coming through three main paths: (1) through design of the physical environment in a manner that supports creativity, (2) through teaching about creativity, the creative process, and creativity techniques, and (3) through training opportunities to help students/employees develop personal habits, characteristics, and other ingredients associated with creative achievement and potential.

Given the multi-dimensionality of the notion of creativity, the environment can positively influence and help develop creative capacities. Studies have shown that the physical environment in which individuals work can enhance their positive emotions and mood and thus their creativity. For example, stimulating working environments might have unusual furniture and spaces that have natural light, windows open to nature, plants and flowers, a relaxing atmosphere and colors in the room (e.g., green and blue), or positive sounds (e.g., calm music or silence), as well as inspiring and energizing colors (e.g., yellow, pink, orange). Furthermore, the arrangement of physical space to promote interpersonal exchange rather than isolation, as well as the presence of tools, such as whiteboards, that support and show the value of exchange, are also important (for reviews, see Dul and Ceylan 2011; Samani et al. 2014).

Although it has been claimed that "creativity is intelligence having fun" (Scialabba 1984; Reiman 1992), for most people, opportunities for fun and creativity, especially in their work environment, appear rather limited. In fact, the social and physical environment often hinders creativity. Corazza et al. (2021)'s theoretical framework concerning the "Space-Time Continuum", related to support for creativity, suggests that traditional education

systems are an example of an environment that is "tight" both in the conceptual "space" it affords for creativity and in the available time allowed for creativity to happen—essentially leaving little room for original ideas to emerge. Indeed, though world-wide data suggest that neither money nor mere time spent in class correlate well with educational outcomes, both policies and pedagogy that direct the ways in which time is spent make a significant difference (Schleicher 2022). Research and common sense suggest that teachers, students, and employees need more space and time to invest energy in the creative process and the development of creative potential.

Underscoring the importance of teaching the creative process and creativity techniques is the demonstration, in a number of contexts, that groups of individuals who generate ideas without a specific method are often negatively influenced by their social environment. For example, unless guarded against, the presence of others tends to reduce the number of ideas generated and to induce a fixation on a limited number of ideas conforming to those produced by others (Camarda et al. 2021; Goldenberg and Wiley 2011; Kohn and Smith 2011; Paulus and Dzindolet 1993; Putman and Paulus 2009; Rietzschel et al. 2006). To overcome these cognitive and social biases, different variants of brainstorming techniques have shown positive effects (for reviews of methods, see Al-Samarraie and Hurmuzan 2018; Paulus and Brown 2007). These include: using (Osborn 1953) initial brainstorming rules (which aim to reduce spontaneous self-judgment of ideas and fear of this judgment by others); drawing attention to ideas generated by others by writing them down independently (e.g., the technique known as "brainwriting"); and requiring incubation periods between work sessions by forcing members of a problem-solving group to take breaks (Paulus and Yang 2000; Paulus and Kenworthy 2019).

It is also possible to use design methods that are structured to guide the creative process and the exploration of ideas, as well as to avoid settling on uncreative solution paths (Chulvi et al. 2012; Edelman et al. 2022; Kowaltowski et al. 2010; see Cotter et al. 2022 for a valuable survey of best practices for avoiding the suppression of creativity and fostering creative interaction and metacognition in the classroom). Indeed, many helpful design thinking-related programs now exist around the world and have been shown to have a substantial impact on creative outcomes (Bourgeois-Bougrine 2022).

Research and experts suggest the utility of many additional creativity enhancement techniques (see, e.g., Thornhill-Miller and Dupont 2016), and the largest and most rapid effects are often attributed to these more method- or technique-oriented approaches (Scott et al. 2004). More long-term institutional and environmental support for the development of creativity, however, should also include targeted training and understanding of personality and emotional traits associated with the "creative person" (e.g., empathy and exploratory habits that can expand knowledge, as well as increase tolerance of ambiguity, openness, and mental flexibility; see Lubart and Thornhill-Miller 2021). Complementing these approaches and focusing on a more systemic level, recent work conducted by the OECD exemplifies efforts aimed to foster creativity (and critical thinking) by focusing simultaneously on curriculum, educational activities, and teacher support and development at the primary, secondary, and higher education levels (see Vincent-Lancrin et al. 2019; Saroyan 2022).

#### *2.2. Critical Thinking*

Researchers, teachers, employers, and public policymakers around the world have long ranked the development of critical thinking (CT) abilities as one of the highest educational priorities and public needs in modern democratic societies (Ahern et al. 2019; Dumitru et al. 2018; Pasquinelli et al. 2021). CT is central to better outcomes in daily life and general problem solving (Hitchcock 2020), to intelligence and adaptability (Halpern and Dunn 2021), and to academic achievement (Ren et al. 2020). One needs to be aware of distorted or erroneous information in the media, of the difference between personal opinions and proven facts, and how to handle increasingly large bodies of information required to understand and evaluate information in the modern age.

Although much research has addressed both potentially related constructs, such as intelligence and wisdom, and lists of potential component aspects of human thought, such as inductive or deductive reasoning (for reviews of all of these, see Sternberg and Funke 2019), reaching a consensus on a definition has been difficult, because CT relies on the coordination of many different skills (Bellaera et al. 2021; Dumitru et al. 2018) and is involved in, and sometimes described from the perspective of, many different domains (Lewis and Smith 1993). Furthermore, as a transversal competency, having the skills to perform aspects of critical thinking in a given domain does not necessarily entail also having the metacognitive ability to know when to engage in which of its aspects, or having the disposition, attitude, or "mindset" that motivates one to actually engage in them—all of which are actually required to be a good critical thinker (Facione 2011).

As pointed out by the American Philosophical Association's consensus definition, the ideal "critical thinker" is someone who is inquisitive, open-minded, flexible, fair-minded, and keeps well-informed, thus understanding different points of view and perspectives (Facione 1990b). These characteristics, one might note, are also characteristic of the "creative individual" (Facione 1990b; Lai 2011), as is the ability to imagine alternatives, which is often cited as a component of critical thinking ability (Facione 1990b; Halpern 1998). Conversely, creative production in any domain needs to be balanced by critical appraisal and thought at each step of the creative process (Bailin 1988). Indeed, it can be argued that creativity and critical thinking are inextricably linked and are often two sides of the same coin. Representing different aspects of "good thought" that are linked and develop in parallel, it seems reasonable that they should, in practice, be taught and considered together in teaching and learning (Paul and Elder 2006).

Given its complexity, many definitions of critical thinking have been offered. However, some more recent work has helpfully defined critical thinking as "the capacity of assessing the epistemic quality of available information and—as a consequence of this assessment of calibrating one's confidence in order to act upon such information" (Pasquinelli et al. 2021). This definition, unlike others proposed in the field (for a review, see: Bellaera et al. 2021; Liu et al. 2014), is specific (i.e., it limits the use of poorly defined concepts), as well as consensual and operational (i.e., it has clear and direct implications for the education and assessment of critical thinking skills; Pasquinelli et al. 2021; Pasquinelli and Bronner 2021). Thus, this approach assumes that individuals possess better or worse cognitive processes and strategies that make it possible to judge the reliability of the information received, by determining, for example, what the arguments provided actually are. Are the arguments convincing? Is the source of information identifiable and reliable? Does the information conflict with other information held by the individual?

It should also be noted that being able to apply critical thinking is necessary to detect and overcome the cognitive biases that can constrain one's reasoning. Indeed, when solving a problem, it is widely recognized that people tend to automate the application of strategies that are usually relevant in similar and analogous situations that have already been encountered. However, these heuristics (i.e., automatisms) can be a source of errors, in particular, in tricky reasoning situations, as demonstrated in the field of reasoning, arithmetic problems (Kahneman 2003) or even divergent thinking tasks (Cassotti et al. 2016; for a review of biases, see Friedman 2017). Though some cognitive biases can even be seen as normal ways of thinking and feeling, sometimes shaping human beliefs and ideologies in ways that make it completely normal—and even definitely human—*not* to be objective (see Thornhill-Miller and Millican 2015), the mobilization of cognitive resources such as those involved in critical reasoning on logical bases usually makes it possible to overcome cognitive biases and adjust one's reasoning (West et al. 2008).

According to Pasquinelli et al. (2021), young children already possess cognitive functions underlying critical thinking, such as the ability to determine that information is false. However, until late adolescence, studies have demonstrated an underdevelopment of executive functions involved in resistance to biased reasoning (Casey et al. 2008) as well as some other higher-order skills that underlie the overall critical thinking process (Bloom 1956). According to Facione and the landmark American Philosophical Association's task force on critical thinking (Facione 1990b; Facione 2011), these components of critical thinking can be organized into six measurable skills: the ability to (1) interpret information (i.e., meaning and context); (2) analyze information (i.e., make sense of why this information has been provided, identify pro and con arguments, and decide whether we can accept the conclusion of the information); (3) make inferences (i.e., determine the implications of the evidence, its reliability, the undesirable consequences); (4) evaluate the strength of the information (i.e., its credibility, determine the trust in the person who provides it); (5) provide explanations (i.e., summarize the findings, determine how the information can be interpreted, and offer verification of the reasoning); (6) self-regulate (i.e., evaluate the strength of the methods applied, determine the conflict between different conclusions, clarify the conclusions, and verify missing elements).

#### 2.2.1. Individual Assessment of Critical Thinking

The individual assessment of critical thinking skills presents a number of challenges, because it is a multi-task ability and involves specific knowledge in the different areas in which it is applied (Liu et al. 2014; Willingham 2008). However, the literature provides several tools with which to measure different facets of cognitive functions and skills involved in the overarching critical thinking process (Lai 2011; Liu et al. 2014). Most assessments involve multiple-choice questions requiring reasoning within a particular situation based upon a constrained set of information provided. For example, in one of the most widely used tests, the California Critical Thinking Skills Test (Facione 1990a), participants are provided with everyday scenarios and have to answer multiple questions targeting the six higher-order skills described previously. Similarly, the Watson–Glaser Critical Thinking Appraisal (Watson 1980; Watson and Glaser 2010) presents test takers with passages and scenarios measuring their competencies at recognizing assumptions, evaluating arguments, and drawing conclusions. Although the Watson–Glaser is one of the oldest and most frequently used assessments internationally for hiring and promotion in professional contexts, its construct validity, like many other measures of this challenging topic, has some limitations (Possin 2014).

Less frequently, case study or experiential methods of assessment are also used. This approach may involve asking participants to reflect on past experiences, analyze the situations they faced and the way they behaved or made judgments and decisions and then took action (Bandyopadhyay and Szostek 2019; Brookfield 1997). These methods, often employed by teachers or employers on students and employees, usually involve the analysis of qualitative data that can cast doubt on the reliability of the results. Consequently, various researchers have suggested ways to improve analytic methods, and they emphasize the need to create more advanced evaluation methods (Brookfield 1997; Liu et al. 2014).

For example, Liu et al. (2014) reviewed current assessment methods and suggest that future work improves the operational definition of critical thinking, aiming to assess it both in different specific contexts and in different formats. Specifically, assessments could be contextualized within the major areas addressed by education programs (e.g., social sciences, humanities, and/or natural sciences), and the tasks themselves should be as practically connected to the "real world" as possible (e.g., categorizing a set of features, opinions, or facts based on whether or not they support an initial statement). Moreover, as Brookfield (1997) argues, because critical thinking is a social process that takes place in specific contexts of knowledge and culture, it should be assessed as a social process, therefore, involving a multiplicity of experiences, perceptions, and contributions. Thus, Brookfield makes three recommendations for improving the assessment of critical thinking that are still relevant today: (1) to assess critical thinking in specific situations, so one can study the process and the discourse related to it; (2) to involve students/peers in the evaluation of critical thinking abilities, so that the evaluation is not provided only by the instructor; and (3) to allow learners or participants in an experiment to document, demonstrate, and justify

their engagement in critical thinking, because this learning perspective can provide insight into basic dimensions of the critical thinking process.

Finally, another more recent and less widely used form of assessment targets the specific executive functions that underlie logical reasoning and resistance to cognitive biases, as well as the ability of individuals to resist these biases. This form of assessment is usually done through specific experimental laboratory tasks that vary depending on the particular executive function and according to the domain of interest (Houdé and Borst 2014; Kahneman 2011; West et al. 2008).

#### 2.2.2. Institutional and Environmental Support for Development of Critical Thinking Skills

The executive functions underlying general critical thinking, the ability to overcome bias (Houdé 2000; Houdé and Borst 2014), and meta-cognitive processes (i.e., meta information about our cognitive strategies) can all be trained and enhanced by educational programs (Abrami et al. 2015; Ahern et al. 2019; Alsaleh 2020; Bellaera et al. 2021; Uribe-Enciso et al. 2017; Popil 2011; Pasquinelli and Bronner 2021; Yue et al. 2017).

Educational programs and institutions can support the development of critical thinking in several different ways. The process of developing critical thinking focuses on the interaction between personal dispositions (attitudes and habits), skills (evaluation, reasoning, self-regulation), and finally, knowledge (general and specific knowledge, as well as experience) (Thomas and Lok 2015). It is specifically in regard to skills and knowledge that institutions are well suited to develop critical thinking through pedagogical elements such as rhetoric training, relevance of information evaluation (e.g., media literacy, where and how to check information on the internet, dealing with "fake news", etc.), deductive thinking skills, and inductive reasoning (Moore and Parker 2016). A few tools, such as case studies or concept mapping, can also be used in conjunction with a problem-based learning method, both in individual and team contexts and in person or online (Abrami et al. 2015; Carmichael and Farrell 2012; Popil 2011; Thorndahl and Stentoft 2020). According to Marin and Halpern (2011), training critical thinking should include explicit instruction involving at least the four following components and objectives: (1) working on attitudes and encouraging individuals to think; (2) teaching and practicing critical thinking skills; (3) training for transfer between contexts, identifying concrete situations in which to adopt the strategies learned; and (4) suggesting metacognition through reflection on one's thought processes. Supporting these propositions, Pasquinelli and Bronner (2021), in a French national educational report, proposed practical advice for creating workshops to stimulate critical thinking in school classrooms, which appear relevant even in non-school intervention situations. For example, the authors suggest combining concrete examples and exercises with general and abstract explanations, rules and strategies, which can be transferred to other areas beyond the one studied. They also suggest inviting learners to create examples of situations (e.g., case studies) in order to increase the opportunities to practice and for the learner to actively participate. Finally, they suggest making the process of reflection explicit by asking the learner to pay attention to the strategies adopted by others in order to stimulate the development of metacognition.

#### *2.3. Communication*

In its most basic definition, communication consists of exchanging information to change the epistemic context of others. In cooperative contexts, it aims at the smooth and efficient exchange of information contributing to the achievement of a desired outcome or goal (Schultz 2010). But human communication involves multiple dimensions. Both verbal and non-verbal communication can involve large quantities of information that have to be both formulated and deciphered with a range of purposes and intentions in mind (Jones and LeBaron 2002). These dimensions of communication have as much to do with the ability to express oneself, both orally and in writing and the mastering of a language (linguistic competences), as with the ability to use this communication system appropriately (pragmatic skills; see Grassmann 2014; Matthews 2014), and with social skills, based on the knowledge of how to behave in society and on the ability to connect with others, to understand the intentions and perspectives of others (Tomasello 2005).

Like the other 4Cs, according to most authorities, communication skills are ranked by both students and teachers as skills of the highest priority for acquisition in order to be ready for the workforce in 2030 (OECD 2019b; Hanover Research 2012). Teaching students how to communicate efficiently and effectively in all the new modalities of information exchange is an important challenge faced by all pedagogical organizations today (Morreale et al. 2017). All dimensions of communication (linguistic, pragmatic, and social) are part of what is taught in school curricula at different levels. But pragmatic and social competencies are rarely explicitly taught as such. Work on social/emotional intelligence (and on its role in students' personal and professional success) shows that these skills are both disparate and difficult to assess (Humphrey et al. 2007). Research on this issue is, however, becoming increasingly rigorous, with the potential to provide usable data for the development of science-based practice (Keefer et al. 2018). Teachers and pedagogical teams also have an important, changing role to play: they also need to master new information and communication technologies and the transmission of information through them (Zlati´c et al. 2014).

Communication has an obvious link with the three other Cs. Starting with critical thinking, sound communication implies fostering the conditions for a communicative exchange directed towards a common goal, which is, at least in educational and professional contexts, based on a fair evaluation of reality (Pornpitakpan 2004). Collaboration too has a strong link with communication, because successful collaboration is highly dependent on the quality of knowledge sharing and trust that emerges between group members. Finally, creativity involves the communication of an idea to an audience and can involve high-quality communication when creative work occurs in a team context.

#### 2.3.1. Individual Assessment of Communication

Given the vast field of communication, an exhaustive list of its evaluation methods is difficult to establish. A number of methods have been reported in the literature to assess an individual's ability to communicate non-verbally and verbally. But although these two aspects are intrinsically linked, they are rarely measured together with a single tool. Moreover, as Spitzberg (2003) pointed out, communication skills are supported by different abilities, classically conceptualized as motivational functions (e.g., confidence and goal-orientation), knowledge (e.g., content and procedural knowledge), or cognitive and socio-cognitive functions (e.g., theory of mind, verbal cognition, emotional intelligence, and empathy; McDonald et al. 2014; Rothermich 2020), implying different specific types of evaluations. Finally, producing vs. receiving communication involve different skills and abilities, which can also vary according to the context (Landa 2005).

To overcome these challenges, Spitzberg (2003) recommends the use of different assessment criteria. These criteria include the clarity of interaction, the understanding of what was involved in the interaction, the satisfaction of having interacted (expected to be higher when communication is effective), the efficiency of the interaction (the more competent someone is, the less effort, complexity, and resources will be needed to achieve their goal), its effectiveness or appropriateness (i.e., its relevance according to the context), as well as criteria relative to the quality of the dialogue (which involves coordination, cooperation, coherence, reciprocity, and mutuality in the exchange with others). Different forms of evaluation are also called for, such as self-reported questionnaires, hetero-reported questionnaires filled out by parents, teachers, or other observers, and tasks involving exposure to role-playing games, scenarios or videos (for a review of these assessment tools, see Cömert et al. 2016; Landa 2005; Sigafoos et al. 2008; Spitzberg 2003; van der Vleuten et al. 2019). Results from these tools must then be associated with others assessing underlying abilities, such as theory of mind and metacognition.

#### 2.3.2. Institutional and Environmental Support for Development of Communication Skills

Although communication appears to be a key employability skill, the proficiency acquired during studies rarely meets the expectations of employers (Jackson 2014). Communication must therefore become a priority in the training of students, beyond the sectors in which it is already known as essential (e.g., in medicine, nursing, engineering, etc.; Bourke et al. 2021; D'Alimonte et al. 2019; Peddle et al. 2018; Riemer 2007), and also through professional development (Jackson 2014). Training programs involving, for example, communication theory classes (Kruijver et al. 2000) and self-assessment tools that can be used in specific situations (Curtis et al. 2013; Rider and Keefer 2006) have had convincingly positive results. The literature suggests that interactive approaches in small groups, in which competencies are practiced explicitly in an open and feedback-safe environment, are more effective (Bourke et al. 2021; D'Alimonte et al. 2019; AbuSeileek 2012; Fryer-Edwards et al. 2006). These can take different forms: project-based work, video reviews, simulation or role-play games (see Hathaway et al. 2022 for a review; Schlegel et al. 2012). Finally, computerassisted learning methods can be relevant for establishing a secure framework (especially, for example, when learning another language): anonymity indeed helps to overcome anxiety or social blockages linked to fear of public speaking or showing one's difficulties (AbuSeileek 2012). Each of these methods tackles one or more dimensions of communication that must then be assessed as such, by means of tools specifically developed and adapted to the contexts in which these skills are expressed (e.g., see the two 4Cs evaluation grids for institutions and for games outlined in Sections 4 and 5, below).

#### *2.4. Collaboration*

Collaborative problem solving—and more generally, collaboration—has gained increasing attention in national and international assessments (e.g., PISA) as an educational priority encompassing social, emotional, and cognitive skills critical to efficiency, effectiveness, and innovation in the modern global economy (Graesser et al. 2018; OECD 2017). Understanding what makes effective collaboration is of crucial importance for professional practice and training (Détienne et al. 2012; Graesser et al. 2018), as evidenced by the long line of research on group or team collaboration over the past 40 years (for a review, see e.g., Salas et al. 2004; Mathieu et al. 2017). Although there is no consensus on a definition of collaboration, scholars often see it as mutual engagement in a coordinated effort to achieve a common goal that involves the sharing of goals, resources, and representations relating to the joint activity of participants; and other important aspects relate to mutual respect, trust, responsibilities, and accountability within situational rules and norms (Détienne et al. 2012).

In the teamwork research literature, skills are commonly described across three classes most often labeled Knowledge, Behavior, and Attitudes (e.g., Cannon-Bowers et al. 1995). Knowledge competencies refer to the skills related to elaborating the knowledge content required for the group to process and successfully achieve the task/goal to which they are assigned. Behavior includes skills related to the actualization of actions, coordination, communication, and interactions within the group as well as with any other relevant interlocutors for the task at hand. Note here that effective collaboration involves skills that have also been identified elsewhere as essential competencies, including communication, creativity, and critical thinking. Finally, several attitudes have been evidenced or hypothesized as desirable competencies in the team context, for example, attitude towards teamwork, collective orientation, cohesion/team morale, etc. Another common distinction lies between teamwork and taskwork. Teamwork refers to the collaborative, communicative, or social skills required to coordinate the work within the participants in order to achieve the task, whereas taskwork refers to specific aspects related to solving the task such as using the tools and knowing the procedure, policies, and any other task-related activities (Salas et al. 2015; Graesser et al. 2018). Furthermore, collaborative competences can have specific (to a group of people or to a task) and general dimensions (i.e., easily transferable to any group or team situation and to other tasks). For example, skills related

to communication, information exchange, conflict management, maintaining attention and motivation, leadership, etc. are present and transferable to a large number of group work situations and tasks (team-generic and task-contingent skills). Other skills can, on the other hand, be more specific to a team or group, such as internal organization, motivation, knowledge of the skills distributed in the team, etc.

#### 2.4.1. Individual Assessment of Collaboration

Assessing collaboration requires capturing the dynamic and multi-level nature of the collaboration process, which is not as easily quantifiable as group/team inputs and outputs (task performance, satisfaction, and changes at group/team and individual level). There are indeed multiple interactions between the context, the collaboration processes, the task processes, and their (various) outcomes (Détienne et al. 2012). The integrative concept of "quality of collaboration" (Burkhardt et al. 2009) encapsulates much of what is currently known about collaborative processes and what constitutes effective collaboration. According to this approach, collaborative processes can be grouped along several dimensions concerning communication processes such as grounding, task-related processes (e.g., exchanges of knowledge relevant for the task at hand), and organization/coordination processes (Burkhardt et al. 2009). Communication processes are most important for ensuring the construction of a common referential within a group of collaborators. Task-related processes relate to how the group resolves the task at hand by sharing and co-elaborating knowledge, by confronting their various perspectives, and by converging toward negotiated solutions. Collaboration also involves group management activities such as: (a) common goal management and coordination activities, e.g., allocation and planning of tasks; (b) meeting/interaction management activities, e.g., ordering and postponing of topics in the meeting. Finally, the ability to pursue reflexive activity, in the sense of reflecting not only on the content of a problem or solution but on one's collaboration and problem-solving strategies, is critical for the development of the team and supports them in changing and improving their practices. Graesser et al. (2018) identify collaborative skills based on the combination of these dimensions with a step in the problem-solving process.

A large body of methodology developed to assess collaboration processes and collaborative tools has been focused on quantifying a restricted subset of fine-grained interactions (e.g., number of speakers' turns; number of words spoken; number of interruptions; amount of grounding questions). This approach has at least two limitations. First, because these categories of analysis are often ad hoc with respect to the considered situation, they are difficult to apply in all situations and make it difficult to compare between studies. Second, quantitative variations of most of these indicators are non-univocal: any increase or decrease of them could signify either an interactive–intensive collaboration or else evidence of major difficulties in establishing and/or maintaining the collaboration (Détienne et al. 2012). Alternatively, qualitative approaches based on multidimensional views of collaboration provide a more elaborated or nuanced view of collaboration and are useful for identifying potential relationships between distinctive dimensions of collaboration and aspects of team performance, in order to identify processes that could be improved. Based on the method of Spada et al. (2005) in Computer-Supported Collaborative Learning (CSCL) research, Burkhardt et al. (2009) have proposed a multi-dimensional rating scheme for evaluating the quality of collaboration (QC) in technology-mediated design. QC distinguishes seven dimensions, grouped along five aspects, identified as central for collaboration in a problem-solving task such as design: communication (1, 2), task-oriented processes (3, 4), group-oriented processes (5), symmetry in interaction—an orthogonal dimension—(6), and individual task orientation (7). This method has recently been adapted for use in the context of assessing games as a support to collaborative skills learning.

2.4.2. Institutional and Environmental Support for Development of Collaboration and Collaborative Skills

Support for individuals' development of collaborative skills provided by institutions and programs can take a variety of forms: (a) through the social impact of the physical structure of the organization, (b) the nature of the work required within the curriculum, (c) content within the curriculum focusing on collaboration and collaborative skills, and (d) the existence and promotion of extracurricular and inter-institutional opportunities for collaboration.

For instance, institutional support for collaboration has taken a variety of forms in various fields such as healthcare, engineering, public participation, and education. Training and education programs such as Interprofessional Education or Team Sciences in the health domain (World Health Organization 2010; Hager et al. 2016; O'Carroll et al. 2021), Peer-Led Team Learning in chemistry and engineering domains (Wilson and Varma-Nelson 2016), or Collaborative Problem Solving in education (Peña-López 2017; Taddei 2009) are notable examples.

Contextual support recently arose from the deployment of online digital media and new mixed realities in the workplace, in the learning environments and in society at large—obviously stimulated and accentuated with the COVID-19 pandemic. This has led many organizations to invest in proposing support for synchronous and asynchronous collaboration (notably remote, between employees, between students and educators or within group members, etc.) in various ways, including the provision of communication hardware and software, computer-supported cooperative work and computer-supported collaborative learning platforms, training and practical guides, etc. Users can collaborate through heterogeneous hybrid collaborative interaction spaces that can be accessed through virtual or augmented reality, but also simple video conferencing or even a voice-only or text-only interface. These new spaces for collaboration are, however, often difficult to use and less satisfactory than face-to-face interactions, suggesting the need for more research on collaborative activities and on how to support them (Faidley 2018; Karl et al. 2022; Kemp and Grieve 2014; Singh et al. 2022; Waizenegger et al. 2020).

A substantive body of literature on teams, collaborative learning, and computersupported technologies provides evidence related to individual, contextual, and technological factors impacting the collaboration quality and efficiency. For example, teacherbased skills that are critical for enhancing collaboration are, among others, the abilities to plan, monitor, support, consolidate, and reflect upon student interaction in group work (Kaendler et al. 2016). Research focuses also on investigating the most relevant tasks and evaluating the possibilities offered by technology to support, to assess (e.g., Nouri et al. 2017; Graesser et al. 2018), and/or to learn the skills involved in pursuing effective and satisfying collaboration (see e.g., Schneider et al. 2018; Doyle 2021; Ainsworth and Chounta 2021).

#### **3. Labelization: Valorization of the 4Cs and Assessing Support for Their Development**

Moving from the nature of the 4Cs and their individual assessment and towards the ways in which institutions can support their development in individuals, we can now address the fundamentally important question of how best to support and promote this 21st century educational mission within and among institutions themselves. This also raises the question of the systemic recognition of educational settings that are conducive to the development of the 4Cs. In response to these questions, the nature and value of labelization is now presented.

A label is "a special mark created by a trusted third party and displayed on a product intended for sale, to certify its origin, to guarantee its quality and to ensure its conformity with the standards of practices in force" (Renard 2005). A label is therefore a way of informing the public about the objective properties and qualities of a product, service, or system. The label is usually easily identifiable and can be seen as a proof that a product or service, a company, or an organization complies with defined criteria. Its effectiveness is therefore closely linked to the choice of requirements set out in its specifications, as well as to the independence and rigor of the body that verifies compliance with the criteria.

#### *3.1. Labeling as a Means of Trust and Differentiation*

As a sign of recognition established by a third party, the label or certification can constitute a proof of trust aiming to reassure the final consumer. According to Sutter (2005), there are different means of signaling trust. First, the brand name of a product or service and its reputation can, in itself, constitute a label when this brand name is recognized on the market. Second, various forms of self-declaration, such as internal company charters, though not statements assessed by a third party, show an internal commitment that can provide reassurance. Finally, there is certification or labeling, which is awarded by an external body and requires a third-party assessment by a qualified expert, according to criteria set out in a specific reference framework. It is this external body, a trusted third party, which guarantees the reliability of the label and constitutes a guarantee of credibility. Its objectivity and impartiality are meant to guarantee that the company, organization, product, or service meets defined quality or reliability criteria (Jahn et al. 2005).

Research on populations around the world (e.g., Amron 2018; Sasmita and Suki 2015) show that the buying decisions of consumers are heavily influenced by the trust they have in a brand. More specifically, third-party assurances and labelization have been shown to strongly influence customer buying intentions and purchasing behavior (e.g., Kimery and McCord 2002; Lee et al. 2004). Taking France as an example, research shows that quality certification is seen as "important" or "significant" by 76% of companies (Chameroy and Veran 2014), and decision makers feel more confident and are more willing to invest with the support of third-party approval than if their decision is merely based on the brand's reputation or its demonstrated level of social responsibility (Etilé and Teyssier 2016). Indeed, French companies with corporate social responsibility labels have been shown to have higher than average growth rates, and the adoption of quality standards is linked with a 7% increase in the share of export turnover (Restout 2020).

#### *3.2. Influence on Choice and Adoption of Goods and Services*

Studies diverge in this area, but based on the seminal work of Parkinson (1975); Chameroy and Veran (2014), in their research on the effect of labels on willingness to pay, found that in 75% of cases, products with labels are chosen and preferred to those without labels, demonstrating the impact of the label on customer confidence—provided that it is issued by a recognized third party. Thus, brands that have good reputations tend to be preferred over cheaper new brands, because they are more accepted and valued by the individual social network (Zielke and Dobbelstein 2007).

#### *3.3. Process of Labelizing Products and Services*

The creation of a label may be the result of a customer or market need, a request from a private sector of activity or from the government. Creating a label involves setting up a working group including stakeholders who are experts in the field, product managers, and a certification body in order to elaborate a reference framework. This is then reviewed by a specialized committee and validated by the stakeholders. The standard includes evaluation criteria that must be clearly defined (Mourad 2017). An audit system is set up by a trusted third party. It must include the drafting of an audit report, a system for making decisions on labeling, and a system for identifying qualified assessors. The validity of the assessment process is reinforced by this double evaluation: a first level of audit carried out by a team of experts according to a clearly defined set of criteria and a second level of decision making assuring that the methodology and the result of the audit are in conformity with the defined reference framework.

#### *3.4. Labelization of 21st Century Skills*

The world of education is particularly concerned by the need to develop and assess 21st century skills, because it represents the first link in the chain of skills acquisition, preparing the human resources of tomorrow. One important means of simultaneously offering a reliable, independent assessment of 21st century skills and valorizing them by making them a core target within an educational system (schools, universities, and teaching and training programs of all kinds) is labelization. Two examples of labelization processes related to 21st century skills were recently developed by the International Institute for Competency Development (2021; see iicd.net; accessed on 20 November 2022) working with international experts, teachers, and researchers from the University of Paris Cité (formerly Université Sorbonne Paris Cité), Oxford University, and AFNOR UK (an accredited certification body and part of AFNOR International, a subsidiary of the AFNOR group, the only standards body in France).

The last two or three decades has seen the simultaneous rise of international ranking systems and an interest in quality assurance and assessment in an increasingly competitive educational market (Sursock 2021). The aim of these labelization frameworks is to assist in the development of "quality culture" in education by offering individual programs, institutions, and systems additional independent, reliable means of benchmarking, charting progress, and distinguishing themselves based on their capacity to support and promote the development of crucial skills. Importantly, the external perspectives provided by such assessment system should be capable of being individually adapted and applied in a manner that can resist becoming rigidly imposed external standards (Sursock and Vettori 2017). Similarly, as we have seen in the literature review, the best approach to understanding and assessing a particular C is from a combination of different levels and perspectives in context. For example, important approaches to critical thinking have been made from educationally, philosophically, and psychologically focused vantage points (Lai 2011). We can also argue that understandings of creativity are also results of different approaches: the major models in the literature (e.g., the "4Ps" and "7Cs" models; see Lubart and Thornhill-Miller 2019) explicitly result from and include the objectives of different education-focused, process-focused, and "ingredient" or component-focused approaches.

The two assessment frameworks outlined in the sections that follow were formulated with these different perspectives and objective needs in mind. Given the complexity and very different natures of their respective targets (i.e., one assessing entire formal educational contexts such as institutions or programs, whereas the other targets the less multi-dimensional, informal educational activities represented by games), the assessment of the individual Cs also represents what experts consider a target-appropriate balance of education- and curriculum-focused, process-focused, and component-focused criteria for assessing each different C.

#### **4. The International Institute for Competency Development's 21st Century Competencies 4Cs Assessment Framework for Institutions and Programs**

One comprehensive attempt to operationalize programmatic-level and institutionallevel support for the development of the 4Cs is the International Institute for Competency Development's 4Cs Assessment Framework (International Institute for Competency Development 2021). Based upon expert opinion and a review of the available literature, this evaluation grid is a practical tool that divides each of the 4Cs into three "user-friendly" but topic-covering components (see Table 1 and definitions and further discussion in the sections that follow). Each of these components is then assessed across seven dimensions (see Table 2, below), designed to cover concisely the pedagogical process and the educational context. Examples for each point level are provided within the evaluation grid in order to offer additional clarity for educational stakeholders and expert assessors.


**Table 1.** Three different components of each C in IICD's 21st Century Skills 4Cs Assessment Framework.

**Table 2.** Seven dimensions evaluated for the 3 different components of each C.


\* Educational-level dependent and potentially less available for younger students or in some contexts.

The grid itself can be used in several important and different ways by different educational stakeholders: (1) by the institution itself in its self-evaluation and possible preparation for a certification or labelization process, (2) as an explicit list of criteria for external evaluation of the institution and its 4Cs-related programs, and (3) as a potential long-term development targeting tool for the institution or the institution in dialogue with the labelization process.

#### *4.1. Evaluation Grid for Creativity*

Dropping the component of "creative person" that is not relevant at the institutional level, this evaluation grid is based on Rhodes' (1961) classic "4P" model of creativity, which remains the most concise model today (Lubart and Thornhill-Miller 2019). The three "P" components retained are: *creative process*, *creative environment*, and *creative product*. Creative process refers to the acquisition of a set of tools and techniques that students can use to enhance the creativity of their thinking and work. Creative environment (also called "Press" in earlier literature) is about how the physical and social surroundings of students can help them be more creative. Finally, creative product refers to the evaluation of actual "productions" (e.g., a piece of art, text, speech, etc.) generated through the creative process.

#### *4.2. Evaluation Grid for Critical Thinking*

Our evaluation grid divides critical thinking into three main components: *critical thinking about the world*, *critical thinking about oneself* (self-reflection), as well as *critical action and decision making*. The first component refers to having an evidence-based view of the exterior world, notably by identifying and evaluating sources of information and using them to question current understandings and solve problems. Self-reflection refers to thinking critically about one's own life situation, values, and actions; it presupposes the autonomy

of thought and a certain distance as well as the most objective observation possible with regard to one's own knowledge ("meta-cognition"). The third and final component, *critical action and decision making,* is about using critical thinking skills more practically in order to make appropriate life decisions as well as to be open to different points of view. This component also addresses soft skills and attitudes such as trusting information.

Our evaluation framework for critical thinking was in part inspired by Barnett's "curriculum for critical being" (2015), whose model distinguishes two axes: one defined by the qualitative differences in the level of criticality attained and the second comprised of three different domains of application: formal knowledge, the self, and the world. The first two components of our framework (and the seven dimensions on which they are rated) reflect and encompass these three domains. Similar to Barrett's proposal, our third rubric moves beyond the "skills-plus-dispositions" model of competency implicit in much theorizing about critical thinking and adds the importance of "action"—not just the ability to think critically and the disposition to do so, but the central importance of training and practicing "critical doing" (Barnett 2015). Critical thinking should also be exercised collectively by involving students in collective thinking, facilitating the exchange of ideas and civic engagement (Huber and Kuncel 2016).

#### *4.3. Evaluation Grid for Collaboration*

The first component of collaboration skills in the IICD grid is *engagement and participation*, referring to the active engagement in group work. *Perspective taking and openness* concerns the flexibility to work with and accommodate other group members and their points of view. The final dimension—*social regulation*—is about being able to reach for a common goal, notably through compromise and negotiation, as well as being aware of the different types of roles that group members can hold (Hesse et al. 2015; Rusdin and Ali 2019; Care et al. 2016). (These last two components include elements of leadership, character, and emotional intelligence as sometimes described in other soft-skill and competency-related systems.) Participation, social regulation, and perspective taking have been identified as central social skills in collaborative problem solving (Hesse et al. 2015). Regarding social regulation in this context, recognizing and profiting from group diversity is key (Graesser et al. 2018). When describing an assessment in an educational setting of collaborative problem solving (with a task in which two or more students have to collaborate in order to solve it, each using a different set of resources), two main underpinning skills were described for the assessment: the social skill of audience awareness ("how to adapt one's own behavior to suit the needs of the task and the partner's requirements", Care et al. 2016, p. 258) and the cognitive skill of planning and executing (developing a plan to reach for a goal) (Care et al. 2016). The former is included in the perspective taking and openness rubric and the latter in the social regulation component in the IICD grid. Evans (2020) identified four main collaboration skills consistently mentioned in the scientific literature that are assessed in the IICD grid: the ability to plan and make group decisions (example item from the IICD grid: teachers provide assistance to students to overcome differences and reach a common goal during group work); the ability to communicate about thinking with the group (assessed notably in the meta-reflection strand of the IICD grid); the ability to contribute resources, ideas, and efforts and support group members (included notably in the engagement and participation as well as the social regulation components); and finally, the ability to monitor, reflect, and adapt individual and group processes to benefit the group (example item from the IICD grid: students use perspective-taking tools and techniques in group activities).

#### *4.4. Evaluation Grid for Communication*

The evaluation grid for communication is also composed of three dimensions: *message formulation, message delivery,* and *message and communication feedback*. *Message formulation* refers to the ability to design and structure a message to be sent, such as outlining the content of an argument. *Message delivery* is about effectively transmitting verbal and nonverbal aspects of a message. Finally, *message and communication feedback* refers to the ability of students and teachers to understand their audience, analyze their social surroundings, and interpret information in context. Other components of communication skills such as theory of mind, empathy, or emotional intelligence are also relevant and included in the process of applying the grid. Thompson (2020) proposes a four-component operationalized definition of communication for its assessment in students. First, they describe a comprehension strand covering the understanding and selection of adequate information from a range of sources. Message formulation in the IICD grid captures this dimension through its focus on content analysis and generation. Second, the presentation of information and ideas is mentioned in several different modes, adjusted to the intended audience, verbally as well as non-verbally. The message delivery component of the IICD grid focuses on these points. Third, the authors note the importance of communication technology and its advanced use. The IICD grid also covers the importance of technology use in its tools and techniques category, with, for example, an item that reads: students learn to effectively use a variety of formats of communication (social media, make a video, e-mail, letter writing, creating a document). Finally, Thompson (2020) describes the recognition of cultural and other differences as an important aspect of communication. The IICD grid aims at incorporating these aspects, notably in the meta-reflection category under each of the three dimensions.

#### **5. Assessing the 4Cs in Informal Educational Contexts: The Example of Games**

#### *5.1. The 4Cs in Informal Educational Contexts*

So far, the focus has been on rather formal ways of nurturing the 4Cs. Although institutions and training programs are perhaps the most significant and necessary avenues of education, they are not the sole context in which 4Cs' learning and improvement can manifest. One other important potential learning context is game play. Games are activities that are present and participated in throughout human society—by those of all ages, genders, and socio-economic statuses (Bateson and Martin 2013; Huizinga 1949; Malaby 2007). This informal setting can also provide favorable conditions to help improve the 4Cs (van Rosmalen et al. 2014) and should not be under-appreciated. Games provide a unique environment for learning, as they can foster a space to freely explore possibilities and one's own potential (de Freitas 2006). We argue that games are a significant potential pathway for the improvement of the 4Cs, and as such, they merit the same attention as more formal ways of learning and developing competencies.

#### *5.2. 4Cs Evaluation Framework for Games*

Compared to schools and educational institutions, the focus of IICD's evaluation framework for games (see International Institute for Competency Development 2021) is more narrow. Thus, it is fundamentally different from the institutional grid: games, complex and deep as they can sometimes be, cannot directly be compared to the complexity of a school curriculum and all the programs it contains. The evaluation of a game's effectiveness for training/improving a given C rests on the following principle: if a game presents affordances conducive to exercising a given skill, engaged playing of that game should help improve that skill.

The game's evaluation grid is scored based on two criteria. For example, as a part of a game's rating as a tool for the development of creativity, we determine the game must first meet two conditions. First, whether or not the game allows the opportunity for creativity to manifest itself: if creativity cannot occur in the game, it is obviously not eligible to receive ratings for that C. Second, whether or not creativity is needed in order to perform well in the game: if the players can win or achieve success in the game without needing creativity, this also means it cannot receive a rating for that C. If both conditions are met, however, the game will be considered potentially effective to improve creativity through the practice of certain components of creative behavior. This basic principle applies for all four of the Cs.

As outlined in Table 3, below, the evaluation grid for each of the four Cs is composed of five components relevant to games that are different for each of the Cs. The grid works as follows: for each of the five components of each C, we evaluate the game on a list of sub-components using two yes/no scales: one for whether it is "possible" for that subcomponent to manifest and one for whether that sub-component is "required for success" in the game. This evaluation is done for all sub-components. After this, each general component is rated on the same two indicators. If 60% (i.e., three out of five) or more sub-components are positively rated as required, the general component is considered required. Then, the game is evaluated on its effectiveness for training and improving each of the 4Cs. If 60% or more components are positively rated as required, the game will be labelized as having the potential to be effective for training and improving the corresponding C.


**Table 3.** Five different components evaluated for each C by the 4Cs assessment framework for games.

The evaluation grid for creativity is based on the multivariate model of creative potential (see Section 2.1.1 and Lubart et al. 2013 for more information) and is composed of four cognitive factors and one conative factor: *originality*, *divergent thinking*, *convergent thinking*, *mental flexibility*, and *creative dispositions*. *Originality* refers to the generation of ideas that are novel or unexpected, depending on the context. *Divergent thinking* corresponds to the generation of multiple ideas or solutions. *Convergent thinking* refers to the combination of multiple ideas and the selection of the most creative idea. *Mental flexibility* entails changing perspectives on a given problem and breaking away from initial ideas. Finally, *creative dispositions* concerns multiple personality-related factors conducive to creativity, such as openness to experience or risk taking.

The evaluation grid for critical thinking echoes Halpern's (1998) as well as Marin and Halpern's (2011) considerations for teaching this skill, that is, taking into consideration thinking skills, metacognition, and dispositions. The five components of the critical thinking grid are: goal-adequate discernment, objective thinking, metacognition, elaborate reasoning, and uncertainty management. Goal-adequate discernment entails the formulation of inferences and the discernment of contradictions when faced with a problem. Objective thinking corresponds to the suspension of one's own judgment and the analysis of affirmations and sources in the most objective manner possible. Metacognition, here, is about questioning and reassessing information, as well as the awareness of one's own cognitive biases. Elaborate reasoning entails reasoning in a way that is cautious, thorough, and serious. Finally, uncertainty management refers to the dispositional propensity to tolerate ambiguity and accept doubt.

The evaluation grid for collaboration is based on the quality of collaboration (QC) method (Burkhardt et al. 2009; see Section 2.4.2 for more details) and is composed of the following five components: collaboration fluidity, well-argued deliberation and consensusbased decision, balance of contribution, organization and coordination, and cognitive syncing, input, and support. Collaboration fluidity entails the absence of speech overlap and the presence of a good flow in terms of turns to speak. Well-argued deliberation and consensus-based decision is about contributing to the discussion and task at hand, as well as participating in discussions and arguments, in order to obtain a consensus. Balance of contribution refers to having equal or equivalent contributions to organization, coordination, and decision making. Organization and coordination refers to effective management of roles, time, and "deadlines", as well as the attribution of roles depending on participants' skills. Finally, cognitive syncing, input, and support is about bringing ideas and resources to the group, as well as supporting and reinforcing other members of the group.

The five components used to evaluate communication in games include both linguistic, pragmatic, and social aspects. Linguistic skills per se are captured by the mastery of written and spoken language component. This component assesses language comprehension and the appropriate use of vocabulary. Pragmatic skills are captured by the verbal and non-verbal communication components and refer to the efficient use of verbal and body signals in the context of the game to achieve one's communicative goals (Grassmann 2014; Matthews 2014). Finally, the grid also evaluates social skills with its two last components, social interactions and social cognition, which, respectively, refer to the ability to interact with others appropriately—including by complying with the rules of the game—and to the understanding of other people' mental states (Tomasello 2005).

#### **6. Discussion and Conclusions**

Each of the 4Cs is a broad, multi-faceted concept that is the subject of a tremendous amount of research and discussion by a wide range of stakeholders in different disciplines, professions, and parts of the educational establishment. The development of evaluation frameworks to allow support for the 4Cs to be assessed and publicly recognized, using a label, is an important step for promoting and fostering these skills in educational contexts. As illustrated by IICD's 4Cs Framework for educational institutions and programs, as well as its games/activities evaluation grid, the specific criteria to detect support for each C can vary depending upon the educational context (e.g., formal and institutional level or informal and at the activity level). Yet considering the 4Cs together highlights some additional observations, current challenges, and opportunities for the future that are worthy of discussion.

#### *6.1. Interrelationships between the 4Cs and a New Model for Use in Pedagogy and Policy Promotion*

One very important issue for understanding the 4Cs and their educational implementation that can be simultaneously a help and a hindrance for teaching them—and also a challenge when assessing them—is their multidimensionality and interrelatedness. In other words, the 4Cs are not entirely separate entities but instead, as Figure 2 shows, should be seen as four interlinked basic "elements" for future-oriented education that can help individuals in their learning process and, together, synergistically "bootstrap" the development of their cognitive potentials. Lamri and Lubart (2021), for example, found a certain base level of creativity was a necessary but not sufficient condition for success in managerial tasks, but that high-level performance required a combination of all four Cs. Some thinkers have argued that one cannot be creative without critical thinking, which also requires creativity, for example, to come up with alternative arguments (see Paul and Elder 2006). Similarly, among many other interrelationships, there is no collaboration without communication—and even ostensibly individual creativity is a "collaboration" of sorts with the general culture and precursors in a given field. As a result, it ranges from impossible to suboptimal to teach (or teach towards) one of the 4Cs without involving one or more of the others, and this commingling also underscores the genuine need and appropriateness of assessing them together.

**Figure 2.** "'Crea-Critical-Collab-ication': a Dynamic Interactionist Model of the 4Cs". (Illustration of the interplay and interpenetration of creativity, critical thinking, collaboration, and communication shown in dimensional space according to their differing cognitive/individual vs. social/interpersonal emphases; (© 2023, Branden Thornhill-Miller. All Rights Reserved. thornhill-miller.com; accessed on 20 January 2023)).

From this perspective, Thornhill-Miller (2021) proposed a "dynamic interactionist model of the 4Cs" and their interrelated contributions to the future of education and work. Presented in Figure 2, this model is meant to serve as a visual and conceptual aid for understanding the 4Cs and their interrelationships, thereby also promoting better use and understanding of them in pedagogical and policy settings. In addition to suggesting the portmanteau of "crea-critical thinking" as a new term to describe the overlap of much of the creative and critical thinking processes, the title of this model, "Crea-Critical-Collabication", is a verbal representation of the fluid four-way interrelationship between the 4Cs visually represented in Figure 2 (a title meant to playfully repackage the 4Cs for important

pedagogical and policy uses). This model goes further to suggest some dimensional differences in emphases that, roughly speaking, also often exist among the 4Cs: that is to say, the frequently greater emphasis on cognitive or individual elements at play in creativity and critical thinking in comparison to the social and interpersonal aspects more central to communication and collaboration (Thornhill-Miller 2021).

Similarly focused on the need to promote a phase change towards future-oriented education, Lucas (2019) and colleagues have suggested conflating creative thinking and critical thinking in order to propose "3Cs" (creative thinking, communication, and collaboration) as new "foundational literacies" to symmetrically add to the 3Rs (Reading, wRiting, and aRithmetic) of previous educational eras. Although we applaud these efforts, from our applied research perspective, we believe that the individual importance of, and distinct differences between, creative thinking and critical thinking support preserving them both as separate constructs in order to encourage the greatest development of each of them. Moreover, if only three categories were somehow required or preferable, one could argue that uniting communication and collaboration (as "collab-ication" suggests) might be preferable—particularly also given the fact that substantial aspects of communication are already covered within the 3Rs. In any case, we look forward to more such innovations and collaborations in this vibrant and important area of work at the crossroads between research, pedagogy, and policy development.

#### *6.2. Limitations and Future Work*

The rich literature in each of the 4Cs domains shows the positive effects of integrating these dimensions into educational and professional curricula. At the same time, the complexity of their definitions makes them difficult to assess, both in terms of reliability (assessment must not vary from one measurement to another) and of validity (tests must measure that which they are intended to measure). However, applied research in this area is becoming increasingly rigorous, with a growing capacity to provide the necessary tools for evidence-based practice. The development of these practices should involve interdisciplinary teams of teachers and other educational practitioners who are equipped and trained accordingly. Similarly, on the research side, further exploration and clarification of subcomponents of the 4Cs and other related skills will be important. Recent efforts to clarify the conceptual overlap and hierarchical relations of soft skills for the future of education and work, for example, have been helpful and promising (e.g., Joie-La Marle et al. 2022; Lamri et al. 2022). But the most definitive sort of taxonomy and measurement model that we are currently lacking might only be established based on the large-scale administration of a comprehensive battery of skill-measuring psychometric tests on appropriate cross sections of society.

The rapid development and integration of new technologies will also aid and change the contexts, resources, and implementation of the 4Cs. For example, the recent developments make it clear that the 4Cs will be enhanced and changed by interaction with artificially intelligence, even as 4Cs-related skills will probably, for the same reason, increasingly constitute the core of available human work in the future (see, e.g., Ross 2018). Similarly, research on virtual reality and creativity suggest that VR environments assist and expand individual and collaborative creativity (Bourgeois-Bougrine et al. 2022). Because VR technologies offer the possibility of enhanced and materially enriched communication, collaboration, and information availability, they not only allow for the enhancement of creativity techniques but also for similar expansions and improvements on almost all forms of human activity (see Thornhill-Miller and Dupont 2016)—including the other three Cs.

#### *6.3. Conclusion: Labelization of the 4Cs and the Future of Education and Work*

Traditional educational approaches cannot meet the educational needs of our emergent societies if they do not teach, promote, and assess in line with the new learner characteristics and contexts of the 21st century (Sahin 2009). The sort of future-oriented change and development required by this shift in institutional practices, programming, and structure will likely meet with significant resistance from comfortably entrenched (and often outdated) segments of traditional educational and training establishments. Additional external evaluation and monitoring is rarely welcome by workers in any context. We believe, however, that top-down processes from the innovative and competition-conscious administrative levels will be met by bottom-up demands from students and education consumers to support these institutional changes. And we contend that efforts such as labelizing 4C processes will serve to push educators and institutions towards more relevant offerings, oriented towards the future of work and helping build a more successful future for all.

In the end, the 4Cs framework seems to be a manageable, focused model for modernizing education, and one worthy of its growing prevalence in the educational and research marketplace for a number of reasons. These reasons include the complexity and cumbersome nature of larger alternative systems and the 4Cs' persuasive presence at the core of a number of early and industry-driven frameworks. In addition, the 4Cs have benefitted from their subsequent promotion by organizations such as the OECD and the World Economic Forum, as well as some more direct support from recent empirical research. The promotion, teaching, and assessment of the 4Cs will require a complex social intervention and mobilization of educational resources—a major shift in pedagogy and institutional structures. Yet the same evolving digital technologies that have largely caused the need for these massive, rapid changes can also assist in the implementation of solutions (van Laar et al. 2017). To the extent that future research also converges on such a model (that has already been found pedagogically useful and policy-friendly by so many individuals and organizations), the 4Cs framework has the potential to become a manageable core for 21st century skills and the future of education and work—one that stakeholders with various agendas can already begin building on for a better educational and economic future together.

**Author Contributions:** Conceptualization, B.T.-M. and T.L.; writing—original draft preparation, B.T.- M., A.C., M.M., J.-M.B., T.M., S.B.-B., S.E.H., F.V., M.A.-L., C.F., D.S., F.M.; writing—review and editing, B.T.-M., A.C., T.L., J.-M.B., C.F.; visualization, B.T.-M.; supervision, B.T.-M., T.L.; project administration, B.T.-M., T.L. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not Applicable.

**Conflicts of Interest:** B.T.-M. and T.L. are unpaid academic co-founder and project collaborator for the International Institute for Competency Development, whose labelization frameworks (developed in cooperation with Afnor International and the LaPEA lab of Université Paris Cité and Université Gustave Eiffel) are used as examples in this review. S.E.H. and M.A.-L. are employees of AFNOR International. No funding was received to support this research or article, which reflects the views of the scientists and researchers and not their organizations or companies.

#### **References**


Peña-López, Ismaël. 2017. *PISA 2015 Results (Volume V): Collaborative Problem Solving*. Paris: PISA, OECD Publishing.


Rhodes, Mel. 1961. An Analysis of Creativity. *The Phi Delta Kappan* 42: 305–10.


Rietzschel, Eric F., Bernard A. Nijstad, and Wolfgang Stroebe. 2006. Productivity Is Not Enough: A Comparison of Interactive and Nominal Brainstorming Groups on Idea Generation and Selection. *Journal of Experimental Social Psychology* 42: 244–51. [CrossRef]

Ross, David. 2018. Why the Four Cs Will Become the Foundation of Human-AI Interface. Available online: https://www.gettingsmart. com/2018/03/04/why-the-4cs-will-become-the-foundation-of-human-ai-interface/ (accessed on 2 November 2022).

Rothermich, Kathrin. 2020. Social Communication Across the Lifespan: The Influence of Empathy [Preprint]. *SocArXiv*. [CrossRef] Rusdin, Norazlin Mohd, and Siti Rahaimah Ali. 2019. Practice of Fostering 4Cs Skills in Teaching and Learning. *International Journal of*


Scialabba, George. 1984. Mindplay. *Harvard Magazine* 16: 19.


Sutter, Éric. 2005. Certification et Labellisation: Un Problème de Confiance. Bref Panorama de La Situation Actuelle. *Documentaliste-Sciences de l Information* 42: 284–90. [CrossRef]

Taddei, François. 2009. *Training Creative and Collaborative Knowledge-Builders: A Major Challenge for 21st Century Education*. Paris: OCDE.


Yue, Meng, Meng Zhang, Chunmei Zhang, and Changde Jin. 2017. The Effectiveness of Concept Mapping on Development of Critical Thinking in Nursing Education: A Systematic Review and Meta-Analysis. *Nurse Education Today* 52: 87–94. [CrossRef]

Zielke, Stephan, and Thomas Dobbelstein. 2007. Customers' Willingness to Purchase New Store Brands. *Journal of Product & Brand Management* 16: 112–21. [CrossRef]

Zlati´c, Lidija, Dragana Bjeki´c, Snežana Marinkovi´c, and Milevica Bojovi´c. 2014. Development of Teacher Communication Competence. *Procedia, Social and Behavioral Sciences* 116: 606–10. [CrossRef]

**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

## *Review* **Reconciling Hard Skills and Soft Skills in a Common Framework: The Generic Skills Component Approach**

**Jeremy Lamri \* and Todd Lubart**

LaPEA, Université Paris Cité & Univ Gustave Eiffel, F-92100 Boulogne-Billancourt, France **\*** Correspondence: jeremylamri@gmail.com

**Abstract:** The distinction between hard and soft skills has long been a topic of debate in the field of psychology, with hard skills referring to technical or practical abilities, and soft skills relating to interpersonal capabilities. This paper explores the generic composition of any skill, proposing a unified framework that consists of five distinct components: knowledge, active cognition, conation, affection, and sensory-motor abilities. Building upon previous research and theories, such as Hilgard's "Trilogy of Mind", the generic skill components approach aims to provide a comprehensive understanding of the structure and composition of any skill, whether hard or soft. By examining these components and their interactions, we can gain a more in-depth understanding of the nature of skills and their development. This approach has several potential applications and implications for various fields, including education, training, and workplace productivity. Further research is needed to refine and expand upon the generic skill components theory, exploring the interactions between the different components, as well as the impact of contextual factors on skill development and use.

**Keywords:** skills; soft skills; hard skills; cognition; conation; affection

#### **1. Introduction**

In today's complex, interconnected world, the importance of having a diverse set of skills for success is undeniable. The ability to define, develop and utilise one's skills is considered a vital part of personal and professional success. This success depends heavily on the acquisition and maintenance of both soft and hard skills. In the modern workforce, employers are searching for the perfect candidate, the one who can bring a combination of skills to the table. Indeed, skills can generally be divided into two main categories—hard skills and soft skills. Hard skills refer to technical or practical abilities, such as programming languages, engineering, accounting, and other occupational skills, whereas soft skills are interpersonal capabilities, such as communication, problem-solving, and emotional intelligence (Cimatti 2016; Laker and Powell 2011).

Although these two types of skills are often categorised separately, it is important to understand their interdependence, as well as their contributions to certain areas of expertise. In recent years, there has been increasing recognition of the importance of soft skills in many areas, including education and business (Andrews and Higson 2008; Succi and Canovi 2020). The so-called "soft skill revolution" has seen a growing interest in developing and assessing these skills, as organisations have become increasingly aware of their value in the workplace. Yet, there is still some debate about what constitutes a soft skill, and to what extent hard skills remain essential for success. Despite the acknowledged value of soft skills, the lack of a standard definition or systematic approach to measuring and assessing these skills poses a challenge when attempting to review and compare them (Dede 2010; Robles 2012; Rasipuram and Jayagopi 2020).

Even before challenging the concept of soft skills, there is the question of what a "skill" is, and how to develop certain skills, as it remains an ongoing area of research for psychologists and educators. Whereas the study of skills has traditionally been associated

**Citation:** Lamri, Jeremy, and Todd Lubart. 2023. Reconciling Hard Skills and Soft Skills in a Common Framework: The Generic Skills Component Approach. *Journal of Intelligence* 11: 107. https://doi.org/ 10.3390/jintelligence11060107

Received: 31 December 2022 Revised: 5 May 2023 Accepted: 9 May 2023 Published: 1 June 2023

**Copyright:** © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

with individual traits such as intelligence and talent, an emerging field of inquiry suggests that the composition of any skill is made up of several core elements. Overall, skills are an important foundation for development, yet much research is needed to understand better the generic components of skills. Although soft skills and hard skills seem very different in the way they are used and observed, what actually makes them inherently different? If both are actually skills, they may have more in common than it seems. In recent years, research into the generic composition of any skill, and the relationship between soft skills and hard skills, has gained increased interest due to its implications for workplace productivity.

Researchers have identified that any workplace skill requires a combination of hard and soft skills (van der Vleuten et al. 2019; Lyu and Liu 2021). They have also elucidated that there are shared components between hard and soft skills which could be seen as the bridge between them (Pieterse and Van Eekelen 2016; Kuzminov et al. 2019). This presents an interesting opportunity for educators and trainers to develop individuals in an integrated manner, allowing for an understanding of both technical and non-technical components of skills.

This paper explores the generic composition of any skill and the common ground between soft skills and hard skills. Hilgard's "Trilogy of Mind" (1980) provides useful insights into the debate, by suggesting that all skills—whether hard or soft—can be understood in terms of three distinct components: cognition, conation, and affection. In this article, we will discuss how Hilgard's theory can be applied in order to describe the composition of any skill, and argue that, theoretically, there is no difference between soft and hard skills, opening the way to a generic skills framework.

#### **2. Critical Literature Review**

#### *2.1. Definition of Skill*

As the distinction between soft and hard skills is not standardised, it is important to consider different definitions of "skill" for the purposes of this article. Skill is a multifaceted concept that has been studied extensively in the scientific literature (Vallas 1990; Clarke and Winch 2006; Green 2011). According to the definition of the 2023 Merriam-Webster dictionary, a skill is "the ability to use one's knowledge effectively and readily in execution or performance" (Merriam-Webster n.d.). It describes the ability that develops from practice, training, and experience to perform a specific task to a certain standard. It has been subject to considerable examination by researchers across different disciplines, including psychology, neuroscience, education and sports science (Gagné and Fleishman 1959; Frese and Stewart 1984; Bo et al. 2008).

Boyatzis (1982) defined "skill" as "an underlying characteristic of a person that has a causal relationship with their average or superior performance in a given function". In more concrete terms, "skill" refers to an individual's ability to accomplish tasks by utilising appropriate resources, including those acquired through training or previous experience (Le Boterf 2000). A skill can be conceptualised as specific know-how that is pertinent to a given situation, resulting in the combination of knowledge, other mental abilities and physical strength, agility, coordination, and motor abilities (Green 2011). This definition provides a clear understanding of the underlying competencies and knowledge necessary to successfully carry out any task, regardless of whether it concerns soft or hard skills. In addition, the success of skills is partially dependent on the direct content of the tasks, abilities, values, interests, and the environment of the individual (Le Boterf 2000).

As noted by DeKeyser (2020), the term "skill" encompasses the ability to process and understand information, interpret, and use it in order to complete a task. It involves both cognitive and motor abilities, which together form a basis for mastery (Roebers et al. 2014; Van der Fels et al. 2015). Both require knowledge and the ability to store and recall information, as well as the ability to interpret and apply it correctly. Through practice and repetition, skills become increasingly automatic and rapid, and proficiency is observed.

As such, "skill" can be seen as the ability to retrieve knowledge and apply it to a task in a proficient manner. Cognitive factors include working memory, various forms of reasoning, and problem-solving (Carroll 2003). Motor abilities include factors such as coordination, muscle and joint strength, and speed (Zajac 1993). In more psychological terms, they can be seen as a component of behavioural abilities. When including motor abilities, the dyad created by cognitive and behavioural components plays an important role in the development and refinement of skills. This is an important concept to recognise when considering the notion of skill, as both the ability to understand and interpret knowledge, as well as the application of what has been learnt are essential for skill development. In conclusion, a skill is an ability that is refined with training, technique, and experience. It is noted to involve a combination of cognitive and behavioural components which interact to allow the effective completion of a given task.

A wide range of skills have been studied, such as motor skills, sensory and perceptual skills, cognitive skills, and social skills (Fischer 1980). Motor skills are defined as the ability to control and coordinate the movements and actions of the body (Newell 1991). Sensory and perceptual skills involve the ability to receive, interpret, and act upon sensory information, such as visual, auditory, and tactile data (Karni and Bertini 1997). Cognitive skills encompass the ability to think logically, problem-solve, and make decisions, whereas social skills involve the ability to interact and communicate effectively with others (Patterson 2008). Overall, skills are multifaceted constructs that enable humans to continue to grow and learn in a variety of contexts, through general practice and experience, as well as through the development of specific tasks and strategies.

#### *2.2. Definitions and Characteristics of Hard Skills*

Hard skills refer to technical, tangible, and quantifiable abilities related to the use of equipment for a specific job, such as driving a car, computer programming, or welding (Lyu and Liu 2021). Hard skills are typically acquired through training and education and are a requisite for performing job duties. They are necessary for specific tasks within an industry that requires specific expertise and proficiency, such as welding, accounting, and using a 3-D printer. As researchers note, hard skills are also differently defined along the lines of work and education. A person with a background in computer science may define hard skills as the technical abilities required for software development, whereas someone with a background in design may define hard skills as the artistic abilities needed for graphic design. The importance of hard skills has long been acknowledged in the workplace, especially because the manipulation of these skills often leads to measurable performance outcomes (Rainsbury et al. 2002; Hendarman and Cantner 2018). Consequently, they are usually emphasised during recruitment processes and have been found to play a determining role in the hiring decisions of employers (Bishop 2017; Huber 2018). Actually, both motivation and hard skills play an important role in positive job performance (Hendarman and Cantner 2018).

#### *2.3. Definitions and Characteristics of Soft Skills*

In 1972, the term "soft skills" was first used by the researcher Paul G. Whitmore, during a training conference in Texas for the US Army Continental Army Command (CONARC). Whitmore used the term "soft skills" to refer to crucial job-related skills that involve little or no interaction with machines (CONARC 1972, cited in Parlamis and Monnot 2019). They may as well be considered behaviours that a person must mobilise in order to reach a given objective competently (Tate 1995). Considering the context of hard skills, soft skills are non-technical abilities that are harder to measure and quantify (Kantrowitz 2005; Byrne et al. 2020). Soft skills involve personal, interpersonal, and intrapersonal abilities that are essential in the workplace (Dell'Aquila et al. 2017). Examples of soft skills include emotional intelligence, communication, creativity, problem-solving, team building, and stress management (Martins et al. 2020).

Unlike hard skills, soft skills tend not to be acquired through formal education and training and often require dedication, self-reflection, and self-improvement (Chell and Athayde 2011; Wisshak and Hochholdinger 2020). This does not mean that hard skills

do not require these same qualities, however, the probability of systematic acquisition seems less predictable for soft skills, and more related to personal qualities, as their use will be specific to every person. Furthermore, soft skills are typically more developed through social experience, which is why they are often referred to as "people skills" (Levasseur 2013).

There are many different terminologies when referring to soft skills, such as social competencies, interpersonal skills, or even emotional intelligence (Matteson et al. 2016). Social competencies encompass a broader range of abilities that enable individuals to navigate effectively interpersonal situations, build and maintain relationships, and work well with others. These competencies include communication, teamwork, adaptability, and cultural awareness (Rychen and Salganik 2003). Interpersonal skills refer to the abilities needed to effectively interact, communicate, and collaborate with others. These skills include active listening, empathy, conflict resolution, and negotiation (Spencer and Spencer 1993). Emotional intelligence encompasses the ability to recognise, understand, and manage one's own emotions and the emotions of others. It is closely related to interpersonal skills and includes self-awareness, self-regulation, motivation, empathy, and social skills (Goleman 1995; Mayer et al. 2008).

With over 119 labels identified in the literature in 600 publications about soft skills over the past 50 years (Joie-La Marle et al. 2022), numerous frameworks have been created to categorise and understand them. Depending on the approach, these frameworks deal with social skills, emotional skills, cognitive skills, or all of them. Their main interest is generally to delineate critical skills needed for the future of work, which is the reason why the field of education is where most frameworks are created. Researchers, schools, and even international organisations have created their own soft skills frameworks. Lamri (2018) reviewed various soft skills frameworks. In 2016, OECD released an overview of the key findings from the OECD Survey of Adult Skills (Kankaraš et al. 2016), which highlights the importance of soft skills in the labour market and discusses policy implications for developing these skills.

Overall, despite the difficulty to agree on frameworks and terminologies, the relevance of soft skills for individual success in the workplace has been widely discussed in the literature. Numerous authors have called attention to the interplay between soft skills and other personal qualities to facilitate individual performance in the workplace or in general (Rychen and Salganik 2003; Kantrowitz 2005; Cimatti 2016; Ibrahim et al. 2017). Further, soft skills can be instrumental in improving work satisfaction and are associated with higher levels of engagement, productivity, and creativity in the workplace (Palumbo 2013; Feraco et al. 2023; the role of particular individual qualities or activities has been shown in numerous studies (Reysen et al. 2019; Feraco et al. 2023).

In terms of educability, Durlak et al. (2011) published a meta-analysis related to categories of self-emotional learning (SEL). This meta-analysis examined the effectiveness of school-based social and SEL programs in enhancing students' skills, attitudes, prosocial behaviour, and academic performance. The researchers analysed data from 213 studies involving more than 270,000 students from kindergarten to high school. The results showed that students who participated in SEL programs had significantly better social and emotional skills, attitudes, and behaviour compared to their peers who did not participate in these programs. Additionally, the study found that students involved in SEL programs also had an 11 percentile-point gain in academic achievement. Another study considers soft skills through the prism of social, emotional, and behavioural skills (Soto et al. 2022).

#### *2.4. Differences and Commonalities between Hard and Soft Skills*

It is important to have both hard and soft skills in order to be successful in the workplace. Research has shown that both types of skills are necessary and having a combination of the two leads to greater success (Rainsbury et al. 2002; Vasanthakumari 2019; Lyu and Liu 2021). For example, software development requires typically a variety of technical know-how and problem-solving capabilities (Groeneveld et al. 2021). For an individual to successfully complete such a task, he or she must often combine soft skills such as creativity and knowledge of various programming methods to come up with a successful solution. Designers must master computer software and physical tools to create prototypes as well as people skills to interact with clients or team members in collective design projects.

Hard skills are necessary for specific knowledge-based tasks and are often taught in universities and technical schools. On the other hand, soft skills are often a better predictor of workplace success than hard skills, as they are essential for personal and interpersonal functioning (Hargood and Peckham 2017). Soft skills can help to identify candidates who have the necessary qualities to lead, manage, and collaborate, which are essential for a successful and productive workplace (Rainsbury et al. 2002). Additionally, soft skills are also important for customer service, which is a required and necessary component of most work environments.

Whereas the different terminologies highlight the various aspects of hard and soft skills, it is important to recognise that these skills often intersect and support one another in various contexts. As the literature continues to evolve, researchers are increasingly examining the interrelationships between hard and soft skills and their combined contribution to individual and organisational success. On many occasions, the differences between soft skills and hard skills are often difficult to discern.

It is possible for an individual to have both strong soft and hard skills, and studies tend to show that it is the combination of both that increases an individual's chances for success in the workforce by providing a well-rounded and competitive toolkit for employers (Rainsbury et al. 2002; Succi and Canovi 2020). Having a mixture of both types of skills is seen as a requirement for many positions.

When seeking to hire candidates, employers should consider the importance of both soft and hard skills. Although employers want typically to find someone who has technical expertise and qualifications, they should consider attributes such as creativity, communication, interpersonal skills, and problem-solving, as well (Lyu and Liu 2021). Research has shown that hard skills become obsolete more quickly than soft skills (Dominici 2019; Schultheiss and Backes-Gellner 2022), so employers should take into account the importance of both types of abilities when hiring. Furthermore, employers should also provide the necessary training and mentorship to ensure that their employees have the correct skillsets for the job (Succi and Canovi 2020).

Generally speaking, the criteria for determining whether a skill is soft or hard depend on the context in which the skill is used. Some researchers argued that soft skills are often seen as being more "Person-Centred" whereas hard skills are classified as "Task-Centred", emphasising the need for individuals to be able to both interact with and help others (Rodríguez-Jiménez et al. 2021). As a result, soft skills are typically viewed as more important when it comes to interpersonal aspects of professional life such as communication, problem-solving, customer service, and teamwork, among others. Hard skills are generally evaluated and valued based on their effectiveness with regard to the completion of a specific task.

Although hard and soft skills have different definitions and uses, they also overlap to some degree (Green 2011; Cinque 2016). For example, communication, although traditionally categorised as a soft skill, also involves technical aspects like data analysis and writing, using software to produce presentations. Similarly, interpersonal skills include specific knowledge about group behaviour and social codes, which could be seen as a hard skill (Bishop 2017). There exists an interdependent relationship between the two, with each trait enabling the other to succeed (Lyu and Liu 2021). As an example, hard skills such as accounting or designing require the support of certain soft skills, like communication and problem-solving, to truly display the potential of the hard skill. Additionally, numerous studies show a positive relationship between soft skills and hard skills performance (Kuzminov et al. 2019; Lyu and Liu 2021), suggesting the need for a synergistic combination of the two that can lead to successful job outcomes.

#### **3. From Skills Theories to the Generic Skills Component Approach**

#### *3.1. Foundations for the Generic Skill Components Approach*

Is the distinction between hard/soft useful? Is there, metaphorically, a scale of "hardness" of skills, like Mohs' scale for the hardness of minerals, ranging from talc (very soft) to diamonds (very hard)? Numerous authors have raised the idea of a continuum from hard to soft skills passing by a vast mid-scale with semi-hard and semi-soft skills (see Andrews and Higson 2008; Clarke and Winch 2006; Dell'Aquila et al. 2017; Hendarman and Cantner 2018; Lyu and Liu 2021; Spencer and Spencer 1993; Rychen and Salganik 2003). Le Boterf (2000) suggests that skills are better understood as a continuum, with some skills containing both hard and soft components.

The generic skill components approach builds upon these recent findings, suggesting that all skills can be understood through a shared framework of five distinct components: knowledge, active cognition, conation, affection, and sensory-motor abilities. This integrated approach has the potential to reconcile the traditional distinction between hard and soft skills, providing a more comprehensive understanding of the complex nature of skills and their development.

#### *3.2. Discrediting Skills as Discrete Entities*

Working on a generic structure for all skills implies that skills are not discrete entities as such. We believe there is a necessity to clarify that aspect, before moving towards the construction of a generic skills approach. Consider the following arguments:

1. Overlapping and interrelated nature of skills: Skills are often interconnected and interdependent, making it difficult to clearly separate them into distinct categories. For example, the successful application of technical skills often depends on the presence of effective interpersonal skills, and vice versa (Kavé and Yafé 2014; Gardiner 2017). This overlap and interrelatedness challenges the idea that skills exist as discrete entities (Greenwood et al. 2013; Bean et al. 2018).

2. Contextual factors: The relevance and importance of specific skills can vary depending on the context in which they are applied. This contextual variability can lead to differing interpretations and classifications of skills, further challenging the idea of skills as discrete and stable entities (Perkins and Salomon 1989; Hall and Magill 1995; Widdowson 1998).

3. Evolving skill requirements: The rapidly changing nature of work and technological advancements requires individuals to adapt continuously and develop new skills. As a result, the boundaries between different skill categories may become increasingly blurred as individuals are expected to possess a diverse and dynamic skillset (Dede 2010; Hargood and Peckham 2017; Dominici 2019).

4. Limitations of terminologies: The use of specific terminologies for hard and soft skills can sometimes oversimplify or constrain our understanding of the multidimensional nature of skills. By focusing on specific aspects or dimensions of skills, these terminologies may inadvertently perpetuate the idea that skills are discrete entities, rather than acknowledging the complex, interconnected permeable nature of skill development and application (Matteson et al. 2016; Lyu and Liu 2021).

The overlapping and interrelated nature of skills, the continuum perspective, contextual factors, evolving skill requirements, and the limitations of terminologies contribute to the difficulty of treating skills as discrete entities. Recognising these challenges can help researchers and practitioners develop more nuanced and integrative approaches to skill development and assessment. Building on this analysis, we believe there is a need for a unified approach to the structure of skills.

#### *3.3. Using Goldstein and Hilgard's Work as a Core Basis*

The ambition to find a generic structure for skills is not new. Goldstein (1989) proposed a framework, with four components structuring any skill: cognitive, affective, motivational, and behavioural. In Goldstein's, cognitive components involve the understanding and

knowledge associated with a skill, such as problem-solving and analytical skills. Affective components involve emotions and attitudes, such as self-awareness and empathy. Motivational components involve the drive and determination to succeed, such as perseverance and ambition. Last, behavioural components involve the actual physical performance of a skill, such as hand-eye coordination and agility.

Although the literature is filled with definitions and discussions about skills, we choose in this article to use the work of Goldstein (1989) as a primary basis. His work, both theoretical and empirical, provides a comprehensive framework for understanding, designing, implementing, and evaluating skills development in organisations.

Applying these four components to hard and soft skills, we can see that all skills are composed of the same elements, but with different weights depending on the context in which they are used. For example, a hard skill such as programming would require a higher level of cognitive ability but lower levels of affection. In contrast, a soft skill such as active listening would require a higher level of affection but lower levels of cognition. In that way, Goldstein's framework seems a relevant basis to reconcile soft skills and hard skills. However, it is necessary to take a step back and take a closer look at Goldstein's components.

Goldstein's work relates to Hilgard's (1980a) 'Trilogy of Mind', which describes human consciousness in terms of three main dimensions: cognition, conation, and affection. Hilgard (1975, 1980b, 1986) examines learning, personality, and hypnosis, and how they interact with one another to shape our understanding of the mind. Hilgard's trilogy is itself based on the 'Trilogy of Mind' that Emmanuel Kant espoused.

Hilgard's conception of these concepts differs from Goldstein's:


In this approach, conation has a clear link with cognition and action, and we believe that, with some adaptations, it can be a promising way to apprehend motivational aspects, known as "volition" in some frameworks. Cognition should be treated as an active dynamic process. In this process, knowledge is acquired, used, transformed, and produced. It is however useful to distinguish the knowledge itself and the information-processing actions in which this knowledge is used.

Affection as seen by Hilgard seems richer than what is envisioned by Goldstein and relates better to the concept of emotional intelligence (Goleman 1995). Goldstein underlines the importance of the body actually taking action. However, calling it behaviour might be confusing, regarding the extensive literature about behaviour, and the way behavioural psychology apprehends it. Following Goldstein's definition, we believe sensory-motor abilities to be more appropriate as a component name.

Considering these adjustments, we propose the following revised framework for any skill, composed of five distinct components:


Using this framework, it becomes possible to describe both soft skills and hard skills in the same way. With time, we believe the distinction between both types of skills may become either obsolete or insufficient. Only the specific content and weight of each component would matter in order to describe a skill, to determine the overlap between two skills, or the transferability from one skill to another.

#### *3.4. Developing the Generic Skill Components Approach*

The generic skill components approach aims to provide a comprehensive understanding of the structure and composition of any skill. This approach posits that all skills, whether hard or soft, can be understood in terms of five distinct components: knowledge, active cognition, conation, affection, and sensory-motor abilities. By examining these components and their interactions, we can gain a more in-depth understanding of the nature of skills and their development.

This approach is supported by previous research that has identified common elements across various types of skills. For example, Rychen and Salganik (2003) propose a model of key competencies that includes cognitive, intrapersonal, and interpersonal dimensions, which align with the active cognition, conation, and affection components of the generic skill components approach. Similarly, other studies highlight the importance of cognitive, affective, and behavioural processes in the development and application of both hard and soft skills (Parlamis and Monnot 2019; Soto et al. 2022). Our approach extends beyond existing models by incorporating sensory-motor abilities, which are often overlooked in discussions of skill development. This inclusion acknowledges the importance of physical and perceptual abilities in the successful application of many skills, particularly in fields such as sports, manufacturing, and healthcare.

This approach has several potential applications and implications for various fields, including education, training, and management. By understanding the generic components of skills, educators and trainers can develop more effective and holistic approaches to skill development, integrating both technical and non-technical components. In the workplace, a greater understanding of the generic composition of skills can help inform hiring decisions, performance evaluations, and employee development programs. If a skill has a major active cognition component, the resulting pedagogic engineering will be very different compared to a skill with a major knowledge component.

Further research is needed to refine and expand upon the generic skill components approach. Future studies could explore the interactions between the different components, as well as the impact of contextual factors on skill development and use. Indeed, the generic skill components approach highlights the importance of context in the development and application of skills, suggesting that educators and trainers should consider the specific environments in which their students or employees will be applying their skills. This may require the development of more context-specific training programs that focus on the unique challenges and opportunities presented by different work environments. Additionally, researchers could investigate the potential for more distinct skill categories and their implications for various domains.

#### *3.5. Tentative Representation of the Generic Skills' Components Framework*

Although the approach needs to be further developed and tested empirically, we propose in this article an attempt at visual representation, displaying the five generic components in a diagram (see Figure 1). This diagram may be seen as a template to be used for skills description, as proposed later.

**Figure 1.** Visual representation of the generic skills' components framework.

Our understanding of generic skills components would be that all components exist independently and need to be associated to create the necessary skill. This implies that they are not relative to each other, meaning that for a given skill, it is possible that all components are required at a very high level of mastery or development. Furthermore, conversely, for another skill, it is possible that all components are required at a very low level. In this manner, all types of combinations are possible, the point being that the necessity of one component at a high level does not determine the level of other components.

#### *3.6. Tentative Representation of Skills Composition Using the Framework*

Below, we propose three examples of using the framework to represent skills: oral communication, Python programming, and logical analysis. At this stage, the assessment is very basic, as it results in a consensus among the authors, having both theoretical and empirical experience in skills expertise. These specific cases of skill descriptions will need to be challenged in order to be considered consensual, but the purpose of this section is rather to show the possibilities offered by the generic skills' components approach. For each skill, we propose:


For the skill "oral communication", which is usually referred to as a soft skill, we describe below on a scale of importance of 1 to 5 for each component, the composition for each component (see Figure 2):

1. Knowledge: 4/5—Knowledge is essential for effective oral communication, as it involves understanding the topic being discussed, the context, and the audience. Having a solid grasp of the subject matter, as well as cultural and social norms, allows the speaker to convey messages accurately and effectively. Additionally, internal knowledge helps the speaker to convey relevant information and experiences to support their points.


It is interesting to observe that using the framework, it appears that all components are relevant to the skill of oral communication. This example shows the value of such skills that can be underestimated in their complexity.

To develop the skill of oral communication using this framework, a pedagogical program could be designed as follows:

	- a. Provide learners with the necessary knowledge related to the subject matter they will be communicating, whether it is through lectures, research, or reading.
	- b. Encourage learners to integrate this knowledge into their communication to increase their credibility and effectiveness.
	- a. Provide learners with opportunities to practise active listening and critical thinking to understand better the needs of their audience and adapt their communication accordingly.

b. Encourage learners to use visual aids or other communication tools to increase their impact and effectiveness.

#### 3. Conation:


#### 4. Affection:

	- a. Provide learners with opportunities to practise their oral communication skills, such as pronunciation, articulation and voice projection exercises.
	- b. Encourage learners to practise clear and effective body language to enhance their overall communication skills.

Overall, a training program created according to the skills generic components approach should emphasise the importance of all five components of the framework and provide learners with the opportunity to develop each one in a holistic and integrated manner. By focusing on all the aspects of oral communication, learners can develop the skills they need to communicate effectively and build positive, meaningful relationships with those around them.

(B) Example 2: Python programming

For the skill "Python programming", which is usually referred to as a hard skill, we indicate the importance of each component on a 5-point scale, and describe, the composition for each component (see Figure 3):


5. Sensory Motor Abilities: 1/5—Sensory motor abilities have minimal importance in Python programming. While basic motor skills are needed for typing and using a computer, the primary focus in programming is on cognitive and knowledge-based skills. However, maintaining proper ergonomics and posture while working at a computer can help prevent physical strain and promote overall well-being.

**Figure 3.** Visual representation of the generic skills' components framework for the skill "Python programming".

It is interesting to observe that using the framework, it appears that active cognition and knowledge seem to be the most important components for the skill of Python programming. However, conation is not to be underestimated. Knowledge is commonly associated with hard skills, whereas active cognition and conation are commonly associated with soft skills. Although knowledge seems more important than the other components, we believe the importance of other components is generally underestimated when considering Python programming as a hard skill, as context matters. This example shows value for such skills that are unfairly considered hard skills with little to no consideration for the potential complexity of the context, or the motivation of the programmer.

To develop the skill of Python programming using the framework of the five components, a pedagogical approach can be designed as follows:

	- a. Begin with teaching the fundamentals of Python, such as data types, variables, control structures, and functions, through a combination of lectures, reading materials, and online resources.
	- b. Introduce more advanced concepts, such as object-oriented programming, error handling, and file I/O, as students progress.
	- c. Teach students about commonly used Python libraries and their applications in various domains.
	- d. Assign small projects or exercises at the end of each topic to reinforce learning.
	- a. Encourage students to practise problem-solving using Python by assigning coding challenges and puzzles that require critical thinking and decision-making.
	- b. Provide opportunities for peer programming, where students collaborate and exchange ideas to solve problems.
	- c. Organise regular code review sessions to help students learn from each other's solutions and improve their problem-solving strategies.
	- a. Set clear expectations and learning goals for students to motivate them to learn and practice Python programming.

#### 4. Affection:

	- a. Teach students about ergonomics and the importance of regular breaks to prevent strain and fatigue while working on a computer.
	- b. Encourage students to engage in physical activities or exercises to maintain overall well-being, which can have a positive impact on their cognitive abilities.

By incorporating these strategies in a Python programming course or training program, learners can develop the required skills while addressing all components of the pedagogical framework.

(C) Example 3: Logical analysis

For the skill "logical analysis", which is ambiguously considered as a soft skill or a hard skill depending on the situation, we describe below on a scale of importance of 1 to 5 for each component, the composition for each component (see Figure 4):


**Figure 4.** Visual representation of the generic skills' components framework for the skill "Logical analysis".

It is interesting to observe that using the framework, it appears that active cognition and knowledge seem to be the most important components for the skill of logical analysis. Knowledge is commonly associated with hard skills, whereas active cognition is commonly associated with soft skills. The dominance of these two components could explain why it seems complicated to categorise logical analysis as a soft or hard skill. This example shows the value of such skills that cannot be consensually categorised.

To develop the skill of logical analysis using the framework based on the five components, a pedagogical approach can be designed as follows:

	- a. Begin by teaching the basic logical concepts, such as premises, conclusions, and logical fallacies.
	- b. Teach various types of logical arguments and structures (e.g., deductive, inductive, and abductive reasoning).
	- c. Provide examples and case studies to illustrate different logical principles and argumentation styles.
	- a. Engage students in debates or discussions to practise identifying and evaluating arguments.
	- b. Provide exercises that require students to identify logical fallacies or errors in reasoning.
	- c. Engage reflection and self-assessment to help students recognise their own biases and assumptions.
	- a. Set clear goals and expectations for students' progress in developing logical analysis skills.
	- b. Provide regular feedback and encouragement to help students stay committed and motivated.
	- c. Create opportunities for students to collaborate and share their learning experiences with peers.
	- a. Teach students how to present their logical analyses effectively and persuasively, while considering the perspectives and emotions of their audience.
	- b. Encourage empathy and active listening during debates and discussions to foster a more open and collaborative learning environment.
	- a. Present information and materials in a clear, visually appealing manner to facilitate understanding.
	- b. Encourage students to take notes or create visual representations (such as diagrams or flowcharts) to help organise and process information.

By addressing each component of the generic framework, this pedagogical approach provides a comprehensive and structured method for developing logical analysis skills in students.

#### **4. Limitations and Opportunities**

Skills have traditionally been defined as a set of competencies or abilities that an individual has, such as problem-solving, analytical thinking, and communication. However, this definition is problematic because it treats skills as discrete entities; this fails to account for the influence of contextual factors on how skills are used in practice. For example, a skill such as communication may be used differently in diverse contexts, with different levels of success. Further, there may be no such thing as a completely "generic" skill—one that functions equally well in all contexts. In short, the idea of skills as abstract entities is a misleading oversimplification.

The definition of skills as abstract entities has a wide range of implications. It ignores the role of context in how skills are applied, which in turn can lead to an over-emphasis on the individual's capabilities and an under-emphasis on environmental conditions (Widdowson 1998). This can lead to a focus on individual differences instead of a collective approach; this in turn can lead to a narrow focus on the individual and an inability to identify external influences on skill use. Further, it can lead to a teleological approach (González Galli et al. 2020), whereby skills are thought to be automatically "transmitted" to the context in which they will be used, without regard to the idiosyncrasies of that context. Finally, it can lead to a focus on skills as an end in themselves, instead of collectively as part of a much larger system.

A systems-based perspective goes beyond the traditional concept of skills as abstract entities and instead focuses on the way in which skills develop within specific contexts, thus treating them not as static entities, but as part of an interactive, evolving system. Through this perspective, the influence of context on skill use is fully acknowledged, with multiple factors—such as culture, power dynamics, and social norms—being taken into account. Therefore, this approach enables the concept of skills to be seen as part of a larger system of behaviour and learning, which is essential to understanding how skills can be effectively developed, practised, and utilised.

Indeed, the scientific literature has challenged the definition of skills as abstract entities and instead advocated for a systems-based approach that acknowledges the role of context in how skills are applied (Tracey et al. 1995; Le Boterf 2000; Sih et al. 2019). However, if skills did not exist, then only knowledge would matter a priori.

Knowledge alone does not lead to successful interactions with others; skill plays an integral role in the development of successful social behaviours (Boyle et al. 2017; Rios et al. 2020). Further, this research indicates that even if a person has a great deal of knowledge, it is not enough to produce the desired results unless they can put the knowledge into practice. Skills need to exist in order to allow professionals, educators, and clinicians to work on isolated and specific constructs, even if variable and not perfect as such. In our contribution, we see the generic components approach as a way to redefine the concept of skill, by embedding environmental factors in cognitive, conative, and affective dimensions.

Although our generic skill framework provides the basis for further developments, it is important to note that other approaches may need to be considered to provide a more comprehensive understanding of the concept of skill in various contexts.

#### **5. Conclusions**

This article has explored the definitions, categories, and impact of both hard and soft skills in order to gain an understanding of the generic composition of any skill. It found that both must be viewed as complementary elements comprising a successful performance and that hard skills are objective and quantifiable capabilities that are easily measured, whereas soft skills are non-technical, interpersonal, and visual qualities that are often learned through experience. Although the two types of skills are often classified separately, understanding their interdependence can help create a more comprehensive skill set. Strategic thinking and action, skills that cut across both soft and hard skills, are essential for making effective decisions.

Research on skills reveals that hard and soft skills often overlap, with various components being shared between them. As such, there is a need to recognise the different components of any skill to develop individuals efficiently and effectively. The generic components proposed in this article open the way to discuss the common ground between hard skills and soft skills, and more broadly the generic composition of any skill. More research is needed to refine the approach on this topic, but it seems a greater understanding of the generic composition of skills can help inform professional, educational, and clinical practices.

**Author Contributions:** The authors were responsible for all aspects of the study, including research, writing and editorial work. Both authors contributed to the conceptualization. The first author drafted the paper. Both authors contributed to the revisions of text. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest or any other ethical considerations with respect to this study.

#### **References**


Dede, Chris. 2010. Comparing frameworks for 21st century skills. *21st Century Skills: Rethinking How Students Learn* 20: 51–76.


Laker, Dennis R., and Jimmy L. Powell. 2011. The differences between hard and soft skills and their relative impact on training transfer. *Human Resource Development Quarterly* 22: 111–22. [CrossRef]

Lamri, Jeremy. 2018. *Les Compétences du 21ème Siècle*. Paris: Dunod.

Le Boterf, Guy. 2000. *Construire les Compétences Individuelles et Collectives*. Paris: Éditions d'organisation.

Levasseur, Robert E. 2013. People skills: Developing soft skills—A change management perspective. *Interfaces* 43: 566–71. [CrossRef]

Lyu, Wenjing, and Jin Liu. 2021. Soft skills, hard skills: What matters most? Evidence from job postings. *Applied Energy* 300: 117307. [CrossRef]

Martins, Helena, Carlos Rouco, Lucia Piedade, and Fernando Borba. 2020. Soft Skills for Hard Times: Developing a Framework of Preparedness for Overcoming Crises Events in Higher Education Students. Paper presented at 17th International Conference on Intellectual Capital, Knowledge Management & Organisational Learning (ICICKM 2020), Online, October 15–16; p. 280.

Matteson, Myriam L., Lorien Anderson, and Cynthia Boyden. 2016. "Soft skills": A Phrase in Search of Meaning. *portal Libraries and the Academy* 16: 71–88. [CrossRef]

Mayer, John D., Richard D. Roberts, and Sigal G. Barsade. 2008. Human abilities: Emotional intelligence. *Annual Review of Psychology* 59: 507–36. [CrossRef]

Merriam-Webster. n.d. Skill. In *Merriam-Webster.com Dictionary*. Available online: https://www.merriam-webster.com/dictionary/skill (accessed on 15 April 2023).

Newell, Karl M. 1991. Motor skill acquisition. *Annual Review of Psychology* 42: 213–37. [CrossRef]

Palumbo, Claudio. 2013. Soft Skills and Job Satisfaction. Two Models in Comparison. *Universal Journal of Psychology* 1: 103–6. [CrossRef]


Rasipuram, Sowmya, and Dinesh B. Jayagopi. 2020. Automatic multimodal assessment of soft skills in social interactions: A review. *Multimedia Tools and Applications* 79: 13037–60. [CrossRef]


Spencer, Lyle M., and Stacy M. Spencer. 1993. *Competence at Work: Models for Superior Performance*. New York: John Wiley & Sons.

Succi, Chiara, and Magali Canovi. 2020. Soft skills to enhance graduate employability: Comparing students and employers' perceptions. *Studies in Higher Education* 45: 1834–47. [CrossRef]

Tate, William. 1995. *Developing Managerial Competence: A Critical Guide to Methods and Materials*. Aldershot: Gower Publishing, Ltd.

Tracey, John B., Scott I. Tannenbaum, and Michael J. Kavanagh. 1995. Applying trained skills on the job: The importance of the work environment. *Journal of Applied Psychology* 80: 239. [CrossRef]

Vallas, Steven P. 1990. The concept of skill: A critical review. *Work and Occupations* 17: 379–98. [CrossRef]

Van der Fels, Irene M. J., Sanne C. M. Te Wierike, Esther Hartman, Marije T. Elferink-Gemser, Joanne Smith, and Chris Visscher. 2015. The relationship between motor skills and cognitive skills in 4–16 year old typically developing children: A systematic review. *Journal of Science and Medicine in Sport* 18: 697–703. [CrossRef]

van der Vleuten, Cees, Valerie van den Eertwegh, and Esther Giroldi. 2019. Assessment of communication skills. *Patient Education and Counseling* 102: 2110–13. [CrossRef]

Vasanthakumari, Sundararajan. 2019. Soft skills and its application in work place. *World Journal of Advanced Research and Reviews* 3: 66–72. [CrossRef]

Widdowson, Henry G. 1998. Skills, abilities, and contexts of reality. *Annual Review of Applied Linguistics* 18: 323–33. [CrossRef]

Wisshak, Susanne, and Sabine Hochholdinger. 2020. Perceived instructional requirements of soft-skills trainers and hard-skills trainers. *Journal of Workplace Learning* 32: 405–16. [CrossRef]

Zagzebski, Linda. 2017. *What Is Knowledge? The Blackwell Guide to Epistemology*. Oxford: Wiley-Blackwell, pp. 92–116. [CrossRef] Zajac, Felix E. 1993. Muscle coordination of movement: A perspective. *Journal of Biomechanics* 26: 109–24. [CrossRef]

**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

## *Article* **The Future of Education as a Creative Ecosystem: A Sociocultural Framework for the Development of Creativity**

**Felipe Zamana**

Laboratoire de Psychologie et d'Ergonomie Appliquées, Université Paris Cité and Univ Gustave Eiffel, 92774 Boulogne-Billancourt, France; fezamana@gmail.com

**Abstract:** This article explores the social and educational impact post COVID-19 on education through the perspective of creativity. This is a reflective and forward-thinking piece of how creativity can transform the future of education. The article is structured into five parts. First, the opportunities and barriers that COVID-19 offers in preparing students for an uncertain future. Second, the recognition of the vital role of creativity in the future. Third, the article discusses the value of creativity in education. Fourth, the teachers' role in stimulating creativity and how its practices can be encountered in 21stcentury education is commented on. Fifth, the last section presents perspectives for the future of education in an uncertain and complex world, introduces the concept of creative ecosystems for education, and summarizes the key points related to the aspects to which education should devote its efforts in the coming years. The article questions if more creativity-focused education is possible in the future and promotes a deep reflection in this particular context for teachers and educational institutions about the topics that need more attention during this time of change.

**Keywords:** creativity; education; future; creative ecosystems

#### **1. Introduction**

Humanity has faced major changes to adapt to as a global civilization due to the latest outbreak of the pandemic virus, COVID-19. These changes have spread across political, economic, and social spheres, having a direct and significant impact, especially in education.

Since World War II, there has been no common event capable of forcing countries to adopt urgent and drastic measures that push educational institutions around the world to suddenly use technological tools to create content and remote learning experiences for students (Arroio 2020; Kaur and Bhatt 2020).

However, even though most of the technological solutions adopted had already been available and had been familiar to us, changing the school environment from a physical classroom to a digital one was a painful change for both teachers and students, showing how much most of the educational systems are unprepared for the future.

However, COVID-19 brought a golden opportunity to reconsider what matters most in education. Creativity plays a decisive role in the significant changes occurring at the moment and in new emerging economies that depend upon these changes, but most schools and universities are not preparing students to be creative and innovative (Wagner 2012). As reported by the World Economic Forum (2018), technologies are enabling new ways of organizing how value is created, and, in this scenario, creativity is among the most critical capabilities to build the future.

Many educational institutions and teachers are researching, evaluating, and implementing various solutions and strategies to strengthen higher education. However, there is not enough focus on changes and methodologies to provide a more creative education. Although there are several research studies on learning and creative teaching strategies, limited research has investigated the relationships between creativity, curriculums, and learning ecosystems using a framework model.

**Citation:** Zamana, Felipe. 2022. The Future of Education as a Creative Ecosystem: A Sociocultural Framework for the Development of Creativity. *Journal of Intelligence* 10: 99. https://doi.org/10.3390/ jintelligence10040099

Received: 7 July 2022 Accepted: 4 November 2022 Published: 10 November 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

#### **2. Preparing the Students for an Uncertain Future**

COVID-19 has become a real threat and has radically changed the world within merely 100 days since the first case (Kaur and Bhatt 2020). The virus has taught even those who had not experienced it so obviously before that human life is vulnerable, nature is unpredictable, and in such a sudden extreme situation, it is hard to trust most institutions since they have reacted in impulsive and random ways (Agnoletto and Queiroz 2020; Azorín 2020). This situation has made many people reconsider the need for a change which cares for both present and future generations, and education has an important role to play in such a change (Wolff 2020).

Nevertheless, the problems in the educational system are long known. Before COVID-19, school systems' detachment from students' needs was already an issue (Robinson 2017). For Illich, schools were first conceived with a highly organized factory structure, where students are "natural resources to be molded by the schools and fed into the industrial machine" (Illich 1972, p. 65). We still can see pieces of evidence of this structure today, as shown by the report, "Schools of the future: Defining new models of education for the Fourth Industrial Revolution" (World Economic Forum 2020a, p. 5):

Many education systems in developed and developing economies still rely heavily on passive forms of learning focused on direct instruction and memorization rather than interactive methods that promote the critical and individual thinking needed in today's innovation-driven economy.

However, the pandemic was a critical moment for most educational institutions. For example, despite the students' social issues, the school dropout rate, especially in higher education, skyrocketed compared to the pre-pandemic years (Nietzel 2021). For Azorín (2020), the central educational problems today also include, among others: the high rates of socioeconomic segregation, school dropouts, and academic failure; the poor culture of networking and collaboration; overcrowded classrooms that hinder quality education; and an obsolete curriculum. Besides these, most high school dropouts name boredom the number one reason they left (Diamandis 2018). From this point of view, teaching must be built to meet the needs of the new generation, but also, in addition to the maturity and experience of teachers and educational institutions, it must bring complementary ways to better prepare students for the future. Thus, this world is not just ours, so it is imperative for our youth to learn that social and collective awareness is indispensable (Usak et al. 2020).

The unprecedented experience of the pandemic showed how collective actions are fundamental to overcoming global challenges, and how every action counts (Arroio 2020). For Beck (1992) and Wolff (2020), humanity needs to be able to think systemically, seeking to anticipate the impact of its actions on multiple levels and contexts, and the ability to prevent these risks depends on access to knowledge and information.

In this sense, researchers are looking into new educational strategies on how we can give technology a better use to overcome former constraints and embrace new learning opportunities, such as the use of e-learning platforms (Wagner and Dintersmith 2016; Oranga and Matere 2022). Moreover, studies have shown that introducing students to future-readiness skills could offer them "[t]he ability to respond flexibly, make in-formed decisions, and adapt to rapid change" (Jalil et al. 2022, p. 1).

The mark of this time is uncertainty, more than ever. The risks in modern society are not distributed according to wealth or social position; they hit the entire society. The coronavirus crises have shown how new alliances of decision making can emerge, which means that the present time is the time to act and make a change.

In brief, there is no educational knowledge or skills that present a definitive solution to face a constantly changing educational or professional environment. This demands constant research and rethinking education to prepare students for the future, especially in a post-digital world. Therefore, teaching these students how to think creatively seems to be a good way to equip them with the capacity and potential to navigate uncertainty and change.

#### **3. Expectations for the Future of Education**

There is a unique opportunity to rethink what the future of education will look like. The situation caused by COVID-19 also surfaced the possibility to create a better education for all, focusing on the student's well-being and needs (Fullan and Gallagher 2020). However, Azorín (2020) and Hargreaves (2020) question if this pedagogical renewal will arrive soon or if it will slowly go back to the same old way of schooling.

Today, creative performance does not seem to be a priority in schools, although the development of student creativity is crucial for economic, scientific, social, artistic, and cultural advancement (Amabile 2012; Cropley 2012; Garcês et al. 2016; Runco et al. 2016; Richardson and Mishra 2017). For Alencar, few students are capable of being inventive and original, and "it is necessary to prepare the student to solve problems that have not yet arisen—and this is only possible through stimulating creativity" (Alencar 2016, p. 18).

For example, with the emergence of new technologies such as artificial intelligence, more specifically machine learning and deep learning, the future of education can expect a blended model where machines will play a bigger role in schools (Mijwil et al. 2022). We are witnessing the rise of digital learning platforms such as Google Classroom and even YouTube, but also huge platforms like Coursera where companies and universities can offer training. According to Sushama et al. (2022), these platforms provide a type of service that saves time (e.g., no need to go physically to a classroom), reduces bureaucracy, and provides the learner with real-time feedback on the learning process.

The COVID-19 crisis also offers us an excellent opportunity to build the right motivation that directly influences characteristics such as curiosity, willingness to take risks, tolerance, dedication, energy, concentration, and fascination with the task, which are fundamental for creative thinking (Morais and Almeida 2019).

Morris et al. (2022) analyzed different global agencies' reports in response to the educational challenges for the post-COVID-19 era, where most of them emphasized the need for a model or framework that can anticipate, optimize, and enhance the role of collaboration between the private sector and governments. Moreover, according to an analysis by McKinsey & Company (Craven et al. 2022):


Therefore, it is necessary that the educational systems be restructured around disciplines that involve more collaboration groups, improvisation, and creative processes, where the student can better respond to the needs and challenges of the current world. For this, a structure is needed that allows the possibility of reviewing and re-evaluating all situations and, if necessary, the courage to change things or keep them as they are.

#### **4. Education as Creative Ecosystems**

New emerging economies are creating and being fueled by major changes, such as the creative economy (see Howkins 2013), and so it is crucial to understand how education can prepare students for it and how creativity can be stimulated to aid them to adapt to these changes (Kadushin 2012; Simonton 2019). For example, good teachers know how important it is to have long-term planning for the student's learning success; to achieve the main objective, communicating the content involves motivating and engaging each student in the teaching and learning process (Agnoletto and Queiroz 2020).

However, with the chaos caused by the COVID-19 pandemic, most educational institutions adopted digital technologies only to ensure that teachers have covered all the topics in the syllabus by the end of the academic year and to "save" the educational programs (Agnoletto and Queiroz 2020; Wan 2020). For Azorín (2020), remote learning is showing very clearly that a significant number of our teachers do not have adequate digital competencies, which leads the teachers to try to replicate the "classroom model" online, ignoring their different approaches. Besides that, the logic adopted is still "one size fits all", ignoring that this "all" is represented by different individuals with their own pace, stories, needs, evolutions, and times.

The context of the pandemic highlights the recognition that education only makes sense if it is anchored in universal values such as human rights, empathy, and solidarity (Arroio 2020). The role of education has often been the rescuer, with a mission to change both individuals and society, but neither educators nor schools are free agents in society. With new technologies, the planet has become a learning space, and the student's profile is entirely different from what it was in the past. In this sense, we can understand education as the integration of groups, social or cultural, where each member contributes to transforming it (Wagner 2012; Runco et al. 2016).

Education is part of an ecosystem; that is, it is made up of subunits—like buildings, classrooms, teachers, and students—and it may itself be the subunit of some broader collectives—such as communities, neighborhoods, and cities—and the dynamic interactions between them (Kauffman 1993; Harrington 2011). In this sense, education is about how people interact and grow together. So, if major characteristics that make us human are interpersonal relationships and the ability to create, creative ecosystems are human relationships directed towards creation—of knowledge, ideas, citizens, and societies (Morin 1999; Amabile 2012; Zamana 2021).

All societies share the same benevolent core principles, most of them essential for fostering creativity (Amabile 2012; Turner and Pennington 2015; Kauffman 2016; Christakis 2019). Creative ecosystems offer agility but integration on larger scales, emphasizing the skills of strategic empathy, collaborative leadership, and communication that rely on their members' strength and creative capacity.

Creativity is present at different scales, from the individual to the social level; but independently of how it manifests, it needs some degree of recognition by the collective (Plucker et al. 2004; Glăveanu 2018; Glăveanu et al. 2019). So, in this perspective, creativity is never an individual act but a systemic interaction between the student and their sociocultural environment. For Glăveanu (2010), creativity cannot be detached from our historical and cultural contexts, especially when we became more aware of our social influences and started to emphasize creative collaboration and co-creation. For collaboration to happen, adaptation should be an indispensable criterion for creativity's analysis, once the individual, as a member of the creative ecosystem, needs to be able to fully engage with its culture and environment (Cohen 2012; Runco 2017; Reeves 2019).

However, individuals can also create the conditions to shape the environment according to its needs and desires or abandon this environment to pursue another more favorable one for the development of their skills and interests. In this sense, learning is related to a culture of shared values and beliefs, which are built together (Fleith 2019). So, the environment should provide opportunities for individuals to develop their capacities but also recognize and encourage them during the process.

The development of students' creative thinking enables them to have the necessary tools to seek knowledge and learn on their own in the future (Alquatahni 2016). The main objective of stimulating creativity in the classroom is to meet the demands of modern life, allowing the student to take advantage of their development opportunities (Amabile 2012; Alquatahni 2016; Morais and Almeida 2019). Therefore, education must continually renew

itself and constantly look for new ways of teaching, focusing on how students learn and how they can appreciate their creativity, investing in the training of students who are able to fully enjoy their creative potential. For Alencar (2016, p. 7):

The awareness that it is necessary to invest in the training of individuals able to make full use of their creative potential has grown exponentially, and education institutions should prepare students for an uncertain future in a complex society marked by numerous challenges and demands.

By enriching the way students experience the world, they will be better prepared for the future, considering their cultural context and adapting to it (Cohen 2012; Runco 2017). Adapting to the context allows the individual to be tolerant of the world's uncertainties or ambiguities, to accept not always having the answers, to be wrong, and to try new alternatives (Robinson 2017). In this case, educational institutions should provide students with a way to recognize their strategies and styles of working and thinking, as well as different forms of learning.

There is a need for something that facilitates the teaching of creativity and its learning methods, which are transversal to all areas and ages. Usually, we are taught to separate, compartmentalize, and isolate learning instead of making connections, and, consequently, our knowledge can become an "unintelligible puzzle lost between different disciplines" (Morin 1999, p. 17). This receptivity to new ideas and experimentation demands that we learn to consider and often look for ways to challenge current beliefs. The more we discover, the more we understand how incomplete our knowledge is, and with creative ecosystems it is possible to expand interest in the new, propose challenging goals, remain open to new experiences, and, above all, collaborate to build a better future together.

#### **5. The Teachers' Role in Stimulating Creativity**

Much is said about stimulating the student's creative potential, but the need to prepare the teacher for this development is forgotten. Like the student, the teacher also needs to study and prepare to be able to teach creatively, developing his teaching techniques and creativity, which will directly influence the quality of student learning (Mullet et al. 2016).

The COVID-19 situation provided most educators with the possibility to create a new way, or at least a different one, of teaching, which represents a chance to rescue the true value of education (Arroio 2020). The teacher should encourage openness but eliminate possible blocks, as the teacher has the vital role of intermediating the learning process through clues, guidance, and rectification. In this sense, to (re)think about education and the role of the teacher in the future to come, three main challenges need to be addressed:

**Less focus on outcomes and more focus on process.** The development of creative maturity involves both an external transformation of a specific field and an internal transformation of the self, which involves openness and willingness to change the current way of thinking for a new perspective (Alencar 2016; McCarthy and Blake 2017). However, the excessive focus on the outcomes leads to an overrated concern with practical knowledge. For example, most teachers have already heard students ask such questions as, "is it going to be on the test?" (Sharma and Scherrer 2014). In this logic, it is more about what the student decides to learn than what we want to teach (Pacheco 2014; Robinson 2017). Specific themes and approaches may seem more relevant for teachers, who are usually aware of the process's importance, but are these themes and approaches relevant to the student?

Thus, the role of the teacher is taking a different path, as the students need more guidance through their learning process. A permissive educational environment can facilitate students' creativity, but they must have the teacher's attention and support to be aware of their potential (Runco et al. 2016). Through these interactions with the teacher, students internalize the problem-solving and learning processes, which will make them better prepared to deal with problems and challenges in the future (Hargreaves 2020). The development of creative maturity helps the students adapt to the environment, fit their plans and theories, and adjust to this new environment (Alencar 2016). Over time, as the teacher develops the student's ability to generalize and transfer what they have learned, the teacher starts to encourage more than guide. When the students see meaning in what they are learning, it is easier to stimulate curiosity and encourage them to explore new perspectives, offering them the freedom to make their own decisions (Kaufman 2018).

**Less focus on the individual and more focus on the collective.** The World Economic Forum (2020b) listed social and emotional skills among the "most wanted" for the future of work, such as collaboration and teamwork. However, besides these skills not being taught at schools, students are penalized for asking questions, sharing their thoughts, and helping their peers in the classroom (Robinson 2017). What is more, we still hope that these same students become good citizens and participate actively in our society.

Moreover, most educational alternatives have proposed to converge toward the idea that individual needs are only met through specialization once we believe that our world is knowable, optimizable, and controllable, and this belief leads to overconfidence and dependence on specialists (Illich 1972; Kauffman 2016; Wolff 2020). Education urgently needs openness to individual choices, understanding how they impact the collective and, vice versa, teaching how we can build more meaningful relationships through solidarity, empathy, and respect (Arroio 2020).

For example, if promoted healthily, competition can favor teamwork and even facilitate understanding of various topics addressed in the classroom (Tang and Werner 2017; Morais and Almeida 2019). For this, education must take into account the students' reality and experiences, creating spaces that favor sharing with a loose and low-supervised structure for learning (Pacheco 2014; Zamana 2021). Why not give the students the opportunity—and responsibility—to learn with each other?

**Less focus on the now (short term) and more on the future (long term).** If education is one of the pillars of any society, we will increasingly depend on decentralized and shared knowledge built by many hands (Kadushin 2012; Christakis 2019; Zamana 2021). Moreover, we should be cautious that the students do not become limited by our own views of the world. For example, the millennial generation, the latest to finish college and join the workforce, are considered the "burnout generation", with high cases of depression and financial problems (Petersen 2020). So, it is hard not to wonder: what happened during their education years for them to have such a troublesome future?

Thus, there is no way to measure learning with tools focusing only on the immediate, specifically memorization models. From this point of view, standards and assessments need to be developed and not be limited to measuring whether students can complete a test, but if they have 21st-century skills such as communication, critical thinking, problem solving, entrepreneurship, collaboration, and, mainly, creativity. The classroom is just a device for learning to happen, but it is not always the best device. Education must stop being centered on traditional teaching in order to be centered on the relationship between individuals. As mentioned before, learning happens through collaboration, one of the most valued requirements for the new education (DaVia et al. 2018; Azorín 2020). The time has come for the educational systems to face the undeniable consequences of being out of date and provide the necessary changes for students to enjoy learning.

#### **6. Conclusions**

Creativity in education has become essential in modern society, and understanding its past not only allows us to understand its future but also to broaden our horizons and to see that our current situation is not constant and immutable since we have many more possibilities ahead of us than we imagine (Morin 1999; Harari 2017; Osmond-Johnson et al. 2020). In short, if there were ever an opportunity to draw on the talents and strengths of the collective capacity, that time is now.

To belong to that future, the individual must be creative and charismatic and able to recognize patterns and create meaning. If the students leave school without knowing how to create and innovate continually, they will be unprepared for society's challenges. The development of this understanding is a task for the education of the future, with a fairer and more supportive educational system that can change lives. Otherwise, we will have missed out on this opportunity.

Creative ecosystems can represent a solution to educational and social challenges once local governments gradually embrace them (e.g., The Great Reset Initiative from the World Economic Forum). These purpose-driven ecosystems aim at solving significant social challenges and are also snowballing in importance, driving greater involvement of the public and not-for-profit sectors. Creative ecosystems can offer an agile and collaborative framework for a shift in education, providing the conditions for the development of social and emotional skills, such as leadership, strategic empathy, and communication.

Creative ecosystems will undoubtedly play a more significant role in the continuing growth of most societies. Engaging with creative ecosystems can provide education the freedom it needs to finally become a space for sharing knowledge, embracing change, fostering relationships, and building desirable futures (Turner and Pennington 2015; Bourgeois-Bougrine et al. 2020). Because of their unpredictability and spontaneity, creative ecosystems offer a greater idea yield from the assembled brains than traditional approaches. Hence, there are gaps for new research opportunities. Finally, with world society facing many new challenges (social, ecological, economic, etc.), perhaps creative ecosystems can help education in preparing students in this regard.

The pandemic situation due to COVID-19 is a wake-up call for education. Before this crisis, it was common to hear about the educational changes needed for the 21st century, but now we see how urgent these changes are. It is not the first time we have faced a pandemic situation, but it seems that we still cannot deal with unforeseen problems properly and act fast to solve them. The main reason is not that we are not capable of thinking creatively but that we are not taught to think that way; in fact, the worst thing you can do in school is to act creatively (Robinson 2017).

Maybe the entire idea of education needs a transformation. Educational institutions should make it possible to question the status quo, offer a safe space to share ideas, build trust and friendships, give up the need for control and stability to make room for experimentation, and emphasize empathy, collaboration, and communication. Changes, even radical ones, bring new beginnings.

Education needs to raise doubts and provoke the students to formulate good questions and not narrow down the student to the "right answer" because an unpredictable future is a "what if?" future. Therefore, we must prepare ourselves for the changes in the best way possible, and education should inspire the students of today and tomorrow to create new ways to make this world a better place for everyone.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

Agnoletto, Rosângela, and Vera Queiroz. 2020. COVID-19 and the challenges in Education. *Centro de Estudos Sociedade e Tecnologia* 5: 1–2.

Alencar, Eunice. 2016. *Como Desenvolver o Potencial Criador*. Rio de Janeiro: Editora Vozes.

Alquatahni, Abdulaziz. 2016. Do Saudi EFL Teachers Promote Creativity in Their Classrooms? *English Language Teaching* 9: 11–23.

Amabile, Teresa. 2012. Componential Theory of Creativity. In *Encyclopedia of Management Theory*. Edited by Eric Kessler. Nova York: Sage Publications, pp. 134–39.

Arroio, Agnaldo. 2020. The value of Education in the context of the covid-19 pandemic. *Problems of Education in the 21st Century* 78: 309–13. [CrossRef]

Azorín, Cecilia. 2020. Beyond COVID-19 supernova. Is another education coming? *Journal of Professional Capital and Community* 5: 381–90. [CrossRef]

Beck, Ulrich. 1992. *Risk Society: Towards a New Modernity*. California: Sage.

Bourgeois-Bougrine, Samira, Peter Richard, Jean-Marie Burkhardt, Benjamin Frantz, and Todd Lubart. 2020. The Expression of Users' Creative Potential in Virtual and Real Environments: An Exploratory Study. *Creativity Research Journal* 32: 55–65. [CrossRef]

Christakis, Nicholas. 2019. *Blueprint: The Evolutionary Origins of a Good Society*. New York: Little, Brown Spark.

Cohen, Leonora. 2012. Adaptation and creativity in cultural context. *Revista de Psicologia* 30: 5–18. [CrossRef]

Craven, Matt., Mark Staples, and Matt Wilson. 2022. Ten Lessons from the First Two Years of COVID-19. Available online: https://www. mckinsey.com/industries/healthcare-systems-and-services/our-insights/ten-lessons-from-the-first-two-years-of-covid-19 (accessed on 20 July 2022).

Cropley, Arthur. 2012. *Creativity in Education and Learning: A Guide for Teachers and Educators*. Londres: Routledge.

DaVia, Lisa, Lisa Ridgley, Gregory Callan, Sareh Karami, and John Ehlinger. 2018. How teachers perceive factors that influence


Glăveanu, Vlad. 2018. Creativity in and for Society. *Creativity: Theories—Research—Applications* 5: 155–58. [CrossRef]


Illich, Ivan. 1972. *Deschooling Society*. New York: Harper & Row.

Jalil, Habibah, Ismi Ismail, Aina Ma'rof, Chee Lim, Nurhanim Hassan, and Nur Nawi. 2022. Predicting Learners' Agility and Readiness for Future Learning Ecosystem. *Education Sciences* 12: 680. [CrossRef]

Kadushin, Charles. 2012. *Understanding Social Networks: Theories, Concepts, and Findings*. New York: Oxford University Press.

Kauffman, Stuart. 1993. *Origins of Order: Self-Organization and Selection in Evolution*. Oxford: Oxford University Press.


McCarthy, Cheryl, and Sally Blake. 2017. Is This Going to be on the Test? No Child Left Creative. *SRATE Journal* 26: 25–31.


Pacheco, José. 2014. *Aprender em Comunidade*. São Paulo: SM.

Petersen, A. 2020. *Can't Even: How Millennials Became the Burnout Generation*. Boston: Houghton Mifflin Hartcourt.


## *Article* **The Creativity Diamond—A Framework to Aid Creativity**

**Peter Childs 1,\*, Ji Han 2, Liuqing Chen 3, Pingfei Jiang 4, Pan Wang 5, Dongmyung Park 6, Yuan Yin 1, Elena Dieckmann <sup>1</sup> and Ignacio Vilanova <sup>1</sup>**


**Abstract:** There are many facets to creativity, and the topic has a profound impact on society. Substantial and sustained study on creativity has been undertaken, and much is now known about the fundamentals and how creativity can be augmented. To draw these elements together, a framework was developed called the creativity diamond, formulated on the basis of reviews of prior work, as well as the consideration of 20 PhD studies on the topics of creativity, design, innovation, and product development. The framework embodies the principles that quantity of ideas breeds quality through selection, and that a range of creativity tools can provoke additional ideas to augment our innate creativity. The creativity diamond proposed is a tool consisting of a divergent phase associated with the development of many distinctive ideas and a convergent phase associated with the refinement of ideas. The creativity diamond framework can be used to prompt and help select which tool or approach to use in a creative environment for innovative tasks. The framework has now been used by many students and professionals in diverse contexts.

**Keywords:** creativity; creativity process; creativity tools; thinking; idea generation and evaluation

#### **1. Introduction**

The term creativity is loosely used in society to describe a wide range of outcomes ranging from ways of playing, dancing, painting, and making music to exploring and experimenting in science and technology. There is evidence of creativity for as long as our history records extend back, ranging from ancient cave paintings and tools to contemporary music and new medical procedures. There have been many definitions of creativity in literature and academia (see Morgan 1953; Welsch 1981; Kampylis and Valtanen 2010; Elliot and Nakata 2013; Yin et al. 2021; Abraham 2022). For example, creativity can be defined as the forming of associative elements into new combinations which either meet requirements or are in some way useful. Conversely, creativity denotes a person's capacity to produce new or original ideas, insights, inventions, or artistic products which are accepted by experts as being of scientific, aesthetic, social, or technical value. A definition with a modern twist on values is that creativity is imagination with responsibility (see Sae Ra Kung 2009).

We are familiar with the years of toil that can lead to a burst of new knowledge that sets a domain that others then occupy. The study of creativity reveals patterns to this type of creative burst. Creativity is often thought to exist on at least two levels, big C versus little c, eminent versus every day (Boden 2004). We can view creativity in terms of brilliance, personal creativity, paradigm or domain creativity, and forced or industrial creativity. This thinking on the significance of creative contribution was further extended by Kaufman and Beghetto (2009) to mini, little, pro, and big C creativity.

**Citation:** Childs, Peter, Ji Han, Liuqing Chen, Pingfei Jiang, Pan Wang, Dongmyung Park, Yuan Yin, Elena Dieckmann, and Ignacio Vilanova. 2022. The Creativity Diamond—A Framework to Aid Creativity. *Journal of Intelligence* 10: 73. https://doi.org/10.3390/ jintelligence10040073

Received: 16 August 2022 Accepted: 14 September 2022 Published: 22 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

<sup>1</sup> Dyson School of Design Engineering, Imperial College London, London SW7 2AZ, UK

There are various historical periods associated with significant creative activity and development of our understanding (Sawyer 2011). In ancient Greece, patrons would support individual artists to creatively reflect the patron's status. For example, the bankers' guild of Florence commissioned a bronze statue of St. Matthew to decorate the church of Orsanmichele during the Renaissance. The Industrial Revolution can be characterized as leveraging the advantages of production and concentration of resource. Twentieth century contributions to creativity include increased understanding emerging from domains such as psychology and neuroscience. The second and third decades of the 21st century have seen great strides in automated and augmented creativity through data mining and artificial intelligence (AI). A modern-day analogy is the online platform Patreon which is a crowdbased service designed to support creators. A repeated insight from the study of creativity is the value of patronage, investment, and support for an activity. From individuals to institutions to nations and states, patronage pays for costly rigs, experiments, research and support teams. It enables innovation and the realization of an idea.

Research on creativity has addressed many areas of the topic, particularly in the years following JP Guildford's call for attention to the topic in 1950 (Guilford 1950), with a focus on education, design, development, domain specificity, process, culture, traits, artificial intelligence, physiology, and neuroscience. In design and product development, creativity plays a vital role, especially during the early stages (Han et al. 2018a). Creativity is often considered a prerequisite for product innovation, ultimately leading to product success, which involves divergent and convergent thinking. The divergent phase of creativity is associated with the generation of many distinctive ideas, while the convergent phase is associated with the evaluation and refinement of ideas.

A number of approaches and tools have been developed for supporting idea generation as a part of the convergent phase of creativity, such as brainstorming (Osborn 1963), morphological analysis (Zwicky 1969), Method 6–3–5 (Rhorbach 1969), the Gallery Method (Pahl and Beitz 1996), SCAMPER (Eberle 1996), and C-Sketch (Shah et al. 2001). A recent study investigated the explorations of new problems in the divergent phase (Obieke et al. 2021). During the convergent phase, the consensual assessment technique (CAT), proposed by Amabile (1983), is often considered the gold standard for assessing creativity. Shah et al. (2003) proposed four metrics, namely, novelty, variety, quality, and quantity, for evaluating ideation effectiveness, while Fiorineschi et al. (Fiorineschi et al. 2022) refined the metrics to extend their applicability. Sarkar and Chakrabarti (2011) employed novelty and usefulness for measuring creativity, while Jagtap (2019) refined the novelty assessment method to better assess product novelty. Han et al. (2021) indicated that evaluating creativity should involve novelty, usefulness, and surprise. However, few studies to date have explored the divergent phase and the convergent phase of creativity as a whole. Therefore, a framework that considers both divergent and convergent thinking is needed to better support creativity.

Research within the groups associated with the principal author of this paper has resulted in 20 PhD theses of relevance to the broad topic of ideation. A common thread in the PhD studies concerned was a contribution toward an understanding of a fundamental facet of creativity with a view to enhancing creative practice, which lies in the field of creativity, design, innovation, and product development. These PhD studies covered a significant range of topics in creativity, including work on the systems model of creativity and creative engines (Wang 2013; Jiang 2015; Lee 2015), functional attributes (Tsai 2008; Michalakoudis 2019; Ekong 2014), creativity tools (Garvey 2016; Mi 2019), product creativity assessment (Hazeri 2019), individual differences (Yan 2017), combinational creativity (Han 2018), AI augmentations (Shi 2018; Chen 2020; DeWulf 2020), linkages among creativity, design, and innovation (Baxter 2017; Park 2018; Sikhwal 2020), innovative cultures (Grønneberg 2019; Garcia-Herrera 2020), and neurocognition (Wang 2021). The studies offered a comprehensive and extensive understanding of both divergent and convergent phases in creativity, whereas most other academic publications, such as journal and conference articles, only focused on specific topics or aspects of creativity.

The intent to enhance creativity is embedded in the multitude of good practice guidance on the topic and the many creativity tools. Therefore, on the basis of a review of the 20 PhD theses, along with the authors' diverse experiences in design education (Childs 2019a), this research paper is aimed at proposing a new framework, consisting of a divergent phase for developing ideas and a convergent phase for refining ideas, to support users in selecting appropriate tools and approaches for innovative tasks in a creative environment. This new framework leverages commonly acclaimed approaches to creativity along with insights arising from the review, followed by experience in the use of the framework so far.

#### **2. Creativity Process**

In order to develop the new creativity framework, along with a review of the 20 PhD theses, this paper first reviews creativity process models to investigate the features and facets of creativity in this section. Creativity models that have been promoted for more than 10 years with the latest version having been evaluated by someone other than the promoters are reviewed. Human cognition is used to process information to achieve the creative process (Sweller 2009), and cognition is, therefore, also included in the review. Then, the paper presents some principal creativity tools and some of the common frameworks for creativity in the subsequent sections to form the basis of the newly proposed creativity framework.

Rhodes (1961) in his article "an analysis of creativity" portrayed four strands (person, process, press, and products), which, when combined, provide the functionality we refer to as creativity. These topics are expanded on in this section as they provide a basis for some of the development in the understanding of creativity.

Person in this context refers to our personality, intellect, temperament, traits, habits, attitudes, values, skills, and behavior. Process refers to the mental activities that occur during thinking. Wallas (1926) regarded creativity as comprising four stages:


Press is used to describe the relationship between a person and their environment that influences and defines behaviors, development, and outcome. Rhodes used the term product to refer to the outcome of an idea being embodied in physical form. Today, we are used to services and systems that may not have a physical form; thus, the term product can be more broadly defined to refer to the final realization or manifestation of an idea, be it in the form of a physical artifact or nonphysical, perhaps in some form of digital media.

The use of the 4Ps, person, process, press, and product, to describe the various facets of creativity has been useful for many decades. Recent advances in neuroscience have suggested the addition of physiology, i.e., the way the brain works, as a further facet, as put forward by Abraham (2018). Sternberg and Karami (2022) further expanded consideration of creativity to purpose, press, person, problem, process, product, propulsion, and public, to provide a more complete treatment of the subject. A recent study by Girn et al. (2020) proposed a new framework to capture the complexity of dynamic shifts in neurocognitive states and the impact on creativity. Further research has been conducted to

define neurocognitive constituents of creative thinking by utilizing neuroimaging during creative tasks to reveal the physiological impact of the prompts (Kenett et al. 2020).

Sustained attempts to describe and formalize the processes associated with creativity have been undertaken (Howard et al. 2008) to further develop our understanding of creativity. Some research has taken the view that creativity is a cognitive process (Miller 2014) related to distinct results (Abraham 2013), and that creativity can be achieved through creative idea generation and convergent thinking processes. The former is mainly based on divergent thinking and finding different creative solutions to problems, while the latter mainly concerns insights to problem solving (Benedek and Fink 2019; Lee and Ostwald 2022). Typically, creative ideas are generated on the basis of a combination of the two processes.

The Cognitive Processes Associated with Creativity (CPAC) scale has a basis in the four-stage model originally developed by Wallas (1926) with phases of brainstorming, perspective taking, metaphorical and analogical thinking, incubation, imagery/sensory, and various subprocesses, as illustrated in Figure 1 (Miller 2014). Jacquet et al. (2020) reclassified the six subprocesses and provided further detailed explanations for each. Ideageneration (often referred to as brainstorming) is a process that attempts to generate as many potential responses or solutions as possible, regardless of the plausibility. Idea manipulation (perspective taking) is an intentional process that attempts to transform perception and perspective, allowing the individual to conceptualize or understand the situation in a different way. Metaphorical and analogical thinking is a process that produces a connection between the current problem and a similar or related situation, and then takes ideas from one context and applies them in a new setting. Incubation is a mental process where the person is engaged in other activities and generates ideas unconsciously. Imagery/sensory process is essential to internal visualization, which is a key element of the creative process. This process triggers creative ideas from the senses from different sensory modalities (such as auditory and tactile). Flow (Csikszentmihalyi [1996] 2019) is an almost automatic and highly focused state of consciousness process that occurs when an individual is engaged in intense work.

**Figure 1.** Schematic illustrating the basis for the Cognitive Processes Associated with Creativity (CPAC) scale.

Basadur and Gelade (2005) promoted a differing four-stage cognitive process of creativity with stages of generating, conceptualizing, optimizing, and implementing. These stages are the combination of two factors: apprehension and utilization. Apprehension concerns the acquisition of understanding of knowledge through physical experience of information or mental processing. Utilization concerns applying understanding of knowledge to evaluate ideas or generate creative ideas (Figure 2). Basadur and Gelade's model is a special cognitive process, combining the definition of problems and generating ideas into one stage, while, in other models, this is a single process where problems are defined and then ideas are generated.

In addition to the four-stage models, other principal models developed to describe creativity including the dual-process models, tripartite models, and tripartite-process models. Dual-process theory indicates that a cognitive thinking process involves two subprocesses: (i) autonomous idea generation; (ii) working memory-dependent idea revision, evaluation, and selection (Gabora et al. 2014).

The idea generation subprocess is considered to be an autonomous, rapid, nonconscious, and automatic process that enables associations through nature, gut reaction, and intuition (Leschziner and Brett 2019). Information residing in long-term memory is rapidly combined or associated with current-context information without effortful thinking and intervention. The idea revision, evaluation, and selection process is a rule-based, slow, controlled, effortful, conscious, and analytic process based on working memory (Leschziner and Brett 2019).

Figure 3 depicts the Genoplore model, referring to the dual processes as generation and exploration (Finke et al. 1992). The generation process in the Genoplore model is a divergent thinking process which involves searching long-term memory, forming associations, synthesis, and transforming items. The exploration process is a convergent thinking process that involves considering potential functions in different situations.

**Figure 3.** The Genoplore model.

Figure 4 shows Gabora's (2010) model, which considers that, in the ideageneration process, people autonomously associate highly and remotely relative items from memory on the basis of stimulation (flat association), select ideas on the basis of individual characteristics and current conception, refine the selected idea, and form connections with task demands through associative and analytic thinking. Flat association is stimulated by current content.

#### **Figure 4.** Gabora's model.

Figure 5 shows Howard-Jones's (2002) model, which in essence is a summary of dualprocess models. The model is based on association and divides the creative process into generative and nongenerative/analytical processes. The model highlights the importance of investigation, idea generation, self-evaluation, idea selection, design development, and outcome evaluation.

**Figure 5.** Howard-Jones's model.

The structure of intellect model (Guilford 1956), shown in Figure 6, can be regarded as a dual-process model with divergent and convergent thinking processes. The divergent thinking process is an associated process where in distraction state, the encoded information is combined with information in current context (Gabora 2010). The convergent process, in contrast, is an analyzing process and is important in detailing and evaluating ideas. In this model, Guilford supports the view that the divergent thinking process happens before the convergent thinking process.

**Figure 6.** Structure of intellect model.

The ideation–evaluation cycle (Basadur et al. 1982) shown in Figure 7 divides the creative thinking process into three stages (problem finding, problem solving, and solution implementation). This model identifies that ideation and evaluation are involved in each stage to different degrees. This indicates that the dual process is repeated in different stages of creativity. However, this repeat theory is challenged. Self-report measurements, such as CPAC, have indicated that the cognitive process of creativity is based on "ideational fluency" where as many ideas as possible are generated, "metaphorical and analogical thinking" where ideas are applied from existing context to new content or combined, "perspective thinking" where people try to generate a specific solution for current problems, and "imagery" which is internal visualization (Miller 2014) (see Figure 1). This principle of CPAC indicates that creativity process is not a cyclic process. However, researchers for various of the creative process models promoted do not identify in which stage incubation and flow exist. According to the structure of intellect model, where incubation is promoted as part of cycles in the model, the incubation sub-process may repeat in the four stages. Whether the process is also cyclic is not certain because autonomous and conscious are contradictory to some degree.

**Figure 7.** The ideation–evaluation cycle.

Nijstad et al.'s (2010) model, shown in Figure 8, presented the cognitive process in creativity on the basis of whether this creative process is conscious or unconscious, supporting

the view that the creativity process is attention controlled and ideas are generated consciously.

Despite being at the core for understanding the cognitive process of creativity, the dual-process theory has limitations. There is no clear boundary between idea generation (or divergent thinking process or flexibility pathway) and idea evaluation (or convergent thinking or persistence pathway). In the idea generation process, evaluating ideas is also needed to select the original ideas (Basadur et al. 1982; Campbell 1960). The two processes intervene with each other (Gabora et al. 2014). Researchers have acknowledged the existence of "intervention". For example, Nijstad et al. (2010) explained this intervention as a shifting ability. The tripartite-process model, shown in Figure 9, separated out a new process from the dual-process theory. In the third process, how people deal with minds and behaviors is monitored and managed, influenced by individual personality and thinking preference, which is called cognitive style (Leschziner and Brett 2019). This tripartite-process model has a limitation as it indicates that the creativity process is an attention-controlled process similar to Nijstad et al.'s model. Therefore, sudden ideas generated from the creativity process are ignored.

**Figure 9.** Tripartite-process model.

Bhattacharya and Petsche's (2005) model focused on which kind of memory was involved in the mental imagery idea generation process. Specifically, the model summarized the mental imagery idea generation process as three subprocesses: long-term memory to remember more patterns; reliance on visual memory or short-term memory to maintain; generation of possible graphic and active visual memory.

The models reviewed in this section can be divided into four principal types: (i) fourstage models, (ii) dual-process models, (ii) tripartite models, and (iv) tripartite-process models. The four-stage models and dual-process models play a core role in understanding the creativity process from cognitive aspects and connect creativity and consciousness. Some of these models are linear models which mainly include stages such as preparation, incubation, insight, verification, evaluation, and elaboration (Amabile 2012). Nearly all the models indicate that each process can be further detailed as more cognitive operations

and cognitive factors (such as memory, association, and combination). Some common important features and facets in creativity mentioned in the diverse models include phasing, incubation, perspective, use of analogy and metaphor, association, exploration, stimulation, convergence and divergence, sequencing, attention, and evaluation. These facets of creativity are often implicitly or explicitly incorporated into process models for creativity and tools for enhancing creativity as considered in Section 3.

#### **3. Creativity Tools**

Creativity tools is a phrase often used to describe an approach offering enhanced generative outcomes. There are many creativity tools that are widely used, such as analogy, boundary shifting, various types of brainstorming, and checklists. Designers often employ these tools to facilitate the generation of creative ideas during early stages in design and product development. There are indeed hundreds of creativity tools available, most of which try to enhance (i) fluency—the quantity of responses, (ii) flexibility—ideas that are distinct from each other, or (iii) originality—the level of uniqueness of the ideas generated.

A creativity tool does not produce ideas. Instead, it can be used to assist in the generative process. Most tools can actually be used at any stage in a problem-solving process, but tend to mainly be focused on problem exploration, idea generation, and concept evaluation. Creativity tools generally function by (Childs and Fountain 2011) (i) ensuring that the problem can be understood in relatively simple terms so that this occupies only a fraction of your short-term memory, (ii) supplying cues to make the search of long-term memory more efficient, and (iii) providing cues to ensure refreshing of shortterm memory and, thus, retention of key information.

There are a wide range of types of creativity tools including systematic or randomization approaches, the use of analogy and metaphor, and a series of brainstorming tools such as list, sticky note, alphabet, grid, and brainwriting. Examples of systematic tools are morphological analysis (Childs 2019b), TRIZ (the theory of inventive problem solving) including tools such as the contradiction matrix, the principle of inventions, Smart Little People, OTSM-TRIZ (General Theory of Powerful Thinking), and Size–Time–Cost Operator, along with tool selection guidelines (Moehrle 2005; Chechurin and Borgianni 2016; Fiorineschi et al. 2018), and SCAMPER (an acronym for the provocations substituting, combining, adapting, modifying, putting to another use, erasing, and rearranging). There is a growing number of computational creativity tools, such as Idea-Inspire 4.0 which can support analogical design employing a searchable biological systems knowledge base (Siddharth and Chakrabarti 2018), the Combinator which produces combinational textual and pictorial stimuli to promote users' creative minds (Han et al. 2018b), and InnoGPS which retrieves design concepts from a multi-technology domain patent database (Luo et al. 2019). In recent years, AI techniques have been applied in computational creativity tools to better support users in creative tasks. For example, Shi et al. (2017) developed a design knowledge retrieval and association tool, B-Link, to support idea generation, which employs a large-scale ontology network constructed using unsupervised learning. Chen et al. (2019) proposed an AI-based approach using a generative adversarial network (GAN) to produce synthesized images of associated ideas in an ontology network to stimulate idea generation. Sarica et al. (2020, 2021) came up with a Technology Semantic Network (Tech-Net) developed using AI-based natural language processing (NLP) techniques, which could support a range of creative tasks, such as idea generation, idea evaluation, and knowledge management. The creativity diamond framework proposed in Section 4 prompts users to the availability of these tools at different stages in a creative challenge, and some of these tools are described in Section 5.

#### **4. Creativity Diamond Framework**

The creativity diamond framework that is presented in this paper and illustrated in Figure 10 arose from a review of over 20 PhD theses supervised by the principal author over the last 20 years looking at aspects of creativity, design, innovation, and product development, along with consideration of previous frameworks and models. A doctoral thesis typically includes an overview of the relevant literature to define the extant knowledge base, as well as knowledge gaps and arising research questions. Consequently, in addition to identifying individual contributions from the research outputs, a given thesis also provides a readily accessible overview of research on creativity, despite some of the theses being up to 20 years old.

**Figure 10.** Creativity diamond framework.

As identified in Section 2, substantive research on creativity has been undertaken, revealing the relevance of diverse facets including phasing, incubation, perspective, use of analogy and metaphor, association, exploration, stimulation, convergence and divergence, sequencing, attention, and evaluation. Many of these principles have been embodied in models and frameworks aimed at encouraging creativity. Examples of frameworks that have emerged in the domain of creativity include models for system creativity as embodied in the creativity engine (Childs and Fountain 2011), the six divergent and convergent phases in the creative problem-solving process (Treffinger et al. 2006), the readiness to recreation process model of Hasirci and Demirkan (2007), the five-stage creativity model of Amabile (1983, 1988) for individual or a small groups, use of the TASC wheel (see, for example, Faulkner 2008), creative design (Gero 1996), and the double diamond (Design Council 2019). Some common features of these include phases of divergence and convergence, as well as iteration.

Thoughts are built by our mind, i.e., how we think, feel, and choose, which generates electrochemical signals within our brain to build the thoughts from associated memories. Leaf (2018) suggests we are likely to have between 8000 and 10,000 thoughts a day, with each thought informed by our trillions of memories. Indeed, thoughts and ideas abound. One of the principles of brainstorming and some other approaches to creativity is that quantity of ideas breeds quality (see Osborn 1963; Childs 2019b). This can be a challenge to accept sometimes as we tend to cherish an idea that we have come up with during an initial phase of work. We may have invested some time in this and have a strong affinity to the idea. In looking at any other ideas, we may judge the newer ideas against the one originally generated and keep on returning to and working on the original idea. This is sometimes referred to as fixation (Jansson and Smith 1991). There may be merit in this original idea and, if so, it is worth keeping this idea available for subsequent consideration. History and research on creativity both indicate the value in exploring an idea space prior to deciding on which idea to select. There may be other areas worth considering. Someone else may have a better idea. In an industrial context, it may be worth exploring the idea space in detail to see what additional ideas exist and how a competitive company might respond to an innovation. Constructs for encouraging divergent thinking include consideration of what options and alternatives there are or might be.

A small group exploring and recording their ideas using sticky notes can produce many ideas over a few hours. Indeed, the use of a few types of brainstorming methods, in

combination with ideas that emerge otherwise, can result in a few hundred ideas that need to be considered in order to identify which warrant further attention. The principles of brainstorming suggest suspension of judgement to promote an environment in which ideas can more readily emerge. Once a body of ideas has been established, a phase of selection and refinement of the ideas can take place. When considering the quality of an idea, specific aspects of the idea might become apparent and require attention to resolve an issue. This is a natural part of the convergent phase in idea generation. Convergent thinking involves focusing attention on the most appealing items in an extensive list of ideas. In evaluating which ideas to consider further and which to discard, one may choose to refer to the original brief to ensure that the idea is relevant and addresses the requirement.

The creativity diamond is a framework developed by incorporating fundamental principles to guide creativity. The framework embodies the principles of divergence, the generation of many ideas, such that, through selection and refinement, convergence can occur with selection of preferred ideas for further development. The creativity diamond framework is a guide that can be used in any domain or multidisciplinary setting to prompt which generative tools might be helpful.

A range of creativity tools and approaches to thinking creatively are presented in the creativity diamond, including various types of brainstorming, morphological analysis (see Garvey 2016; Childs and Garvey 2015), the principles of invention from TRIZ—the theory of inventive problem solving (see Gadd 2011; Tsai 2008), analogical reasoning (Han 2018; Han et al. 2018a; Mi 2019), use of metaphor (Mi 2019), systems thinking, design thinking, deductive and inductive reasoning (Park 2018; Park et al. 2020), and critical and analytical thinking.

The supporting information for the creativity diamond helps guide the selection of which creativity tool to use for a given type of person and when. For example, one might choose to use sticky note brainstorming, morphological analysis, aspects of design thinking, and use of metaphor during a divergent process to help generate ideas for a given opportunity or problem. Then, during a convergent phase of idea selection and refinement one might make further use of one of the types of brainstorming, critical thinking, the principles of invention, and analogical reasoning.

The creativity diamond presents activity as commencing in response to a need or opportunity. It may be that an idea to explore an area just arises or comes in response to a defined need as part of your work. The convergent activity may result in the selection of a preferred idea along with some refinement and embodiment of this idea. It is also possible that the convergent phase identifies a few or several ideas worth further consideration. The creativity diamond incorporates an iterative loop such that further phases of divergence and convergence can occur with these ideas in order to further develop the ideas, solve problems, and refine aspects as needed. Iteration is also implicit to many of the creativity tools and approaches to thinking which may result in cycling around an issue to address it within both the divergent and the convergent phases.

#### **5. Types of Thinking and Tools**

The creativity diamond presents a divergent phase to encourage the generation of many ideas for consideration and a convergent phase for the selection of preferred ideas and their refinement. Some of the many approaches to creative thinking are presented by means of the central graphic. These can be selected on the basis of preference, timeframe, a desire to use different types of approaches to creativity, and application. Table 1 provides a brief introduction and guide to the various approaches presented.


**Table 1.** Short description and merits/limitations of various creativity tools and approaches to thinking.


#### **6. Use and Justification**

Much of the experience in the use of creativity tools for the groups concerned with the authors of this paper arises from experiences in the multidisciplinary group project run across all undergraduate degrees at Imperial College London, Innovation Design Engineering (IDE) program, a double master's program run by the Royal College of Art and Imperial College London (see Childs and Pennington 2015), its "sibling" program Global Innovation Design (GID) (see Stevens et al. 2015), and the Design Engineering mEng in the Dyson School of Design Engineering at Imperial College London (see Childs 2019a). In addition, experiences have arisen from interactions with other universities, in industry, and in the Creative Thinking Tools for Success MOOC (Massive Online Open Course) on the Coursera and EdX platforms, in which over 300,000 people have engaged to date. In the case of the Coursera Creative Thinking Tools for Success MOOC, at the point of analysis, 238,749 people had enrolled with 3479 ratings at an average of 4.7/5, 97% positive endorsements, and 1046 reviews. The text responses were diverse and with 97% positive endorsements, overwhelmingly favorable, highlighting the overview of and access to the diverse creativity tools and thinking approaches.

Diverse creativity tools have been introduced to generations of students on the IDE, GID, and Design Engineering MEng programs to supplement innate approaches. The culture for the programs concerned is often characterized by student choice. Despite definition of learning outcomes, each student's pathway is unique as a result of definition of their own solo and group projects and electives. It is noted that the selective nature of the programs means that the population of students will not be representative of society, with high levels of qualification attainment and the freedom to select an area of domain preference within which to study. A typical introduction of a particular tool or approach to thinking will involve exposure to the approach, exploration of the fundamental principles of operation along with use of the tool for some examples or application. Students are subsequently free to use the approach at their discretion. The nature of design portfolios and narrative means that it is sometimes possible to identify use of a tool or approach through, for example, inclusion of a snapshot of hundreds of ideas on sticky notes, a matrix showing systematic exploration of an idea space, or the student's reflective narrative.

Students tend to have multiple exposures to the diverse forms of brainstorming such as sticky note, list, grid, alphabet, and brainwriting across different modules in their studies. Circle brainstorming, with its origin at Imperial, tends to be used for specific modules such as the Billion Dollar Question module where repeated pitching of an idea and its rapid revision based on consultant input prior to further pitching is compatible with the intense nature of the module. Systematic tools such as morphological analysis, TRIZ, and SCAMPER are introduced in specific sessions. Biomimicry, in a similar way to brainstorming, is widely introduced across different modules. Subjects such as the use of metaphor and analogy have historically tended to be introduced in passing, allowing a student to develop their skills and abilities in the subject through application across diverse projects. The use of AI to augment creativity and AI tools is usually associated with seminar presentation and students are subsequently given access to the tools concerned. Design thinking and systems thinking are introduced across a broad range of modules and projects ranging from Context in Design Engineering and throughout the first year of the Innovation Design Engineering and Global Innovation Design MScs. Specific topics on thinking such as critical and analytical thinking are introduced through principles and examples.

The wide range of tools and approaches available, particularly the approaches to thinking, can be overwhelming to a student group with recourse to familiar approaches such as sticky-note brainstorming and biomimicry common in the past. The "single graphic" in various forms (see, for example, Figure 11) for the presentation of the different approaches to creativity, along with encouragement to use more than one approach, has challenged this culture with students and groups of students exploring the use of a few approaches within both divergent and convergent phases of activity. The use of attribute tables such as that presented in Table 2, along with resource cards and associated links (Figure 12), has also been quickly adopted, allowing individual preferences, application, and time factors to be quickly considered in the selection of an approach. For example, "list brainstorming" is a low-difficulty tool that can be used either solo or in a group for less than 1 h. It is a tool suitable for extroversion personality traits aimed at producing little c outputs. The examples of resource cards presented in Figure 12 are part of a set of cards representing each of the creativity tools and types of thinking, serving as a prompt for an individual or team for the tool concerned.


**Figure 11.** Creativity diamond graphic.


**Table 2.** Creativity approach attributes. X = indicative; times are approximate; levels are indicative; L = low; M = medium; H = high; U = unspecified; h = hour.

**Figure 12.** *Cont.*

**Figure 12.** Examples of resource cards for some of the creativity tools.

#### **7. Conclusions**

The creativity diamond is a framework developed to aid users in the selection of an approach to augment creativity and creative thinking in divergent and convergent phases of a challenge or opportunity, particularly in design and product development. The framework has arisen from a review of over 20 PhD theses on creativity, design, innovation, and product development, which in turn build on their respective domain literature reviews. The framework promotes several forms of brainstorming, a series of systematic approaches to creativity, and several thinking and reasoning approaches. To accompany the framework, several resources have been developed, including upskilling materials in each of the approaches, summary information, and selection tools to aid identification of an approach at a given stage. The framework has been introduced across several degree programs at Imperial and other universities, as well as in industry, through a massive online open course. The creativity diamond framework is proposed as a guide that can be used in any domain or multidisciplinary setting to prompt which generative tools might be helpful to aid creativity, ultimately leading to innovation success.

**Author Contributions:** Writing—original draft preparation, P.C., J.H., L.C., P.J., P.W., Y.Y., E.D. and I.V.; writing—review and editing, D.P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** The principal author would like to express his gratitude to the 20 former PhD students, as well as current students, for their work in advancing the understanding of creativity. This of course is built on the years of creativity study and practice across all domains—a worthwhile endeavor.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Nomenclatures**

The following abbreviations are used in the article and associated graphics:


#### **References**

Abraham, Anna. 2013. The promises and perils of the neuroscience of creativity. *Frontiers in Human Neuroscience* 7: 246. [CrossRef] Abraham, Anna. 2018. *The Neuroscience of Creativity, Cambridge Fundamentals of Neuroscience in Psychology*. Cambridge: Cambridge University Press.

Abraham, Anna. 2022. Creativity or creativities? Why context matters. *Design Studies* 78: 101060. [CrossRef]

Amabile, Teresa M. 1983. *The Social Psychology of Creativity*. Berlin/Heidelberg: Springer.

Amabile, Teresa M. 1988. A model of creativity and innovation in organizations. *Research in Organizational Behavior* 10: 123–67.

Amabile, Teresa M. 2012. *Componential Theory of Creativity*. Boston: Harvard Business School.

Basadur, Min, and Garry A. Gelade. 2005. Modelling applied creativity as a cognitive process: Theoretical foundations. *Korean Journal of Thinking & Problem Solving* 15: 13–41.

Basadur, Min, George B. Graen, and Stephen G. Green. 1982. Training in creative problem solving: Effects on ideation and problem finding and solving in an industrial research organization. *Organizational Behavior and Human Performance* 30: 41–70. [CrossRef]

Baxter, Weston. 2017. Designing Circular Possessions: Exploring Human-Object Relationships in the Circular Economy. Ph.D. thesis, Imperial College London, London, UK.

Benedek, Mathias, and Andreas Fink. 2019. Toward a neurocognitive framework of creative cognition: The role of memory, attention, and cognitive control. *Current Opinion in Behavioral Sciences* 27: 116–22. [CrossRef]

Bhattacharya, Joydeep, and Hellmuth Petsche. 2005. Drawing on min"s canvas: Differences in cortical integration patterns between artists and non-artists. *Human Brain Mapping* 26: 1–14. [CrossRef]

Boden, Margaret A. 2004. *The Creative Mind: Myths and Mechanisms*. London: Routledge.

Campbell, Donald T. 1960. Blind variation and selective retentions in creative thought as in other knowledge processes. *Psychological Review* 67: 380–400. [CrossRef]

Chechurin, Leonid, and Yuri Borgianni. 2016. Understanding TRIZ through the review of top cited publications. *Computers in Industry* 82: 119–34. [CrossRef]

Chen, Liuqing. 2020. Data-Driven and Machine Learning Based Design Creativity. Ph.D. thesis, Imperial College London, London, UK.


Sarica, Serhad, Jianxi Luo, and Kristin L. Wood. 2020. TechNet: Technology semantic network based on patent data. *Expert Systems With Applications* 142: 112995. [CrossRef]

Sarkar, Prabir, and Amaresh Chakrabarti. 2011. Assessing design creativity. *Design Studies* 32: 348–83. [CrossRef]

Sawyer, R. Keith. 2011. *Explaining Creativity: The Science of Human Innovation*. Oxford: Oxford University Press.


Sweller, John. 2009. Cognitive bases of human creativity. *Educational Psychology Review* 21: 11–19. [CrossRef]


Wang, Zhihua. 2013. Effects Based Stimulation of Ideas in Design and Engineering. Ph.D. thesis, Imperial College London, London, UK.


## *Article* **Creative Process and Multivariate Factors through a Creative Course "Keep Calm and Be Creative"**

**Aleksandra Vuichard 1,\*, Marion Botella <sup>2</sup> and Isabelle Capron Puozzo <sup>3</sup>**

	- 1890 Saint-Maurice, Switzerland

**Abstract:** Creativity has been studied for a long time and it has become a more significant topic of research in educational fields in recent decades. The present paper outlines a multivariate approach to creativity and substantiates this approach by investigating the creative process and multivariate factors through a creative course for master's students at the University of Teacher Education in Switzerland. Our goal is to examine more specifically the stages of the creative process and the emerging multivariate factors in different creative activities. The article reports findings from the analysis of students' creative report process diaries as well as semi-structured interviews. Drawing on experiential learning, this pilot study was conducted in collaboration with master's student teachers (*n* = 10). The results show that the different microlevels of the creative process are the subject of variations from one creative experience to another. Most factors of the multivariate approach emerge from this kind of creative training. The discussion will allow for a review of the research results and also a better understanding of the creative process in the pedagogy of creativity.

**Keywords:** creativity; creative process diary; multivariate factors; preservice teacher training

#### **1. Introduction**

The current society is constantly evolving and changing. The students of today will have to solve more and more complex problems and situations. Skills such as creativity, flexibility and collaboration can serve both educators and students to find ways to innovate in their classrooms in order to develop such competencies. In the education field, creativity is particularly important in psychology. This discipline offers various definitions of creativity that promote a holistic understanding of individuals, going beyond the cognitive dimension (Guilford 1950; Sternberg and Lubart 1995, 1996; Torrance 1972).

This research draws on Lubart et al.'s (2015) multivariate approach to creativity. Creativity is defined as the ability to produce work that is both novel and appropriate. In this theoretical framework cognitive, conative, emotional and environmental factors are central to creativity. Recently Lubart proposed a new model of creativity, called the 7 C's model. According to this model, creativity is a complex concept that can be explored through seven different facets (Lubart 2017). In this paper we focus especially on the creator's characteristics that play a role in creativity and the creative process. More specifically, we would like to evaluate the creative process and multivariate factors of future teachers.

In his 7 C's model, Lubart identifies the following invariants: (1) the creator (personcentered characteristics); (2) the creating (the creative process); (3) the collaboration (cocreating); (4) the contexts (environmental conditions); (5) the creation (the nature of creative work); (6) the consumption (the adoption of creative work) and (7) the curriculum (the development and enhancement of creativity) (Lubart and Thornhill-Miller 2019).

**Citation:** Vuichard, Aleksandra, Marion Botella, and Isabelle Capron Puozzo. 2023. Creative Process and Multivariate Factors through a Creative Course "Keep Calm and Be Creative". *Journal of Intelligence* 11: 83. https://doi.org/10.3390/ jintelligence11050083

Received: 31 December 2022 Revised: 11 April 2023 Accepted: 14 April 2023 Published: 28 April 2023

**Copyright:** © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

The creator encompasses individuals who actively participate in creative thinking. The creative process is defined as the succession of thoughts and actions that leads to an original and adaptive production (Lubart et al. 2015). The creative process can be described at two levels: a macro level, which presents the stages of the creative process, and a micro level, which explains the mechanisms underlying the creation of ideas, for example, divergent or convergent thinking (Botella et al. 2016). This will be analyzed in the second part of this contribution.

Regarding collaboration, Glăveanu et al. (2016) highlights the importance of social dimensions in the creativity field. He shows that distributed creativity is seen as the interaction between the individual and the world that takes place over time. The focus is on interactions with other people and the artifacts that embody group knowledge which are important contributors to the process. Aragon and Williams (2011) underscore the significance of collaborative efforts when solving problems and advocate for interdisciplinary approaches that bring together experts with diverse skill sets, rather than relying on individual efforts. Creative activity develops from the relationship between an individual and the world of work, as well as from the ties between an individual and other human beings (Fischer et al. 2005). In that sense, some authors have been interested more specifically in collaboration and its impact on creativity. Glăveanu (2010) emphasizes that the focus of research has gradually shifted to a more social paradigm—the we-paradigm—in which creativity is characterized in terms of communication, collaboration and development as a result of socialization and social interaction. The we-paradigm aims to "put the social back" into the theory of creativity (Hennessey 2003; Glăveanu 2010). The recent study of Peilloux and Botella (2016) describes both the collaboration and collective dimension of the creative process. The collective dimension refers to the process during which different individuals have to collaborate. These authors highlight the relationship between the social functioning of the group and the dynamics of the creative process.

In terms of context, the systemic model of Csikszentmihalyi (1996) suggests that creativity is the result of the interaction between the individual and the socio-cultural context. Creativity depends, therefore, on the conjunction between a domain, an environment and a person. The significance of social interaction in creative activities—both in terms of its potential and its realization in daily life, professional and eminent creativity—should not be overlooked (Kaufman and Beghetto 2009). Kaufman and Beghetto share Csikszentmihalyi's view that creative processes and outcomes are shaped through the interactive interplay between the creator and the environment.

The traditional concepts about creativity and the ways of measuring and recognizing it in educational settings tend to emphasize creative accomplishments and traits that are static, rather than the fluid and constantly evolving nature of creative thinking and behavior (Beghetto and Corazza 2019). Recently, Corazza (2016) highlighted that creativity is a dynamic phenomenon. The path to a potentially creative outcome shifts over time and is shaped by the various social, cultural, historical and physical factors present in a particular situation. In this sense, he suggests a "dynamic" definition of creativity: "creativity requires originality and effectiveness" (p. 259) where it becomes possible for an idea to be enhanced or considered creative in a way that is delayed over time. This definition of creativity is still evolving, especially in terms of the factors that influence creativity (Capron Puozzo and Audrin 2021; Capron Puozzo et al. 2019; Vuichard and Capron Puozzo 2021).

According to the different facets of Lubart's 7 C's model, considering a pedagogy of creativity involves designing a school environment where the teacher allows divergent thinking and problem solving and seeks to connect learning objects with emotions. In the pedagogy of creativity, collaboration is systematically promoted allowing the broadening of horizons of possibilities, the sharing and co-construction of knowledge. In order to train creative teachers so that their learners also become creative, it is therefore imperative to work in collaboration with them to allow for the better development of creative capacities (Besançon et al. 2005). Presently, there is a new focus in the research on co-creativity (Walsh et al. 2014; Stenning et al. 2016; Schmoelz 2018) as a process that merges the individual,

collaborative and communal aspects of creativity. Co-creativity is a life-wide phenomenon (Craft 2005) that is personally and internally judged (Runco 1996, 2003) instead of externally evaluated. It is collaborative (John-Steiner 2000) and communal (Chappell et al. 2012) rather than individual, and it involves conflict (Chappell 2008) and breaking away from the routine (Stenning et al. 2016) instead of being in a state of flow. However, there is still a lack of research on how students display co-creativity during classroom activities and how we can foster co-creativity in a classroom setting. Schmoelz (2018) conducted a study based on a playful pedagogical design using narrative-Socratic dialogues with teachers and students. The main findings indicate that classroom activities that incorporate playfulness offer an opportunity for co-creative reframing, meaningful dialogue, emotional expression and collaborative storytelling that is filled with actions co-determined by all participants.

Considering that each creator may involve and bring the ingredients at various points throughout the creative process (Lubart and Thornhill-Miller 2019), in the following sections we try to explain the creative process and the individual differences, taking into account multivariate factors, particularly the collaborative aspect.

#### **2. Stages of the Creative Process**

Cognitive and differential psychology approaches are beneficial for comprehending the creative process by considering the creator's knowledge and personality. On the other hand, a combination of social and cognitive psychology approaches enables the exploration of the role of different factors in either promoting or hindering the creative process. For Lubart et al. (2015), the creative process is viewed as "a succession of thoughts and actions that leads to original and adapted creations" (p. 111). Since Wallas' four-step model of creativity (1926), including preparation, incubation, illumination and verification, different studies have explored the creative process (Botella et al. 2016; Cropley and Cropley 2012; Lubart 2001; Sadler-Smith 2016). According to recent research on Wallas' four-stage model, it has been reaffirmed that there is a lack of consensus among researchers regarding the number of stages. While Cropley and Cropley (2012) identified seven stages, Sadler-Smith (2016) based on Wallas' book, found five stages. Various authors have suggested models of the creative process that arrange the different subprocesses involved. For example, Mumford et al. (1991) delineated a series of fundamental processes for creativity that operate on information organized in categorical structures. These processes are problem construction, information encoding, category search, specification of the best fitting categories, combination and reorganization of category information to find new solutions, idea evaluation, implementation of ideas and monitoring. The model is dynamic and allows for cycling between different processes as deemed necessary during problem solving. Indeed, this model of creative thinking processes highlights a series of new measures that could be used to assess creative potential (Mumford and McIntosh 2017). This model not only holds significance in creating novel measures for evaluating creative potential, but also suggests some new ways for developing individuals' creative abilities. Scott et al. (2004) found that the most effective creativity training programs were those which used instruction and exercises expressly intended to develop these key creative thinking processes. Indeed, other work by Marcy and Mumford (2007) and Osburn and Mumford (2006) has shown the value of providing instruction in contributing to the more effective execution of the specific operations held to underlie the application of certain creative thinking processes. Drawing on existing creative and artistic processes, Botella and collaborators (Botella 2011) explore what the process of artistic creativity might entail. According to this more precise model, the creative process can be described at two levels: a macro level, which presents the stages of the creative process, and a micro level, which explains the mechanisms underlying the creation of ideas, for example, divergent or convergent thinking (Botella et al. 2016). Sixteen steps were held as important to investigate the artistic creative process: immersion, thinking, research, inspiration, insight, gathering, ideation (to think of new ideas), selection (to select ideas or documents), testing, precision (to refine), experimentation (to produce, to interpret), judgement (to step back, to discuss), to expound (to present one's work to

others), incubation (to let the ideas flow alone), abandonment (to renounce), and planning (to organize oneself).

Furthermore, for Botella and Lubart (2019, p. 272) "the creative process is dynamic by its components itself, their organization, their combination, the successive interactions it maintains with the environment, the unfolding nature of a phenomenon over time and its cyclical nature". This more recent definition introduces both complexity and dynamism to the initial proposal. In his 7 C's model, Lubart (2017) highlights that the creating or process is based on an analogical combination of more or less distant elements. The creative process as a dynamic phenomenon could be directly associated with cognitive, conative, emotional and environmental factors involved in the creative process (Botella et al. 2019). This research is based on Botella and collaborators' (Botella 2011) 16 steps model, which was adapted to the preservice training context. This model is chosen for the self-observation of the creative process during the learning situation. Fourteen stages were proposed to evaluate a student's creative process: problem definition, questioning, documentation, taking into account constraints, illumination, association, experimentation, evaluation, structuring, leaving it to chance, realization, finishing, pause and abandonment.

More recent studies have focused on exploring the creative process within a pedagogical context. Capron Puozzo and Botella (2018) emphasized the connection between emotions and learning by examining the creative process through the creative tasks provided in a training context with future teachers. The findings enabled the identification of the emotions experienced by the participants based on the creative activities utilized. It was found that most of the micro-processes varied during the training, and all the factors of the multivariate approach emerged. Another piece of research by Bonnardel and Didier (2016) investigated the impact of pedagogical methods on students' creative processes and the assessment of their creative outputs. The results showed that students who underwent brainstorming-inspired training generated more ideas than their peers, while the latter proposed more constraints. Additionally, Didier et al. (2022) conducted a study that focused on verbalizing and modeling the creative process when creating technical objects. These scholars aimed to gain a better understanding of the various phases involved in developing a technical object by using observation diary of the creative process (Botella and Didier 2016). Although these different studies have explored the creative process in the educational context, no existing model comprehensively describes the interplay between all components of the creative process, the collaborative aspects in preservice teacher training and the multivariate factors. While some studies have explored one or two of these components simultaneously, there is currently no model that integrates all these elements into a cohesive framework.

In this pilot study, it is crucial to recognize the significance of investigating the creative process within a pedagogical context, particularly for the target population studied (i.e., future teachers) and the specific creative activities analyzed.

#### **3. Multivariate Factors Fostering Creativity**

Botella et al. (2016) suggest that the creative process can be described at two levels: the macro level explaining the stages of the creative process (such as preparation, incubation, illumination, etc.) and the micro level, which refers to the mechanisms underlying the creative process (e.g., divergent thinking, convergent thinking, associative thinking). Those types of thinking are part of the cognitive factors including all the intellectual capacities that help the development of creative thinking. The cognitive process refers to the mental ability to comprehend and assign significance to perceived information and thoughts. This human capacity for cognition is utilized in the creative process, which involves activating subsidiary faculties and mechanisms for generating and organizing novel ideas or building upon existing ones. It should be emphasized that an individual's personal attributes can offer diverse levels of creative potential, contingent on the specific task or area of expertise. Additionally, the notion of cognitive style is frequently discussed, referring to the distinct ways in which an individual perceives, recalls, processes, assimilates, transforms

and applies information. This style of information processing is intricately linked to an individual's learning style (Guerra and Villa 2019). Concerning the conative factors, Lubart et al. (2015) explain that they bring together all the behaviors that are part of an individual's habits, such as motivation, perseverance, openness to new ideas, risk taking, etc. For example, Lubart and Thornhill-Miller (2019) have demonstrated that some studies indicate that individuals' inclination towards taking risks may differ depending on the specific domain of activity. To illustrate, an individual may exhibit a willingness to take a risk in a sporting context and endeavor to execute a new technique while ice-skating in a competition yet may not demonstrate the same level of openness to experimentation when engaging in a creative pursuit, such as the visual arts. Similarly, another person may display an eagerness to invest their efforts into a novel business concept but may not feel comfor2presenting new ideas in a written task (Lubart and Thornhill-Miller 2019). Both positive and negative emotions play an important role in the creative process (Audrin et al. 2020; Botella 2011). According to Davis (2009), a positive mood state can boost productivity in divergent thinking, possibly because it leads to a more relaxed standard for assessing the merit of an idea. Moreover, a study by Peilloux and Botella (2016) shows a specific affective profile for each stage of the creative process. These authors noticed that the stages that mostly occur at the beginning of the process, such as immersion, thinking or research, are associated with positive effects. On the other side, the stages that occur mostly at the end, such as judgement or exposition, are associated with negative emotions. However, these affective profiles can also be explained by the nature of the task. For example, a task that is perceived as challenging or novel may elicit a positive-activation affective profile in individuals, characterized by high levels of positive affect and arousal. In contrast, a task that is perceived as uninteresting may lead to a low-activation affective profile, characterized by low levels of both positive and negative affect.

Finally, environmental factors are characterized by all situations and interactions, including professional, family, school, social, cultural, national, local, etc. environments. They can be decisive in helping to reveal, activate and develop creative capacities. For Lubart and Thornhill-Miller (2019) the creative context is comprised of both physical and social spheres. Thus, the context refers to the favorable or unfavorable conditions for the development of creativity and is linked to environmental conditions. Some authors demonstrate that creativity and creative performance are influenced by the work environment (Dul et al. 2011; Amabile 1996; Amabile et al. 1996; Amabile and Pratt 2016). In this direction, Amabile initiated her research on the relationship between work environment, organizational factors and creativity, i.e., organizational creativity, combining for the first time managerial and psychological aspects. School can also be viewed as an organization which can benefit greatly from a creative environment. In the school context, Besançon et al. (2005) point out that the diversity of cultural activities as well as family structuring, which are environmental parameters, have a direct influence on creativity and the construction of creative thinking in children.

In this paper, our focus is centered on the social and collaborative dimensions, as well as on the environmental-social factors of students' creative processes in experienced creative activities.

#### **4. From Social Creativity to Collaborative and Collective Creativity**

In the late 1970s, more psychologists started to understand that creativity takes place in a social context (e.g., Harrington 1999). With this in mind, Simonton (2000) demonstrated that creativity is greatly influenced by social factors. He also showed that there was a change among psychologists in recognizing that creativity is not an individualistic phenomenon, but rather occurs within a social context. This recognition was further developed in the 1980s, when a distinct social psychology of creativity was established to complement the existing cognitive, differential and developmental perspectives on creativity. As evidence of this trend, scholars such as Harrington (1999), Amabile (1983) and Westmeyer (1998) began to publish works that explicitly explored the social aspects of

creativity. For Westmeyer (1998), the creativity of a product is determined by evaluators or judges who possess expertise in certain production domains. The social context in which the evaluation takes place also plays a role, and the passage of time may affect the evaluation. Moreover, Amabile (1996) investigates the role of social factors such as the cultural context, the dynamics of a group or team, etc. in the creative process. She argues that the social environment can have a significant impact on an individual's creativity. For example, supportive and positive social environments can enhance creativity by giving individuals the freedom to explore and experiment with new ideas without fear of criticism or negative judgement. Conversely, negative social environments, such as those that are highly competitive or critical, can inhibit creativity by creating an atmosphere of fear and anxiety that limits an individual's ability to think and create freely.

In the process of transition from an exclusively individual psychology of creativity to a more socially orientated one, three main concepts emerged: social creativity, collaborative creativity (Paulus et al. 2012) and collective creativity (Fischer and Vassen 2011; Pundt 2021). In order to better understand the social dimension of creativity, we will define these terms.

Social creativity is a large topic in which different forms such as collaborative and collective creativity can be discussed. For Fischer et al. (2005, p. 483), social creativity is the product of different forces: "the individual; the mix among individuals (the distinctive interests, skills and knowledge that compose specific communities); and the interactions between them and their social and technical environment". Referring to social creativity, people collaborate with each other by taking up tasks that fit well with their knowledge and personal interests (Fischer et al. 2005). For Purser and Montuori (2000), the term social creativity emerged by showing that creativity is the result of human interaction and collaboration and demonstrating a renewed interest in group creativity (Nemeth and Ormiston 2007; Paulus and Nijstad 2003 cited in Glăveanu 2010). Research in the field of social problem solving, as demonstrated by the works of Dodge (1986) and Spivack and Shure (1974), has emphasized the importance of creative abilities in the development of social skills. Mouchiroud and Bernoussi (2008) conducted a study focusing on the type of creativity that can be expressed in solving social problems. The results show that social creativity performance is linked to socially relevant variables, such as social competencies, popularity and parenting style. They indicate that the capacity for social creativity, as evaluated through divergent thinking tasks, is associated with a range of individual and experiential factors that are all related to how well an individual adapts to their social environment.

The term collaboration seems to be a significant element in social creativity. Glăveanu (2011, 2021) refers to group or team creativity as collaborative creativity. He makes a distinction between the socio-cognitive and socio-cultural aspects of this topic. The sociocognitivists tend to isolate the impact on others of some variables measured at the level of the individual only, while the socio-culturalists think that the social and the individual dimensions are linked by the act of collaboration (Glaveanu et al. 2021).Collaborative creativity does not describe only instances of social interaction in groups or organizations, but equally applies to individual creative processes. For Lubart and Thornhill-Miller (2019), collaboration refers to the interactions in which two or more individuals participate in a joint development, often with different or complementary skills, often creating something that they could not or would not create on their own. Some creativity is simply more easily recognized and labeled as "collaborative" because of its proximity in time or space to the others that helped make it happen. Thus, creativity is more of an individual act in some societies, whereas it is inherently more collaborative in others. In that sense, creative thinking cannot be isolated from the cultural matrix that promotes it and is fundamentally formed by it (Glăveanu et al. 2019).

Some authors mention a collective creativity (Sawyer 2012; Pundt 2021). Developed on the basis of biographical information, Pundt (2021) suggests a unique model of collective creativity. Although the model focuses on collective musical creativity, the model itself can be applied to other areas. The model proposed by this author includes an interaction between individual creativity focused on inspiration, elaboration and evaluation, and team or collective creativity that involves the same processes. Not surprisingly, the process of communication figures significantly in this model as the mechanism by which individual creativity will manifest itself as collective creativity. Sawyer (2012) also highlights that historical research confirms the increase in collaboration. Primarily, this is due to the growth in the complexity of the problems to be solved in research and in other fields. This is why collective creativity is more important than individual creativity, because it incorporates more points of view and expertise (Sawyer 2012). For this same author, a group is more creative than isolated individuals, provided that the team members have been working together for some time, but especially when they share certain conventions and have common knowledge.

Creativity as a collective phenomenon, therefore, implies dynamics of collaboration and even cooperation. In the pedagogical field, the principal difference between cooperative and collaborative learning is that the former is based on the principle of interdependence and the latter on sharing and bringing together knowledge (Baudrit 2005). According to the Johnson and Johnson model (1989), cooperative learning is instruction that involves students working in teams to accomplish a common goal by optimizing the learning of each person. Moreover, working cooperatively helps students to develop their social skills and take control of their learning. These authors underline that students in cooperative groups can achieve their learning goals if, and only if, the other students with whom they are cooperatively associated achieve theirs. One of the most significant conditions of cooperation is the fact that members of a work team must perceive that their success is conditional on the success of other team members in achieving a common goal (Johnson and Johnson 1989, 2009).

#### **5. Objectives and Hypotheses**

Creative processes can differ in terms of the number of stages involved and the characteristics of each stage. The aim of the present study is to directly question student teachers on the stages of the creative process based on Botella and collaborators' model (Botella 2011). Our goal in this paper is:


We hypothesized that:


To be specific, students do not experience the same stages of the creative process during every creative activity. In order to test these hypotheses, we conducted a study which evaluated students' creative processes and the aforementioned factors.

#### **6. Method**

This research design drew on action research (Barbier 1996; Van der Maren 2003), in which professors and students are involved in the process as collaborative actors. Furthermore, this course is based on experiential learning (Mandeville 1998, 2001) which allows students to live creative experiences from which they are then invited to make connections with the theoretical contents of the course (Capron Puozzo and Botella 2018). The aims of this interdisciplinary elective course are not only to train in creativity and its pedagogy, but also to improve the transversal capacity of students' creativity by bringing together multiple disciplines, such as history, art, science, design thinking techniques, etc.

#### *6.1. Participants*

This research was conducted at the University of Teacher Education in Switzerland in the fall semester 2018–2019. Participants in the research comprised student secondary school teachers (*n* = 10) on a master's degree course (at the beginning of the program). This interdisciplinary course included future teachers of different disciplines such as mathematics, French, physical education, science, etc. Three students were female and seven were male (mean age = 28.6; sd = 2.22; span = 23–31). The students voluntarily participated in this elective course and provided their informed consent. We provided clear and concise instructions that explained the purpose of the study and the response format.

#### *6.2. Material*

#### 6.2.1. Evaluation of the Creative Process through the Creative Process Report Diary (CRD)

The creative process report diary (CRD) is a relevant analytical tool for evaluating a student's creative process and the multivariate factors involved. Thus, the CRDs are used in order to respect an ecological approach to the creativity process and are particularly used when a study is conducted outside the laboratory (Botella 2011). The adapted diary is designed to collect both quantitative and qualitative data to track the process experienced in terms of micro levels (Botella et al. 2016) and learning.

The quantitative element is based on the closed questions of a CRD (Botella et al. 2011; Botella et al. 2019) in order to understand the creative process experienced by future teachers and the multivariate factors involved. This element was collected immediately after the training. The students had to:


#### 6.2.2. Interview Guide

The second tool used for the qualitative element one and a half years after the training was a semi-structured interview based on evocation (Vermersch 2011; Aden 2016). The complementary interview method was conducted in order to better understand what persisted from this creativity course and whether it had an impact on the students' teaching practices in the long term. The students were interviewed at the end of their master's degree. We chose this methodological approach because it allowed us to have access to the students' perception, that is the narration of the links they make between the training and their practices. This type of interview allowed us to see the students' representations in terms of creativity and training. By reliving the past situation, the student can analyze his or her practice and bring to his or her consciousness what he or she did not know at the time. Specifically, the goal was to understand whether the creativity course and its pedagogy had: (a) contributed to their perceptions in terms of creative training; (b) had an impact on interpersonal relationships; (c) contributed to their professional development in terms of

changes in their teaching practices and (d) had an impact on their emotions. We therefore developed a grid (Ghiglione and Matalon 1992). We determined the categories that we used to score the interview data. These categories are based on the research question and the data collected during the interview. In the interviews, we asked questions about their professional and skills development, the long-term impact of the creative course on students and any remarkable moments of creative activities. In this paper, we are interested in the social and collaborative aspects mentioned by students.

#### 6.2.3. Procedure

This creative course "Keep calm and be creative" covers three levels. The first level is theoretical and refers to the learning of content concerning creativity and its pedagogy. The second level is pragmatic, as it deals with the didactic transposition of theoretical content. The objective is for students to transpose these elements to learning situations in a school context. Finally, the third level focuses on experiential learning (Mandeville 1998, 2001), which allows students to experience and experiment with creative techniques. Scott et al. (2004) conducted a study through the quantitative meta-analysis of program evaluation efforts, drawing on 70 prior studies. The findings indicate that well-designed creativity training programs generally lead to improved performance, with these effects observed across different criteria, settings and target populations. Factors contributing to the effectiveness of these training programs were also examined, revealing that programs which focus on the development of cognitive skills and heuristics, using realistic exercises appropriate to the relevant domain, tend to be more successful. Our creative course shares a common foundation with other creativity programs, such as divergent thinking, problem solving and performance. In our research, we also focus on a cognitive, social, personality and motivational framework in the design of course content.

After having lived this creative experience, the students are then encouraged to reflect on their experience, in relation to the theoretical content of the course, through the CRD (Botella et al. 2011).

At the end of each creative activity students filled in their course diaries in order to keep a record of their experiences throughout the semester. Nine creative activities were experienced. Seven of the nine activities (described in Table 1 below) were carried out collectively promoting collaboration, exchange and group well-being. Two of them were conducted individually.


**Table 1.** Description of creative activities and experiences during the semester.


The quantitative data from the questionnaires were analyzed using SPSS (IBM SPSS Statistics, 26) software while NVIVO (NVivo, 12) was used for the qualitative data analysis.

#### **7. Results**

#### *7.1. Quantitative Data: Creative Process*

In order to define and analyze the stages of the students' creative processes, we used Friedman's non-parametric ANOVA (repeated measures) to test the variation between nine activities. This result indicated that the medians of each of the nine activities were not equivalent. Significant variations were present in these stages: problem definition, questioning, illumination, evaluation and finishing.

Moreover, after ANOVA, we conducted post-hoc analyses using the Durbin–Conover test to examine the pairwise differences between activities. The results of five significant stages are presented in Table 2. We chose the standard threshold of a *p*-value below .05 to assess the significance of our results.


**Table 2.** Significant stages.

We found that the test was significant for problem definition (Chi2 (8) = 16.6, *p* =.035). The definition served to discover the theme and to focus on the subject. This stage corresponds to the preparation stage in Wallas' model. The results indicated that during the world café, creative environment and museum activities and the escape game many participants checked the definition stage, which was not the case for the marshmallow challenge or the crea-experience. For the questioning stage (Chi<sup>2</sup> (8) = 21.8, *p* = .005), we observed that in the marshmallow challenge, the creacapture, world café, land art, creative environment, and museum activities, the escape game and the Bionique activity this stage was chosen by most students while for the crea-experience this phase was not selected by many participants.

The test was also significant for the illumination stage (Chi2 (8) = 20.4, *p* = .009), we found that in activities such as the creacapture, world café, land art, creative environment and museum activities this stage was more present than in the crea-experience. Moreover, the test was significant for the evaluation stage (Chi2 (8) = 22.2, *p* = .005); in the marshmallow challenge, the creacapture, world café and museum activities, the escape game and the Bionique activity most students were in this stage and were not in the crea-experience. For the finishing stage the test was also significant (Chi2 (8) = 16.9, *p* = .032), indicating that in the Bionique activity there were not many students noting this stage but in the world café, land art, creative environment and museum activities a lot of them checked this stage. Finally, this test was not significant for documentation, taking into account the fact that constraints, association, experimentation, structuring, leaving it to chance, realization, break and abandon were not significant. It also meant that all these stages did not differ between activities.

In order to identify the stages of the creative process and to test our first hypothesis we observed the frequencies of each of the stages during the nine training creative activities. In particular, we analyzed the percentage of participants who checked a level in the process at the end of each creative activity experienced during the course (Table 3). Looking at Table 3, we can see that most stages oscillate between the various activities. For example, the students demonstrated a marked interest in questioning and evaluation in the marshmallow challenge which was carried out with the intention of being an ice breaker. The second activity, creacapture, asking students to represent creativity via a video, image, recording, etc., involved problem definition, questioning and evaluation. During the creativity marathon, students participated in two different creativity activities: the world café and land art acitivities. For the world café, they mostly reported definition, questioning, illumination, evaluation and finishing. The students also participated in an artistic activity, the land art activity and had to create a work/piece with materials from nature. In this activity the majority of the levels were present: problem definition, questioning, illumination and finishing. In the creative environment where students used the ordinary and reverse brainstorming technique, all participants experienced the following levels: problem definition, questioning, illumination and evaluation. The museum activity and escape game were both characterized by problem definition, questioning, illumination and evaluation. On the other hand, the crea-experience activity did not mobilize these stages. The last creativity technique called Bionics mobilized the students in terms of problem definition and evaluation.


**Table 3.** Frequency of micro-processes reported by students in their CRD.


**Table 3.** *Cont.*

#### *7.2. Quantitative Data: Multivariate Factors*

In this section we look at the multivariate factors and try to find out their intensity.

#### 7.2.1. Cognitive and Conative Factors

Significant variations were present for the following cognitive, conative and environmental factors: patience, discipline, getting organized, decision-making, dynamics, collaboration and implication.

In order to test our second hypothesis, we noted that among the cognitive and conative factors, patience was the most used factor during the land art activity (M = 4.7), where students had to create a work together from the materials found in nature. In addition, discipline increased with the escape game (M = 4.7), where students were invited to collaborate in order to solve enigmas related to the theoretical content. The participants gave higher scores for organization during the land art activity (M = 4.8). On the other hand, the marshmallow challenge (M = 3.11) did not score highly for this factor. For the decision factor we observed the highest score in the museum activity (M = 4.67) and the lowest in the marshmallow challenge (M = 2.67). During the world café and land art activities, the dynamic factor increased (M = 4.70). Regarding environmental factors, the Table 4 shows that implication was the most relevant in the creative environment (M = 4.88) and museum activities (M = 4.78) in the techniques of idea generation through different brainstorming and collective challenges. The results were similar for collaboration (M = 4.75/4.78) during these two techniques.


**Table 4.** Significant multivariate factors.

In addition, the Friedman's ANOVA and Durbin–Conover post hoc tests indicate that the medians of each activity were not the same. For the multivariate factor, patience, (Chi2 (6) = 20.1, *p* = .003) we observed that in the marshmallow challenge, creacapture activity and the creative environment, this factor was chosen by most students, while this was not the case for the land art activity. The test was significant for discipline (Chi2 (6) = 12.9, *p* = .044); we found that in activities such as creacapture, the land art and museum activities and the escape game this multivariate factor was more present than in the marshmallow challenge. The results show that the test was significant for the factor, getting organized (Chi2 (6) = 13.4, *p* = .037), indicating that during the marshmallow challenge, creative environment and museum activities and the escape game this factor was more present than in the land art activity. The test proposed that the decision factor (Chi2 (6) = 14.4, *p* = .025) was observed in the creacapture, world café, land art, creative environment and museum activities and the escape game, but just a few students reported this factor during the marshmallow challenge. For the dynamic factor (Chi2 (6) = 13.5, *p* = .036), the results showed that during the world café, land art and museum activities and the escape game this stage was selected more than in the marshmallow challenge. For the collaboration (Chi2 (6) = 7.6, *p* = .007) and implication factors (Chi2 (6) = 12.7, *p* = .048), the test showed that in activities such as the creacapture, world café, land art, creative environment and museum activities and the escape game this factor was more present than in the marshmallow challenge. Lastly, for multivariate factors such as being kind, perseverance, being a perfectionist, workforce, concentration, quality, discussion and listening the tests were not significant.

#### 7.2.2. Emotional Factors

We also observed the emotional factors in Table 5. We found significant variations for curiosity, surprise, frustration, enthusiasm, awakening, hesitation and stress. For curiosity, the highest score was noted for the museum activity (M = 4.67) and the lowest for the crea-experience (M = 3.31). During the Bionique activity, surprise increased (M = 3.20). The Table 5 shows that frustration was most relevant in the marshmallow challenge (M = 3.22). For enthusiasm the lowest score was observed for the crea-experience (M = 2.13) while the highest was present for the marshmallow challenge and the museum activity (M = 4.22). For awakening the participants gave a higher score during the marshmallow challenge (M = 4.22) and a low score (M = 2.20) during the crea-experience. Hesitation was the most reported factor during the escape game (M = 3.10).


**Table 5.** Significant emotional factors.

Regarding these factors the result of the Friedman's ANOVA and post hoc Durbin– Conover tests indicated that the tests were significant for curiosity (Chi2 (7) = 25.8, *p* = .001), whereby in the marshmallow challenge, the land art, world café and museum activities, the escape game and the Bionique activity this factor was more present than in the creaexperience. The test was also significant for surprise (Chi2 (8) = 19.1, *p* = .014), indicating that during the marshmallow challenge, the creacapture, creative environment and creaexperience activities this factor was mobilized a lot which was not the case for the Bionique activity. In addition, for frustration (Chi<sup>2</sup> (8) = 17.9, *p* = .022), the test shows that in the creacapture, land art, creative environment museum and Bionique activities, frustration was more present than in the marshmallow challenge. For enthusiasm (Chi2 (8) = 18.9, *p* = .016), the test indicates that in the marshmallow challenge, the creacapture, land art, world café, creative environment and museum activities, and the escape game and Bionique activity this factor was chosen much more than in the crea-experience. A similar result was observed for the awakening factor (Chi2 (8) = 20, *p* = .010), whereby in the marshmallow challenge, the land art, world café and museum activities and the Escape game and Bionique activity this factor was chosen much more than in the crea-experience.

For the factor, hesitation, the test was significant (Chi2 (8) = 20.1, *p* = .010) and presented more during the creacapture, land art, world café, creative environment and museum activities and the crea-experience than in the escape game. For stress (Chi<sup>2</sup> (8) = 21.8, *p* = .005), the test showed that in the creacapture, land art, world café and creative environment activities this factor was selected much more than in the escape game.

For boredom, anxiety, deception, being proud, inspiration, satisfaction and exhaustion the test was not significant.

#### *7.3. Qualitative Data*

In addition to the quantitative data from the CDB, we conducted four semi-structured interviews based on evocation (Vermersch 2011; Aden 2016), one and a half years after the training. These interviews helped us to understand what persisted of this creativity course and whether it had an impact on the students' teaching practices in the long term. We used a thematic analysis by listing the major themes, then we made an inductive and intuitive reading of the interviews in order to observe what the students were really expressing (Paillé and Mucchielli 2016). In this paper we present the social and collaborative dimension expressed by the students.

#### 7.3.1. Interaction with the Group

One of the first memories was the sharing with others as well as the interaction with the group. One student reported: "the advantage was that we were in small groups so we had this opportunity to exchange well and that was an advantage too ... what I also remember was the discussions that allowed us to compare our reactions a little". For one student the group was seen as an activator: "In a good group that was important because it made everyone show their stuff and in the end it worked very well. At the beginning I would say a shy one and then quite enthusiastic". He also added: "We really saw what everyone was thinking, and then we put together each part of the group's thought". These verbatims also show that individuality is one of the important traits for collaboration. Fostering individuality in groups can promote alternate points of view.

Moreover, another participant reported: "there were interactions, I remember a little bit of reactions, or enthusiasm, or a little bit of excitement", or during the land art activity: "And then, I remember the interactions there was, and I remember that we had built this thing and in the end, it wasn't really land art, because we had put on ... and then you came in and you said it wasn't really land art because we put on clothes and stuff, so it was kind of funny, that's what made it stand out". It seems that this kind of interaction can influence a group and individual satisfaction by creating a good ambience.

Some students emphasized how important it was to be in a good group: "a nice group that worked well, everyone was involved", or, "we were a good team that participated well". Some of them mentioned that the change of group encouraged dynamics: "That was interesting because I was not at all with the same people the first and second time during these two challenges, but I felt much more comfortable the second time with the people I was with."

#### 7.3.2. Role of the Environment

The importance of the environment was described by these verbatims: "But discovering this place, uh interactive offices, for me it was something that was also striking because it is not a universe that we know in our academic world. We probably know it more and more in the world of entrepreneurship", or even further, "The environment, the fact that we ate there, it wasn't just that we were trying to do something, but it was embedded in a whole universe. It's like if you want to do a class on a theater stage and with costumes, it's pretty much the same thing". The fact that it was important to have the lessons in different environments was observed in this statement: "( ... ) it's important because it allowed you to completely erase what was there before and pass on something completely new each time. And each time, we would reset completely". Another student reported, "it was the way we were put in context to test the different approaches".

#### 7.3.3. Difficulties in Group Work

Nevertheless, some participants had a critical stance towards the proposed activities, which was expressed through difficulties in collaboration, especially at the beginning of the activities: "The first time, I had completely faded away because I saw that they were going off in all directions, then there were one or two who were trying to take the lead and trying to really ... wanted to give their idea without necessarily making compromises", or, "I think we had thought too much, we hadn't experimented and so it was the fact of being in a group, we can't necessarily, we didn't hurt . . . well, I had the impression that we had lost time to finally discuss, what, to understand instead of saying and getting into it, so that's why it was frustrating". In the situation where two students had to cooperate with the aim of making a funny presentation during the museum activity, one of them wrote: "it was a bit difficult ... but it's true that we are not necessarily the same personality, so I thought it was someone who was quite... it's someone who marked me enough".

According to these verbatims based on the social and collaborative dimensions, we can notice that the participants were in general enthusiastic when they were invited to collaborate and co-create together. Sharing experiences, ideas and knowledge seems to be important for activities in a group, as well as the work environment. It is also evident that collaboration depends on group members and can to some extent create difficulties when participating in some activities.

#### **8. Discussion and Conclusions**

The results show that the different microlevels of the creative process are subject to variation from one creative experience to another. The same observation is related to the collaborative aspects. Moreover, the CRD allowed us to notice that the creative process is not linear: lots of levels appeared at the same time (definition, questioning, association, experimentation, etc.). Most factors of the multivariate approach emerged from such a creative course. The discussion will allow us to come back both to the research results but also to show the importance of collaboration in the creative process.

#### *8.1. Stages of the Creative Process*

Regarding the microlevels of the creative process, these may be related to the nature of the task. By testing the variations between the activities of each stage, the test was significant for definition, questioning, illumination, evaluation and finishing.

Many participants reported a problem definition stage during the activities where they had to define the problems in relation to the school through various brainstorming sessions. That was not the case during the activities where the focus was on a challenge or the environment. The definition stage serves to discover the theme and to focus on the subject. This stage corresponds to the preparation stage in Wallas' model (1926). According to him, it is a preliminary analysis which allows us to define and pose the problem. The preparation stage in both Botella (2011) and Mumford et al.'s (1991) models is crucial for the creative process as it involves acquiring knowledge and expertise. While both models share the preparation stage, the main difference is in the middle stages. Mumford's model includes incubation and insight, which Botella's model combines into a single stage of generation (creating ideas and alternatives). In the context of our creative course, this stage was supported using brainstorming sessions which provided an opportunity for the students to gather information, research and develop expertise on a particular topic. By doing so, students are better equipped to generate a wide range of ideas and alternatives during the subsequent stages of the creative process.

The levels of questioning, illumination and evaluation were chosen in a large number of activities by the students. Questioning corresponds to understanding and reflection, which refers to the development of ideas. It seems that in the activities where the students had to propose different solutions or solution tracks these levels were more significant (Botella 2011). This process of generating new ideas or solutions to problems though collaboration with others can encourage individuals to contribute their ideas and perspectives. In collaborative creativity, team members need to search their memory for knowledge or ideas relevant to the problem, share these ideas and process them for further elaboration (Paulus et al. 2018). However, most student did not report these levels in the crea-experience. This may be related to the nature of this activity which did not consist in solving problems. The students performed the same task (summarizing texts) in the two different environments (relaxed versus hostile) during this activity.

In Mumford's model, the evaluation stage involves critically assessing the idea or solution generated during the insight stage. This assessment may involve considering the potential strengths and weaknesses of the idea or solution and identifying any areas for improvement or refinement. However, Mumford's model does not include a separate stage for selecting the best idea or alternative. This is the case in Botella's model. Her model provides a more explicit framework for selecting the most adaptable idea or alternative, while Mumford's model focuses more broadly on assessing and refining the idea or solution generated. In our creative course, at the end of the idea generation process, the students shared their proposals and evaluated the proposed solutions together. They then selected the solution that was most suitable for the school context.

We can confirm that most of the levels emerged in the students' creative processes.

#### *8.2. Multivariate Factors*

Variations were present for some multivariate factors, such as patience, discipline, getting organized, decision-making, dynamics, collaboration and implication. Implication was important during the technique for proposing the generation of ideas in a creative environment, as well as in the activity at the museum and in the escape game. Based on the motivational dynamics proposed by Viau (2009), it originates mainly in the perceptions that a student has of the proposed educational activity.

Variations were also present for some emotional factors, such as curiosity, surprise, frustration, enthusiasm, awakening, hesitation and stress. It is interesting to notice that curiosity and surprise were more significant in activities where the students were generating new ideas or solutions to problems through collaboration with others. In this sense, collaborative creativity can be an effective way to generate a diversity of ideas and perspectives, and can lead to more innovative solutions than would be possible through individual efforts. However, these activities can cause stress and frustration. We suppose that this may be related to the fact that students do not always come up with enough ideas during the brainstorming sessions. We can confirm our hypothesis suggesting that different multivariate factors emerge during the creative process.

Regarding environmental factors, implication and collaboration were most relevant in the creative environment and the museum activity. These creative techniques required idea generation through different brainstorming and collective challenges. Paulus et al. (2016) suggests that group and collaborative creativity refers to the generation of new and useful ideas or products by two or more persons who deliberately engage in a creative/idea generation task. When students work together on a project or task, they are exposed to different perspectives and ideas, which can inspire and stimulate their own creativity. Collaboration also allows students to bounce ideas off each other and work together to solve problems, which can lead to the development of new and innovative solutions. The ability to cooperate is essential because it develops reflexivity and creativity, thanks to the problematization process at work in the cooperative group, a learning organization (Perraud 2019). Furthermore, Marková (2003) shows that exchanges and external and internal dialogues are co-constitutive, and does not reduce groups to the individual, but precisely recognizes the social and dialogical nature of the individual mind (Glaveanu et al. 2021).

The limitation of this pilot research is certainly the sample size (10 participants). While the results of the quantitative analyses are encouraging, caution should be exercised regarding the potential generalizability of the results; a larger sample replication is needed to support these preliminary results.

Despite these limitations, this study provided valuable insights into students' creative processes. Specifically, it enabled us to identify several distinct stages involved in the process in the creative training context. The variations between the activities for each level help us to better understand what multivariate factors come into play in the creative process.

However, our goal was to create a supportive and open-minded group culture that facilitates collective creativity and encourages the flow of ideas, inviting all members to contribute. Structured creative techniques such as a brainstorming or design thinking adapted to the pedagogical context can be helpful in guiding the creative process and ensuring that all ideas are given equal consideration. This approach can also be embedded in the pedagogy of creativity with the aim of thinking through tasks differently (Puozzo Capron 2013). This also means that, instead of individual activities that stimulate thinking skills, it is important to create opportunities for collaboration and interaction with material and cultural artefacts. According to Dewey's (1902) research cited in Glăveanu et al. (2015a), this view emphasizes teacher–student collaboration and the grounding of education in everyday life practices, allowing students to relate new knowledge to prior experiences and make use of it in practice. It can deepen their understanding of the world and the possibilities of acting within it. The transformative potential of rethinking pedagogy from

a new perspective, however, enables us to transcend traditional educational models and concentrate on dialogical experiences, playful engagement with educational content and a rethink of existence and materiality (Glăveanu et al. 2015b). Making the students aware of the importance of creativity may lead them to teach creativity and divergent thinking in their future classrooms.

**Author Contributions:** Conceptualization, A.V. and M.B.; methodology, M.B.; software, A.V.; validation, A.V., M.B. and I.C.P.; formal analysis, A.V.; investigation, A.V. and I.C.P., resources, A.V., data curation, A.V.; writing—original draft preparation, A.V.; writing—review and editing, M.B. and I.C.P.; visualization, A.V., supervision, I.C.P.; project and administration, A.V.; funding acquisition, A.V. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding. The APC was funded by Haute école pédagogique Vaud (HEP Vaud).

**Institutional Review Board Statement:** Ethics approval was not required according to our institution's guidelines and national regulations. After the participants provided their informed consent, the students completed their CRD during their course on creativity. All authors have had the data collection protocol validated by their teams ahead of launching the study.

**Informed Consent Statement:** All subjects gave their informed consent for inclusion before they participated in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.

**Acknowledgments:** The creative process report (Botella 2011) diary was adapted to the preservice training context.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


Amabile, Teresa M., and Michael G. Pratt. 2016. The dynamic componential model of creativity and innovation in organizations: Making progress, making meaning. *Research in Organizational Behavior* 36: 157–83. [CrossRef]


Baudrit, Alain. 2005. Apprentissage coopératif et entraide à l'école. *Revue française de pédagogie* 153: 121–49. [CrossRef]

Beghetto, Ronald A., and Giovanni E. Corazza, eds. 2019. *Dynamic Perspectives on Creativity*. Berlin: Springer. [CrossRef]


Craft, Anna. 2005. *Creativity in Schools: Tensions and Dilemmas*. London: Psychology Press.


Dewey, John. 1902. The school as social center. *The Elementary School Teacher* 3: 73–86. [CrossRef]


Ghiglione, Rodolphe, and Benjamin Matalon. 1992. *Les Enquêtes Sociologiques: Théories et Pratiques*. Paris: Armand Colin.


Hennessey, Beth A. 2003. The social psychology of creativity. *Scandinavian Journal of Educational Research* 47: 253–71. [CrossRef]


Lubart, Todd. 2017. The 7 C's of Creativity. *The Journal of Creative Behavior* 51: 293–96. [CrossRef]


Runco, Marc A. 1996. Personal creativity: Definition and developmental issues. *New Directions for Child Development* 72: 3–30. [CrossRef]


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

## *Concept Paper* **Lessons from the Conservatory Model as a Basis for Undergraduate Education and the Development of Intelligence**

**Robert J. Sternberg 1,\*, Linda Jarvin <sup>2</sup> and Ophélie Allyssa Desmet <sup>3</sup>**


**Abstract:** We review the musical conservatory as a model for educators to learn how to enhance admissions, instruction, and assessment in liberal arts collegiate settings. Although conservatories serve primarily students wishing to enter musical careers of various kinds, the model on which they are based can, in many ways, serve any student and any school. We review some of the history of conservatories and describe how they work. Next, we explore how they develop a wide range of technical, cognitive, affective, and conative skills. Finally, we show how the skills they develop are important not just for music students but also for all students who will enter the world of work and face difficult and unexpected adaptive challenges.

**Keywords:** conservatory; liberal arts; musical performance; entrepreneurship; problem solving; adaptive intelligence

#### **1. Introduction**

Future musicians may choose between a traditional undergraduate liberal arts education and an education in a musical conservatory. The two models are rather different and focus on somewhat different knowledge and skills. The liberal arts model provides more of a general education, whereas the conservatory model is seen as providing a more strictly preprofessional education. However, the differences may not be quite what they appear to be, and an ideal education for any student might combine elements of each.

In this article, we discuss whether a blend of the liberal arts and conservatory models might have advantages not only for musicians but also for all students who wish to prepare for the adaptive challenges of the mid-21st century. One key advantage of the conservatory model is to explicitly aim to develop not only students' academic skills, but also their psychosocial skills as they apply to real-world careers and related adaptive contexts. In contrast, the traditional liberal arts model is primarily concerned with academic and cognitive skills. We even live in a time when some states in the United States are trying to eliminate prosocial skills, even at the elementary level where they are usually taught, using the argument that such skills represent a hidden "political" agenda (Goldstein and Saul 2022).

In liberal arts education, at least in the United States, students generally engage in a variety of courses designed to teach them basic knowledge about the world in the first two years of college. They then specialize in a major field of study in the last two years (Sternberg 2016). Sometimes, there are required courses, or required choices from lists of prespecified courses. Generally, the idea is that students will learn basic knowledge about, and skills for coping with the world in the first two years, and then the subject matter that will serve as a basis for specialization in the second two years. Under this model, even an engineering student might take courses in literature, philosophy, or the arts during the first two years, and then move on to the specialized concentration in engineering in the latter half of their college career. This liberal arts approach to higher education, with an emphasis on breadth of academic courses early on, is gaining traction in other countries, including

**Citation:** Sternberg, Robert J., Linda Jarvin, and Ophélie Allyssa Desmet. 2022. Lessons from the Conservatory Model as a Basis for Undergraduate Education and the Development of Intelligence. *Journal of Intelligence* 10: 34. https://doi.org/10.3390/ jintelligence10020034

Received: 2 May 2022 Accepted: 14 June 2022 Published: 15 June 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Europe (van der Wende 2017). Such a broadening of the liberal arts approach makes sense, because technical information changes, but the broad thinking skills one needs to adapt remain relatively constant, although their emphases may change. For example, recognizing conspiracy theories and distinguishing disinformation from true information are skills that have become especially important in recent times (Sternberg Forthcoming).

Although the liberal arts approach sets out to develop a broader range of skills than a professional education does, we argue that it does not, in its current form, sufficiently focus on building the cognitive, emotional, and social skills that are needed to deal with the adaptive challenges of the world of the 21st century.

The conservatory model also typically provides liberal arts education but in a different context. For example, consider a student studying to be a brass player (trumpet, horn, tuba, trombone, euphonium) at the New England Conservatory of Music. The future musician will take liberal arts and modern-language courses, but also studio, brass, music theory, music history, entrepreneurial musicianship, ensemble, chamber music, and recital courses (https://necmusic.edu/sites/default/files/documents/2021-2022%20Academic% 20Catalog.pdf (accessed on 10 June 2022)). Specialization in other areas will yield a similar course program geared toward the other specialty, such as strings, voice, or composition.

We believe that the conservatory model offers learning-related and general-intelligencerelated skills beyond those provided by the typical liberal arts curriculum. What particular skills does the conservatory model offer for all students, not just for students of music?

#### **2. Basis in the Theory of Adaptive Intelligence**

Using the theory of adaptive intelligence (Sternberg 2021a) as a theoretical framework, we explore several skills central to the conservatory model that would benefit all students. Adaptive intelligence comprises a collection of skills, attitudes, and behaviors based upon creative, analytical, practical, and wise thinking (Sternberg 2021a). The goal of adaptive intelligence is, as its name implies, a broad adaptation to a given environment. Broad adaptation encompasses narrower adaptation, which is modifying oneself to fit an environment. However, it also encompasses shaping an environment to provide a better fit of the environment to oneself, and selection of environments to account for the fact that, sometimes, the environment in which one finds oneself is a bad fit or even a potentially deadly fit, as in much of Ukraine at the time we are writing in mid-2022 (e.g., Murphy 2022).

We suggest that the conservatory model successfully identifies and promotes adaptive intelligence. Furthermore, we believe that all students would benefit from developing their adaptive intelligence through aspects of the conservatory model. As such, there are several lessons that liberal arts programs could carry over from the conservatory model. These are lessons that can be learned from adaptive intelligence as applied directly to the domain of musical composition and performance (Sternberg 2020c).

Another related take on musical intelligence is that of Gardner (1983, 2011). Gardner views musical intelligence as one of eight multiple intelligences, each independent of the others. Musical intelligence involves skills needed for playing a musical instrument, composing, conducting, singing, dancing, and other musical activities. The other seven intelligences in Gardner's theory are linguistic, logical–mathematical, spatial, naturalist, bodily–kinesthetic, interpersonal, and intrapersonal.

In Sternberg's (2020c) theory, to succeed as a professional musician, one first needs to be highly analytical. One needs to understand what musical messages the composer intended to convey, how they are conveying the messages, and how those messages can be translated into instrumental (or voice) music. For example, what particular actions of the hands or breath (or voice) can achieve the desired effects? How can these effects be made to be more effective?

Memorizing pieces may be part of one's task; however, as in the liberal arts education, mere memorization does not prepare one for the challenges one will face in one's life and career. Moreover, and importantly, one must be able to analyze one's own performance so that one can practice deliberately to improve one's performance. Simply playing the music

repeatedly will not improve one's performance much, if at all. Mindless repetition is not a viable path to expertise (Ericsson and Pool 2017; Sternberg and Kibelsbeck 2021). In the same way, simply memorizing the material in a textbook will not improve one's ability to cope with life and career. In life and career, one must constantly analyze problems and one's own processes of solution, seeking to improve one's ability to face the challenges the world presents. Jarvin and Subotnik (2010) interviewed gatekeepers in classical music (artistic directors, newspaper critics, talent managers) about the factors that they perceive to be most important to a performer's success at different stages of their development, from student to internationally acclaimed professional. Respondents identified three key analytical skills: ability to analyze music, knowing one's strengths and weaknesses, and pertinent risk taking. The ability to analyze music is especially important in the early stages, when a performer must refine their understanding of the patterns and structures in music so that, when they enter a professional career, they have mastered the ability to learn new material quickly. As students grow older, though, it is important for them to analyze their own strengths and weaknesses to develop the necessary self-knowledge to capitalize on strengths and make the right choices for the future. Additionally, for the professional, risk taking is perceived as a sign of artistry. It requires an analysis of the available options to take those risks that will prove rewarding.

Second, to succeed as a professional musician, one needs to be technically proficient and also be *creative* in one's interpretation of the music (Subotnik et al. 2003). Only creative performers can imagine different options and take the appropriate risks discussed above. Child prodigies may be unusually technically proficient, but they cannot succeed as professional musicians unless they go beyond technical proficiency. Their musical interpretation must be (a) novel—distinguishing them from other musicians; (b) appropriate—it matches the music and its period; and (c) effective—it connects with its audience. In contrast, many undergraduate programs place relatively little emphasis on creativity, and even fewer assess students' creative potential. Yet, in the mid-21st century, creativity will matter in practically any professional or other life pursuits: The ways of the past are not adequate when the world changes at such an astonishing rate, and when we need to equip students with creative problem-solving skills (Grant 2017; Kaufman and Sternberg 2019, 2021; see Plucker 2016).

Third, to succeed as a professional musician, especially in current and predicted near-future times, one needs to develop a vast repertoire of practical skills. Practical skills encompass emotional skills such as persistence and restoring self-confidence after a setback, and social skills such as collegiality and effective and appropriate self-promotion (Subotnik et al. 2016). How does one prepare for and perform effectively in an audition? How can one be entrepreneurial in creating new forms of musical performance and collaboration and in reaching existing and new audiences? How can one succeed when conventional audiences are sparse (as in the times of COVID-19)? If one seeks a career as a soloist, then how can one project a kind of charisma that singles one out from the pack? If one seeks a career as an orchestral, band, or chamber player, then how can one collaborate with others to produce music that is more than just the sum of its separate parts—that harmonizes and synergizes with other members of the group? How can one teach these skills to younger people, some but not all of whom will become professional musicians? Similarly, in any career, one needs to develop the practical skills needed to survive in one's life and in one's occupation—the tacit knowledge of one's professional success (Sternberg 1994). Too often, students graduate from college without these skills, thereby slowing down their integration into the professional world. Students who have not acquired these practical skills in their home environment (and have not been provided them in college) will be at a particular disadvantage. If they do not quickly pick these skills up on the job, they fall behind or potentially find themselves jobless.

Fourth, good conservatory training will also develop in musicians the *wisdom* to make the world a better place. This wisdom is seen in a variety of ways—the organization of benefit concerts to help charitable causes, the performance of music to groups in hospitals and

schools and homes for the aged who need spiritual sustenance, the commissioning of music from struggling (and not-so-struggling) composers, playing music to calm and distract an audience that is sheltering from bombs (https://www.themoscowtimes.com/2022/03/1 7/violin-becomes-weapon-of-resistance-in-ukraine-shelters-a76981 (accessed on 10 June 2022)) or in-between air raids (https://www.thestrad.com/news/solo-cellist-performs-in-c entral-kharkiv/14642.article?utm\_source=adestra&utm\_term=&utm\_medium=email&utm \_campaign=25628 (accessed on 10 June 2022)), and the communication to audiences that music is a common language that everyone can speak, even if there are somewhat different "dialects" through which it is conveyed.

The original meaning of the "conservatory" was as a place for orphans to find refuge and be educated in the shared language of music—a wise way to encourage social cohesion. Great musicians are wise musicians. Similarly, great professionals in any field are wise as well (Glück and Weststrate Forthcoming; Grossmann et al. 2020; Jeste and LaFee 2020; Schwartz and Sharpe 2011). They give back to the world—they help to achieve some kind of common good that goes beyond their own resources or reputation (Sternberg and Glück 2022a, 2022b).

Fifth, musicians have different patterns of talents. No two musicians have exactly the same pattern of gifts and talents. A successful musician figures out what they do well and what they do not do as well, and then develops, as possible, a repertoire and techniques that enable them to show what they do particularly well. However, the same principle applies in all fields. Writers, artists, scientists, teachers, nurses, businesspeople, and everyone else have unique patterns of strengths and weaknesses. They succeed in their careers to the extent they can capitalize on their strengths and correct or compensate for their weaknesses (Sternberg 2020c). They show what their unique contribution to the world can be. In the conservatory model of education, this is acknowledged, and students are guided toward developing the best version of themselves that they can be. Would not all students benefit from such a pursuit?

Finally, successful conservatory training helps develop a *passion* for music as a way to make the world not only a different place but also a better place. Music can help bring the world together. It is universal. The world is so fraught with problems today—violence, climate change, pollution, poverty, dictatorships, and the like. If there has ever been a time when we need to focus on not just individual but also species wellbeing and survival, this is one such time. In any field, people need to leverage their skills in a positive and transformational way, not in a way that merely makes some individual or group a profit.

There are three basic differences, we would argue, that typically differentiate conservatory from liberal arts education:


accreditation agencies have pushed for more culminating capstone experiences for graduating seniors (Gray and Schermer 2011), at least in the United States, most Bachelors' programs somehow do not culminate in an ultimate performance criterion that matters in its own right. A grade-point average (GPA) is simply an average of course performance—it is not a distinctive performance in its own right.

3. *The various strands of knowledge and skills need to be merged into the capstone performance.* Although students take a variety of courses in a conservatory—as they do in a liberal arts college program—because the conservatory has the ultimate performance criterion, the various aspects taught need somehow to be synthesized into a successful performance. In college, one can have isolated knowledge bases for all the different courses one takes, and it may not matter. In a conservatory, that will not work—if one cannot somehow synthesize them into successful performance, they will not matter. In that sense, the conservatory model not only encourages transdisciplinarity, but also makes the combination of learning-related skills a requirement for success.

Conservatories explicitly, then, do what all undergraduate institutions need to do. They prepare their students for the world the students will encounter, not just for a hypothetical world of multiple-choice problems with quick solutions and seemingly "correct" answers (Sternberg 2020b). All educational institutions could learn from conservatories how to provide a better education—to create more adaptively intelligent individuals—that prepares students for the world of the future.

Any form of education, including the conservatory form, has limitations that need to be acknowledged as well. What might some of these limitations be for the conservatory model? Consider three potential limitations.

First, although conservatories typically offer liberal arts options, these options are usually limited. The emphasis is on disciplinary rather than interdisciplinary education, although there might be interdisciplinarity within the world of music (e.g., instrumental music and voice). Additionally, music—or at least the theory of music—has a certain structure that not all disciplines have. The model is more similar to the European model of education, where students choose a specialization right from the beginning of their undergraduate years. In Germany, for example, one applies for a particular field of specialization in the undergraduate program of the university, and admission is by specialization.

Second, a conservatory has the upside that, if a student wants to enter a musical career, employers know that the student is very well prepared for a career in music. However, if the student decides on a career in another profession or decides for advanced training in another field, decision makers may view the student as too narrowly educated to be worthy of the risk in selecting them.

Third, styles in all fields change over time. For example, the best players of the Bach Six Suites for Cello of the 20th century, such as Mstislav Rostropovich, sound quite different from the best players a generation later, such as Yo-Yo Ma. Similarly, Fritz Kreisler's romantic style of playing the violin is unlikely to be mistaken for a predominant style in 2022. Part of what a liberal arts education prepares one for, at least in theory, is the fact that there is little stability in life—whatever preparations one makes will last only so long.

These limitations are only potential. Any conservatory can, in theory, offer any liberal arts courses it wishes to. Employers and graduate schools can look at transcripts and other records and realize that, just because a school follows a conservatory model, it does not mean the education that it offers is narrow. Additionally, students can be prepared for changing trends in any educational environment—liberal arts, conservatory, or otherwise. These are challenges, but not insurmountable ones. In the end, the best preparation for life, we believe, combines aspects of conservatory education with aspects of liberal arts education.

#### **3. The Special Functions of Conservatory Education Elaborated**

The crucial aspect of conservatory education, we would argue, is in the name conservatory. The education is based on conservation. We suggest that such a notion

would apply particularly well to the world today because there is such a pressing need to conserve—in the traditional sense of conserving natural resources, of course, but also of conserving civilization as we know it. Civilization is under so many threats today dictators massacring people and, in one case, destroying a country, to feed their egos and their fantastical notions about their countries' historical place in the world; a pandemic that has killed over six million people as of March, 2022 (https://www.who.int/emergencies/ diseases/novel-coronavirus-2019 (accessed on 10 June 2022)); the devastation of natural resources, in part because of irresponsible leadership in countries with leaders who do not care about the environment they leave to future generations; global climate change, which is progressively making the world less and less livable not only for humans but also for millions of other species; extreme poverty and famine killing people while billionaires take luxury cruises into outer space. We need to use our intelligence to advance not only our own selfish interests, but the conservation of the world for our own species and for many others (Sternberg 2021b).

The other crucial aspect is the integration of knowledge and skills. Performance on a musical instrument (including the human voice), in composition, or directing, is the ultimate test of success in conservatory education. In music, one will be judged not on one's independent bits of knowledge or assorted isolated skills, but on how one can integrate one's knowledge and skills into performance. This integration is often lost in traditional college education. One often is judged in college on little more than one's coursework and perhaps a GPA averaging grades in that coursework. Even if there is a "senior project" or capstone, it rarely fully integrates the knowledge and skills one has acquired during one's college education. Conservatory education is devoted to the solution of a problem: How can I perform in a way that will display my musical excellence and set me apart from others who may be technically proficient but who lack the particular unique creativity, analysis, practical skills, and wisdom I can bring to my profession? These questions promote lifelong learning, as they will be answered differently throughout a performer's development. We suggest that a college education in a liberal arts tradition might do the same.

Conservatories have one other special feature. Students are continually evaluated and compared not only to each other, but also to absolute standards of excellence in musical performance. In contrast, it is possible, in many liberal arts programs today, to just get by. Because of grade inflation, just getting by may result in grades that are respectable and even high. Sternberg (2016) has discussed trends in current education that lead to "mediocracy", or the valuing of a mediocre performance as good enough, or even, in some circumstances, as excellent (such as when students are evaluated for adherence to some particular dogma) (see also Hermanowicz 2013). There is much to be said for a curriculum whose very structure ensures a certain level of excellence.

#### **4. Applying the Conservatory Model**

In musical training, one always keeps in mind that one is training to make a difference to a musical audience. In the theory of adaptive intelligence, the audience is humanity—not just humanity of the present but also of the future—our children and our children's children, onward into the future.

As in musical training, one approaches the problem of doing what is best for one's audience in a multidisciplinary way. For example, in musical performance one must have the technique to play pieces successfully, but also creative, analytical, practical, and perhaps wisdom-based skills. How might this "play out" for solving a problem for one's current and future audience, such as global climate change? For one thing, one is taking a problembased approach, as one would, say, if one wished to provide the best rendition of the Bach Six Suites for Cello.

First, just as one learns in a conservatory musical theory, in solving the problem of global climate change, one would need to understand the theory underlying climate change and why it is happening. How and why do greenhouse gases accumulate? How does the accumulation of these gases lead to changes in the climate? What kinds of chemical and meteorological changes take place in response to them (see IPCC 2022)?

Second, in a conservatory, one learns music history. To understand the phenomenon of global climate change, one needs to understand its history and how it has come to be so problematic for the world. How have the changes accumulated over time? Why are they so much more severe in our lifetime than in much of the past? Is the history a continuous one, or do the changes become catastrophic after reaching a tipping point? What approaches have been tried in response to climate change, and how were they effective, or not?

Third, in a conservatory, one learns about playing in ensembles, chamber music groups, orchestras, and/or bands. That is, one learns to solve musical problems in groups of different sizes and composed of individuals with different experiences and areas of competency. Real problems of consequence are almost always solved in groups, not individually. Problem solvers with different kinds of expertise (as in an ensemble) work together toward a common goal. Often, one can get through a college education without having learned to work with others. Even in the instances when group projects are assigned, it is rare that students are explicitly taught to work effectively in teams; most of the time, those students with prior leadership skills take the lead, rather than all participants learning about project management and effective governance. The result is graduates, such as many of our politicians, who become show-boaters. They cannot work with others but only for their own self-advancement, with a thin veneer of offering or pretending to help their constituents. To solve a problem such as climate change, people will need to work together, as professionals—climatologists, meteorologists, chemists, biologists, and of course, businesspeople and laypeople—toward the common goal of producing a climate that is more conducive to quality of human life and the lives of other species (Berry et al. 2018; Mahaffy et al. 2021).

Fourth, in a conservatory, students study musical entrepreneurship, which has become increasingly important, especially in the age of COVID-19. Students learn how to build a business, whether a social business (Yunus 2013) or a profit-driven one that showcases their talents, individually or collectively. It has become clear that the problem of global climate change will not be solved without entrepreneurs of many kinds offering products and services that are alternatives to those of the big oil, gas, automotive, and other companies that have large investments in currently used fossil fuel fields.

Fifth, conservatory students develop and practice conative and motivation-related skills and attitudes to help them persist through the highs and lows that accompany a musical career. Pursuing a musical career is notoriously difficult because of the highly competitive nature of the field. Therefore, being technically proficient is not at all sufficient to navigate a professional musical career successfully. Successful artists are selfconfident, persistent, passionate, and intrinsically motivated about their music (Jarvin 2017; Subotnik et al. 2003). They possess a willingness to take risks and to continuously engage in self-assessment for self-improvement (Subotnik et al. 2003). Therefore, students are explicitly taught conative and motivation-related skills. Although not all talent domains are as competitive as the musical arts, all students, regardless of their chosen career, would benefit from explicit instruction and practice of conative skills and attitudes. Learning to persist and assess your own strengths and weaknesses, for example, will set students up to be lifelong learners in any field.

Sixth, and perhaps most important, conservatory students know they will be judged, in the end, on their performance, whether in a studio or as a soloist, or as a member of a group. The "grade" is not what matters. What matters is whether they have the knowledge and skills effectively packaged for action. This, too, has been missing from a college education, where actual activities often take the form, at best, of extracurricular activities. Students need, whether through internships, service learning, or cooperative education, to get out and deploy the knowledge and skills they have learned in the classroom. Climate change, like all complex issues, requires students to engage in systems thinking and understand how the different components of the issues interplay and impact one another. In music, technical

wizards who cannot play in harmony with others in the orchestra will not produce a great performance, and neither will a collegial quartet where each individual lacks technical skill or emotional depth in their interpretation. When it comes to climate change, this complexity can be experienced by participating in an activity such as constructing a mural showing how the factors identified in the 2022 Intergovernmental Panel on Climate Change report interact. In music, this complexity is experienced in attending a performance, which brings together the technical skills of the interpreters, the harmony of their interaction with each other, and the creativity expressed in the interpretation of the score.

#### **5. Admissions**

Admission is a way for an institution to signal what is important to them. What are they looking for, and what are they trying to develop? Who is considered suitable to study in their institution? A school that counts almost exclusively on mathematics and science performance would send a very different message from that of a school that looks more broadly. Admission to an undergraduate program can be either open or competitive. In conservatories and most liberal arts schools, admission is at least somewhat competitive. However, the criteria are quite different.

In a conservatory, the ultimate goal is to produce musicians of the first order. The best way to predict future musicianship is to assess current musicianship. This assessment usually is carried out through an audition of some kind, typically in person but, especially in the times of COVID-19, possibly online or by other means that do not require in-person performance. The performance matters a lot. Someone may have achieved straight A's in high school or high standardized test scores, but those are not going to help much in the prediction of musical success. Even grades in music courses will not predict success well because conservatories are more competitive than typical high schools. Even music programs in traditional liberal arts schools are typically much more competitive. The ultimate goal of a conservatory is to produce accomplished musicians, so the audition of musical performance will matter a great deal (Subotnik and Jarvin 2005) because it is probably the best indicator, at least prior to entry, of how candidates will fare as performers.

Traditionally, in competitive conservatories, the audition is a one-shot opportunity to play a predetermined repertoire. It is a highly stressful event, and the experience mimics the realities of a performance in front of an audience (Jarvin 2017). Another key feature of the audition process is that it can be carried out behind a screen, meaning that the committee rating the audition performance cannot see the candidate (Bennett 2008). The rationale is that the assessment of the performance will thus be less impacted by the candidate's visible and physical attributes, creating a more equal playing field.

Additionally, conservatory admissions procedures focus on identifying students' teachability and potential (Jarvin 2017). Music education and higher levels of talent development in any field require a level of individualized learning trajectories. To accommodate those individual learning trajectories, conservatory teachers implement ongoing, formative assessments focused on the students' responsiveness to the curriculum and instruction. This teaching philosophy is present in the admissions procedures as well. Many conservatory programs offer students who have demonstrated great potential, but lack readiness or technical proficiency, the opportunity to participate in a preparation year. This preparation year allows students to receive training and support to best prepare them for successful participation in the conservatory program. Conservatory educators have realized that conative aspects such as passion, intrinsic motivation, persistence, self-confidence, and risk taking can make up, to some extent, for limited technical proficiency (Jarvin 2017). Thus, the admissions procedures are designed to assess not only technical proficiency, but also teachability and the aforementioned conative factors.

Liberal arts programs do not have a comparable means of assessment. Students may submit grades from high school, standardized test scores, the Common App, or some other application answers and essays, letters of recommendation, and perhaps even some kind of work they have done. Many of these (e.g., name of high school, experience described

in the personal statement, profession of the recommenders) would provide information about the applicant's background and thus possibly influence the admission committee's perception of the candidate and preclude a "behind a screen" appraisal of the performance without extraneous and possibly misleading contextual clues about socioeconomic status. In addition, many, if not most applicants to liberal arts colleges do not yet know what they want to do, so it is not clear how the audition experience would be mimicked.

We suggest that there are four ways of supplementing traditional admissions assessments that would reflect the emphasis of conservatories on the skills that really matter for future endeavors. First, one can require a major project in any area of endeavor that resonates with the applicant, and that shows what the applicant can do, including attained level of technical proficiency, in an area of interest to them. The idea is to assess their knowledge and skills in actually completing a project rather than merely in the abstract. Second, one can measure creative, practical, and wisdom-based skills in addition to the memory and analytical skills that are traditionally measured by standardized tests and grades in high school (Sternberg 2010, 2015). Third, one can measure more traditional analytical skills, but in discipline-relevant ways (e.g., Sternberg 2020a). Fourth, one can measure conative and motivation-related skills and traits. Any of these procedures would represent an enhancement of current practice. The problem with current practice is not only that it represents a too-narrow range of skills, but also that it sends the wrong message about what matters for future success, both in college and beyond the college years.

#### **6. Conclusions**

The conservatory model of undergraduate education has useful implications for how schools might rethink admissions, instruction, and assessment to further both domainspecific knowledge and more wide-ranging adaptive creative, analytical, practical, and wisdom-based intellectual skills. Educators often think of the conservatory as a place exclusively for technical training, but that is not what the modern conservatory is. Rather, it is a place to learn how to learn, learn how to think, learn how to adapt, and learn how to perform in real, consequential worldly settings. We suggest that, to the extent schools wish to re-envision education, in general, especially as education develops adaptive skills, conservatory training can teach educators as much as—or more than—it teaches the students who participate in it.

There are various ways of testing some of the ideas in this essay. First, one could test openness to experience with a typical performance measure and cognitive flexibility with a maximum performance at the beginning and end of college to determine how a liberal arts curriculum versus a conservatory curriculum impacts each, if at all. Some colleges, such as Oberlin, Ithaca, and Gettysburg Colleges, have both kinds of curricula, enabling something of a direct comparison. Second, one could survey human relations offices across diverse fields, including music, to survey their openness to hiring both liberal arts and conservatory graduates. The study even could be carried out as an experiment where hypothetical applicants are presented with identical credentials, except that each hypothetical applicant is presented either as a conservatory or a liberal arts graduate. The experimenter then could determine whether one kind of graduate is preferred over the other, independent of actual credentials. Third, one could compare professionally valued accomplishments of graduates of liberal arts and conservatory programs in a variety of professions, including, but not limited to, music.

We believe, in the end, though, that the comparison between conservatory and liberal arts programs should not be viewed simply as a competition for which is better than the other. Rather, the two kinds of programs should be viewed as each offering the other features that can optimize the education they provide to their students. Each has unique advantages that, when thoughtfully combined, might create a truly outstanding instructional program at the undergraduate level.

**Author Contributions:** All authors were involved in conceptualizing and writing the paper. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


Gardner, Howard. 1983. *Frames of Mind: The Theory of Multiple Intelligences*. New York: Basic Books.

Gardner, Howard. 2011. *Frames of Mind: The Theory of Multiple Intelligences*, rev. ed. New York: Basic Books.


Grant, Adam. 2017. *Originals: How Non-Conformists Move the World*. New York: Penguin.


Hermanowicz, Joseph. 2013. The culture of mediocrity. *Minerva* 51: 363–87. [CrossRef]


Jeste, Dilip, and Scott LaFee. 2020. *Wiser: The Scientific Roots of Wisdom, Compassion, and What Makes Us Good*. Gardiner: Sounds Good.


Larousse. 2005. *Dictionnaire de la musique*. Paris: Larousse.


Schwartz, Barry, and Kenneth Sharpe. 2011. *Practical Wisdom: The Right Way to Do the Right Thing*. New York: Riverhead.

Sternberg, Robert J. 1994. Tacit knowledge and job success. In *Assessment and Selection in Organizations*. Edited by Neil Anderson and Peter Herriot. Hoboken: Wiley, pp. 27–39.

Sternberg, Robert J. 2010. *College Admissions for the 21st Century*. Cambridge: Harvard University Press.


Sternberg, Robert J. 2020c. Toward a theory of musical intelligence. *Psychology of Music* 49: 1775–85. [CrossRef]


Sternberg, Robert J., and Judith Glück, eds. 2022b. *The Psychology of Wisdom: An Introduction*. Cambridge: Cambridge University Press. Subotnik, Rena F., and Linda Jarvin. 2005. Beyond expertise: Conceptions of giftedness as great performance. In *Conceptions of*


## *Article* **Fostering Engagement, Reflexivity, and 21st-Century Skills in Middle School: A Pilot Collaborative Action Research on Identity Formation with Adolescent Co-Researchers**

**Pascale Haag 1,2,\*, Titouan Fantoni <sup>1</sup> and Stéphanie Dubal <sup>3</sup>**


**Abstract:** Identity construction during adolescence constitutes a primary psychosocial developmental task. A growing body of research has addressed the importance of school education in fostering adolescents' identity formation and the skills they need to thrive. Although several studies aimed at defining the factors contributing to a coherent, stable, and integrated identity formation, none sought to investigate this question from the adolescents' perspective. This contribution aimed to explore new ways of fostering 21st-century skills among adolescents through action research. Five adolescents aged 13 to 15 participated in the research process, creating a survey to answer a research problem mainly focused on identity construction in adolescence. A reflexive analysis of the co-research process highlighted the interest in involving adolescents as co-researchers to foster their social and emotional skills. The deployment of the resulting survey in a sample of 1210 adolescents from the general population highlighted the importance of gender diversity for constructing various dimensions of identity.

**Keywords:** adolescence; social and emotional skills; identity; children co-researchers; action research; gender

#### **1. Introduction**

How education can foster healthy identity development among adolescents and the acquisition of transversal skills—collaborative, social, emotional, and civic skills—to enable them to face the challenges of the 21st century is a challenging question for educators, as well as for parents and researchers. Identity construction during adolescence constitutes a primary psychosocial developmental task (Branje 2022; Erikson 1968; Helve 2019; Negru-Subtirica et al. 2017). This process involves a "complex interplay of intrapsychic processes and interpersonal experiences" (Abbasi 2016). The transition from childhood to adulthood is rendered increasingly difficult in a rapidly changing world, where new challenges, such as globalization, development of technologies, increasing individualism, and climate change are likely to influence the development of adolescents, their social relations, and their mental health (Patel et al. 2007). School interventions are needed to foster adolescents' identity development, support them in developing a positive and coherent sense of self, and help them acquire the skills they need to thrive (Lavy 2020; Tiwari et al. 2020; Verhoeven et al. 2019). Following this lead, we conducted a co-research with adolescents (study 1) that resulted in a second research about essential dimensions of identity (study 2).

#### *1.1. Identity Formation during Adolescence*

Erikson's psychosocial development remains an essential theoretical framework for studying identity formation during adolescence (Erikson 1968). He conceives identity as a

**Citation:** Haag, Pascale, Titouan Fantoni, and Stéphanie Dubal. 2022. Fostering Engagement, Reflexivity, and 21st-Century Skills in Middle School: A Pilot Collaborative Action Research on Identity Formation with Adolescent Co-Researchers. *Journal of Intelligence* 10: 64. https://doi.org/ 10.3390/jintelligence10030064

Received: 1 August 2022 Accepted: 5 September 2022 Published: 6 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**<sup>\*</sup>** Correspondence: ph@ehess.fr

"fundamental organizing principle which constantly develops throughout the lifespan". This principle is considered a synthesis of elements from the past (personal history), from the present (needs and personality), and from expectations of the future. This synthesis process is cardinal to adolescent development when one explores various social roles. However, as pointed out by Phillips and Pittman (2007), Erikson's theory does not lend itself easily to empirical research methods.

Refining Erikson's work, the identity status paradigm proposed by Marcia (1966, 1980) goes one step further. It is characterized by the adolescent's levels of identity exploration and commitment to self-chosen goals. Marcia differentiates two processes of identity construction: Exploration—the search for different alternatives for oneself in an area of life—and Commitment—the adhesion to a set of values, aims, and beliefs. Depending on their combinations, four statuses are identified (Marcia 1966; Marcia et al. 1993).

Figure 1 represents the four statuses' main characteristics that can be summarized as follows (cf. Marcia et al. 1993, pp. 7–8):


Many longitudinal studies found these statuses stable (Meeus 2011). Empirical research found that the statuses with higher engagement and exploration levels show a better

psychosocial integration in society, as well as higher levels of well-being and self-confidence and fewer depressive symptoms (Arnold 2017; Meeus 2011) and that the knowledge and understanding of these statuses have solid implications for therapeutic and educational interventions (Kroger and Marcia 2011).

This psychosocial framework of identity development is coherent with Cuin's (2011) sociological approach to adolescence. For this author, adolescence's "crisis" would be related to the experimental nature of adolescent behaviors: adolescents tend to move away from previous normative models and test and adopt new ones while privileging those that seem most valuable to them. The adolescents' ability to manipulate social norms depends on two principles:


These two dynamics embody a subjectivation process, i.e., the construction of a social subject, both agent and actor of social norms. Sociology constructs the theoretical framework of adolescence as a moment of autonomy without independence (de Singly 2006), during which the dynamics of integration and strategy allow this subjectivation process. Autonomy refers to identity criteria, whereas independence refers to statutory requirements (Galland 2008). Thanks to social media and other contemporary changes, today's adolescents have significant decision-making power over their own lives, especially regarding the constitution of their peer groups: they, therefore, have more control over how they fit into social norms (Galland 2008; Metton-Gayon 2006). In this way, sociological and psychological frameworks of identity complement each other, showing how healthy identity development relates to one's adhesion to social norms.

Finally, Erikson's psychosocial development theory does not consider that adolescence is the moment identity elaborates as a stable entity for life: identity evolves constantly. This is consistent with Marcia's model, which considers a person's identity determination as a process resulting from individual commitments. Such commitments are not made once and for all but can be questioned throughout one's life.

A healthy process of adolescent formation of identity guarantees a better integration into society. For this, considering well-being and self-confidence in developing identity is essential. Since Erikson's work, many studies have corroborated that well-being and identity formation are strongly related (Luyckx et al. 2006). The links between wellbeing and identity styles have also been investigated, indicating a negative association between a diffuse/avoidant style—lack of exploration and commitments, difficulty in setting goals—and various indices of well-being and a positive, hopeful outlook toward the future (Phillips and Pittman 2007). All these studies indicate a positive association between social and emotional skills on the one hand and healthy identity development and well-being on the other.

#### *1.2. Fostering 21st-Century Skills among Adolescents*

Since the beginning of the millennium, there has been an increased interest in the question of social and emotional skills. They are variously referred to as 21st-century skills, psycho-social skills, non-academic skills, character strengths, soft skills, life skills, or transversal skills (Borghans et al. 2008; Heckman and Kautz 2012). There is no single exhaustive list since different authors worked with other lists. The World Health Organization defines them as "abilities for adaptive and positive behavior that enable humans to deal effectively with the demands and challenges of life" (WHO 1994). It recognizes ten skills grouped into five pairs (problem-solving, decision making, creative thinking, critical thinking, self-awareness, empathy, interpersonal relationship, good communication, management of stress, and management of emotions.)

Several studies have elaborated on this definition and have come to consider these skills as a coherent and interrelated set of psychological abilities involving specific knowledge, intra-psychological processes, and attitudes, which make it possible to increase individuals' autonomy and empowerment, to maintain a state of psychological well-being, to promote optimal individual functioning and to develop constructive interactions (Kankaras and Suarez-Alvarez 2019; Lamboy et al. 2022; Schoon 2021). In a synthesis compiled for Santé publique France, Lamboy et al. (2022) propose a taxonomy of 22 skills classified under three broad categories—cognitive skills (e.g., awareness, self-control, thinking critically, ability to achieve goals, to make responsible choices or to solve problems creatively); emotional skills (e.g., identifying and understanding emotions and stress, ability to regulate emotions and to manage stress in everyday life, coping skills); social skills (e.g., pro-social attitudes, assertiveness, and constructive conflicts resolution).

A growing body of studies shows their decisive role in the development of mental health, physical health, work performance, and social relations (Mikolajczak et al. 2020). The development of these skills thus represents a significant issue in public health, education, and social action today (Lamboy et al. 2022). School climate and pedagogical practices contribute to the development of a wide variety of skills among pupils and students, such as self-efficacy (Dweck 2016; Usher and Pajares 2008), problem-solving (Baraké et al. 2015), cognitive flexibility, divergent thinking, and creativity (de Vries and Lubart 2019; Scheibling-Sève et al. 2017), social and emotional skills (Oberle and Schonert-Reichl 2017; Osher and Berg 2017). School climate and pedagogical practices can also favor intrinsic or self-determined motivation (Deci and Ryan 2000) as well as prosocial behaviors and civic engagement (Denney 2022).

#### *1.3. Fostering Identity Formation: Lack of Interventions*

As Schwartz and Petrova (2018) pointed out, the field of identity interventions is still relatively young, and etiological work suggests that interventions may facilitate identity consolidation. Connecting schoolwork with "real-world outcomes" is one of their recommendations to foster adolescent identity development. Incorporating identity development into prevention programs is another avenue of intervention. Moreover, the inherent limitations of interventions designed solely by adults being widely established, relying on peers, promoting adult-youth partnerships to conceive interventions, and placing young people in positions of leadership are likely to help young people develop a healthy and consolidated sense of identity, supported by advocacy and empowerment and leadership (Schwartz and Petrova 2018).

Studies show that a feeling of consistency and coherence within one's sense of identity is associated with higher levels of well-being and lower levels of depression or anxiety (Meca et al. 2015). By contrast, lack of family and community support, and struggle to integrate various aspects of identity (gender, sexual, religious, cultural, etc.) relate to higher risks of health-compromising behaviors (Schwartz and Petrova 2018). In addition, short-term intervention efforts fail to produce long-term gains (Kroger and Marcia 2011). School support for students' exploration of their identity is related to civic engagement and positive psychosocial development in adolescence (Crocetti et al. 2014; Kaplan et al. 2014). Finding efficient ways to promote healthy identity development in adolescents is therefore essential.

#### *1.4. Children as Co-Researchers*

Since the early 2000s, the inclusion of children or young people themselves as coresearchers to better understand their perspective has been the subject of much deliberation, both about the benefits of these new approaches and about their limitations (Bradbury-Jones and Taylor 2015; Camponovo et al. 2021; Lundy et al. 2011; Smith et al. 2002). For young people, participating in a project as co-researchers, thus being involved in the elaboration of a research question, the collection of data and its analysis can contribute to building their self-confidence, improving their critical thinking, autonomy, engagement, and sense of competence (Kellett 2010; Suleiman 2021). In terms of research outcomes, their participation provides more direct access to knowledge derived from children's own

understanding of their environment and subcultures. It, therefore, provides new insights that complement other approaches and enrich the knowledge gained as a result of the research (Bradbury-Jones and Taylor 2015)—having the opportunity to actively contribute to an authentic research project, whether as co-researchers or as joint authors, alongside experienced researchers, also affects adult-adolescent relationships and allows them to make their voices heard rather than being incorporated as a 'data source' (Groundwater-Smith and Mockler 2016).

There are, however, also practical and ethical limitations to this approach: on the one hand, children are not trained in research, and it is, therefore, necessary to provide them with some knowledge and skills to participate fully in the project; on the other hand, it is essential to take into account the asymmetric nature of the child- or adolescent-adult relationship, to be aware of the power relationships involved, and to ensure that children's participation is safe, with their consent, by the ethics of research, and with the possibility of withdrawing from participation at any time (Bradbury-Jones and Taylor 2015; Camponovo et al. 2021; Fielding 2011).

One additional premise of this research was that by involving adolescents in a project which directly resonates with their concerns and by letting them take part in the decisionmaking, they would gain a deeper understanding of scientific research methods and requirements and establish meaningful relationships with significant others (peers and adults) while sharpening their critical thinking and problem-solving skills (Jacquez et al. 2020; Suleiman 2021). Such practices—i.e., integrating research on topics that are meaningful to the students in the classroom—improve learning-related attitudes, self-efficacy, autonomy, communication skills, teamwork, and collaboration and ultimately lead to increased social support and community transformations (Jacquez et al. 2020).

#### **2. Theoretical Framework and Research Question**

This paper presents a research process comprising two interconnected studies—the first embedding the second. It aims to explore new ways of fostering 21st-century skills such as critical thinking, collaboration, and sociability among adolescents through a collaborative research project involving up to five adolescent co-researchers (study 1). They have been actively involved at all stages of the co-research process, and the initial discussions led to the design of a questionnaire assessing some cardinal dimensions of identity development from the point of view of adolescents themselves, with an emphasis on gender identity. This questionnaire was used to conduct a survey, the results of which are presented in study 2.

In the framework of collaborative research involving co-researchers of different ages and statuses, the interplay and dynamics emerging at the same time as the work is carried out make it difficult to detect and objectively assess any transformation among the participants who are immersed in the process and are thus not necessarily able to take a step back. Since this project is intended as a pilot study focusing on the effects of being involved in a research project on adolescents' psychosocial skills, it was necessary to find an appropriate way of assessing the transformations induced by the research setting among its participants.

This led us to resort to Engeström's *Activity theory*. According to Engeström (1987), one of the limitations of traditional psychological and sociological research lies in the difficulty of understanding change in numerous everyday situations within complex contexts. This observation led him to propose the use of the concept of an Activity System as a unit of analysis (Engeström 2000) and "to understand individual action and support individual and system development, we must study action in the context of the broader activity in which it is taking place" (Daniels and Cole 2002, p. 311). This cross-disciplinary approach is widely recognized as a valuable approach for studying human practices in various fields involving human activity, such as psychology, education, management, culture, and information systems—where individual and social levels are interconnected (Monaghan 2016; Vandebrouck 2018).

We propose to study the implemented procedure by analyzing it as a "system of activity" instead of focusing on each factor taken in isolation. In the Activity theory framework, an activity or set of activities is considered as mediated by different contextual elements: subject, object, artifacts, etc. The object, sometimes also called the goal, is what motivates the activity. According to Vandebrouck (2018, p. 679), the object is "a characteristic that distinguishes one activity from another"—in our case, "ordinary" classroom activity vs. intentional and systematic attempt to nurture 21st-century skills. The rather heterogeneous category of artifacts also mediates the activity, sometimes also called tools, instruments, or technologies. Artifacts refer to all the resources—already available or created by the subjects—to reach the object; they can be concrete (e.g., digital tools, surveys) or immaterial (e.g., thoughts, decisions, researchers' skills, feelings). The object leads to an outcome adolescents' ability to use such skills. Throughout the process, subjects and objects "form a dialectic unit: subjects transform objects, and at the same time, subjects are transformed" (Vandebrouck 2018, p. 679). For this study, the system is analyzed from the point of view of the adolescent co-researchers. Our activity system can thus be broken down into its component parts and represented as in Figure 2.

**Figure 2.** Theoretical framework using Engeström's Activity theory to analyze the transformation of subjects in this research.

The scientific angle chosen for the current project falls within a trend of reflection on the researchers' posture regarding their research objects and on the place of all the actors involved in a scientific investigation (Camponovo et al. 2020; Lyet 2017). In this context, the usual division of labor between researchers who are "producers of knowledge" and respondents who only possess knowledge of experience and action, excluded from the field of "legitimate knowledge", is not an option: participants in this project adhere to what some researchers call a new 'science-society contract,' which recognizes the role of all actors in the production of knowledge, and for which the key words are 'participation' and 'reflexivity' (Barré 2017; Bonny 2017). Our approach seeks to overcome the hierarchy of powers and knowledge in line with critical epistemologies. It falls into the broad category of collaborative and partenarial research, which refers to a reflexive partnership aiming at the co-production of 'actionable' knowledge, i.e., knowledge built in and for the sake of action (Juan 2021).

More specifically, the authors are entirely in line with the approach known as transformative action research, promulgated by Bilorusky (2021), where research, inquiry, and

action are brought together in transformative ways to make a difference. Transformative action research is an organic, evolving process in which action and research affect, influence, and transform each other, acknowledging the use of improvised strategies as part of the process by actively involved actors in the social reality being studied.

This project owes a great deal to that of Camponovo et al. (2020) in the sense that we aim to bring together the points of view of adolescents with those of a research team on a given topic to obtain the most nuanced, comprehensive, and integrated possible perspectives on the knowledge thus produced.

#### **3. Study 1: New Ways of Fostering 21st-Century Skills by Involving Adolescents as Co-Researchers**

#### *3.1. Context and Participants*

Lab School Paris is the first French school inspired by the North American model of laboratory schools, pioneered by John Dewey in Chicago at the end of the 19th century. Founded in 2017, it started with 27 pupils aged 8–11. In 2021–2022, around one hundred students from 6 to 15 years old (elementary and middle school levels in the French educational system) were enrolled. Since its opening, Lab School Paris has maintained regular collaborations with a network of researchers linked to various institutions. In particular, since 2019, it has been participating in a European Erasmus+ project entitled LabSchoolsEurope: Participatory Research for Democratic Education that aims to develop and share democratic practices for teaching in heterogeneous classroom settings (Haag 2021).

At the beginning of the school year 2021–2022, some middle school students started discussing gender and sexual orientation issues. For some, these issues caused such discomfort and anxiety that they hindered their learning at school. This was a situation without precedent: since its opening as an elementary school in 2017, the school has been growing along with its students, opening new levels every year. The enrolment of middle school students gave rise to further questions and challenges, such as welcoming adolescents' concerns or fostering their intellectual and emotional development. In the absence of a predefined framework within the school to enable these questions to be voiced in a safe environment, the founder of the school and first author of this article—a trained psychologist and assistant professor at the École des hautes études en sciences sociales (EHESS, Paris)—proposed to create an ad hoc group for students wishing to participate. This group would be facilitated by an intern from Lab School Paris holding a Master's degree in philosophy. The idea was to co-construct a framework with the students to express themselves freely without fear of being judged.

Initially, four 8th and 9th grade students, aged 13–15, joined the discussion group named "Gender and Society" for weekly meetings. The initial goal of the meetings was to create a safe space to explore and reflect upon questions related to gender and sexual orientation (this specific topic will be covered in Section 4). The four students involved knew they were welcome to share their thoughts and ideas with their teachers and classmates during weekly student councils. They could also offer suggestions to ensure that all students in the class felt welcome regardless of their gender identity or sexual orientation. The "Gender and Society" group presented their work to the rest of the class1.

Neither the direction team nor the teachers participated in these meetings. The fact that the intern was a philosophy graduate with a good knowledge of gender issues may have contributed to creating such a safe space. The second author of this paper, appointed research assistant at the beginning of the action research, is a Master's student in Gender studies, working on the sociology of gender and education.

In early 2022, the group's discussions became less active, as if the initial goal had, at least to some extent, been reached—all the students feeling comfortable enough to share their concerns and thoughts with their peers and teachers. The school's founder then proposed holding a debriefing meeting to reflect upon what they had learned. At this stage, one of the primary outcomes was the students' realization that (1) gender is one facet of identity but not the only one, and (2) that it is possible to explore these questions without

necessarily opting for definitive labels. This could have been the last session. However, a question eventually arose about what could be done to extend this experience to other schools. Informed about research projects involving children or teenagers as co-researchers, the group participants enthusiastically agreed to launch a study. This decision instilled a new dynamic in the working group and marked a significant turning point in the project.

#### *3.2. Procedure*

The new research group was composed of the four students of the working group, the intern who had facilitated it, a new pupil (age 14) who had joined the school in the meantime, and the first and second author of this paper. During weekly meetings of approximately one hour, the research project was elaborated: definition of a research topic, design, and methodology. Initially, the adolescents were mostly thinking about conducting interviews with other adolescents outside of school to question their perceptions of identity. The main principles of qualitative analysis were briefly explained to them. Once they realized that this approach implied a transcription and analysis of the interviews, they opted for conducting an online survey which was more realistic from a practical point of view, as the end of the school year was nearing.

New questions arose: How do you design a questionnaire? How do you frame the questions? In what order? To what extent can you ask personal questions without risking the participants leaving the survey without completing it (e.g., about gender or sexuality)? All those questions mostly came from the adolescents themselves. During the weekly meetings, they thought about how they would structure the survey and formulate the questions and wondered what would be interesting to ask to collect interesting and relevant data.

Once the questionnaire was ready, we tested a pilot version on a few adolescents outside school. We requested the co-researchers to ask someone they knew to fill in the questionnaire and to give feedback, especially in case something was unclear so that we could make changes to the survey before its dissemination. The corrected questionnaire version was tested among the 26 middle school students at Lab School Paris. After some minor formal changes, the co-researchers decided to conduct a survey using a snowball sampling method: all the co-researchers sent the survey link to as many people as possible; we also sent the survey link to middle schools and high schools found in the French national education Ministry website.

At the end of the school year (June 2022), two meetings were held with the third author of this paper to explain to the co-researchers how to analyze the survey data with statistics; debriefing sessions were also organized so that the students could share their impressions and feedback about the whole research process: what they had learned throughout the process, whether it had changed them and in which way, etc.

Qualitative data –from recordings of the meetings and interviews, as well as notes from the sessions—was analyzed to determine whether participating in this research project had served the purpose of fostering adolescents' 21st-century skills. The research assistant conducted the interviews during the summer of 2022 in individual zoom meetings or phone calls. The questions followed an interview guide constructed by the first author of this paper. The interviews were relatively short (10–15 min), and we discussed at the end of the interviews which information could be shared publicly in the case that some information was confidential.

Quantitative data from the online survey conducted by the co-researchers in study 1 is presented in Section 4 (study 2).

#### *3.3. Results*

The qualitative data was examined using the Activity Theory framework: a content analysis was performed on the relevant sections of the interviews and focus groups on understanding how they felt, what they learned, and the kind of change the project had brought up from their point of view. In this section, we only consider the adolescents' voices, although the whole process has included regular formal and informal sharing of reflections among adults throughout the project.

3.3.1. Artifacts and Division of Labor: Learning and Contributing According to Each One's Expertise

The students shared their thoughts about what they had learned, which aspects of the project most interested them, and in which ways they had contributed:

*[I] feel like I participated in a little bit of everything too, which is pretty nice; it allows me to have a little bit of experience, see a little bit of the whole process.*

Through the research process, they discovered how to build a survey, participated in its dissemination, and began to get some insight into statistical analysis of the data thus produced:

*I participated in finding out what we were going to do; I also participated in a radio show to disseminate the survey. And I also helped to find the questions for the* survey *[*... *].*

*[I have acquired] the skills to form a* survey *and the skills of research work.*

*I think [what I participated in the most] was the questions when we wrote them because that's what we mostly did [*... *] because we weren't going to do the statistics, obviously, and writing the article is more complicated. We couldn't do it ourselves, so that's what we could do the easiest.*

They also recognize how their personal experience would benefit the whole group:

*I think [I contributed with my] perspectives because we all have different views, my experiences from living in New York, I could contribute with that experience, and so by, like, those questions in the survey, I felt like I could contribute with. And also, by being able to share this survey with people I knew all around the world.*

3.3.2. Rules and Social Relationships within the Community

All the participants showed appreciation for the quality of the relationships among the students who participated in the research and in the broader community: they highlighted how much they felt accepted, regardless of their different identities, and how free they felt to express their points of view.

*I felt like I was with people who kind of understood my vision of things. And they didn't impose their opinion on me, so we had civic conversations about it, which was pretty cool. It's not just about gender issues; it's about listening to each other's opinions instead of saying, "no, you're wrong, and I'm right". There was an atmosphere of caring in the group that was quite nice.*

*I experienced an openness to express my own opinions [*... *]. It's a friendly atmosphere where there can be conflicting views, yet it's scarce in life in general! There are a lot of opinions, pros, and cons, there are too many opinions, and it messes up everything, everywhere.*

This feedback points out that they felt it was possible to express opinions without being judged. The space of discussion provided by the explicit rules within the research group—confidentiality and absence of judgment—made them feel comfortable. That feeling allowed them to elaborate their thoughts and to gain from others' perspectives:

*It was a very good environment; it was a very open environment. I feel like we could all really express our positive and negative thoughts. I also felt like I could build my thoughts onto others, and others could build their thoughts onto mine, so we were just helping each other and supporting each other.*

They also felt supported by the adult community in this endeavor:

*I found them [the adults] very open-minded about all these issues [*... *] they were just there to help us put the thing together, because we were kind of the ones formulating* *the thing, doing the thing, and they were there to guide us, to prevent us from getting overwhelmed. I thought it was pretty cool.*

*I also felt it was nice to have adults in the group to like, guide us and show us how to have a formal survey and guide us into those conversations.*

Not only did the co-researchers benefit from the project, but also, at various stages, we communicated with the rest of the school ecosystem—students, teachers, and parents –whether through discussions with the whole class or the presentation of the first results. This allowed conversations about identity and gender identity within the school with adolescents who were not part of the research group, although some participants expressed their regrets about not having more diversity among the students participating in the research project:

*I would hear other people talking about their relationships with their parents in terms of their sexuality or gender, and I would see how things were going in other families, and that would allow me to see a little bit how I could react to them too. [*... *] Because my parents didn't talk about it either, until very recently.*

*I think it would have been nice if we had tried not to include but like engage other students more so we could have gotten more into their perspectives. I think it would have been interesting to understand how the project would have affected them, but yeah, it just pushed me to have discussions with other people in the school. [*... *] We were all already pretty close friends, we were all LGBTQ, and I think it would have been beneficial if we've had at least like cishet teenagers or just get their perspective, or a person of color as well because we're all white so, yeah, yeah.*

3.3.3. Reaching the Object: The Point of View of the Subjects

During the final discussions, we explained to the student co-researchers that a critical feature of this project was also to create different relations and collaborations between young people and adults inside the school, to explore new approaches to teaching and learning, and to foster skills and abilities that we considered necessary more than ever in the current context, trusting them with responsibilities and giving them autonomy. We asked them for feedback and how we could improve the process in the future.

*Actually [*... *] you weren't directing anyone; you were showing paths. [*... *] What I find cool in life is that you can take a word, a text, and there can be thousands of paths [*... *]. And the goal of adults in life, I think, is to support selecting paths and to guide as much as possible.*

*You were super open, super ok to talk about this kind of subject, you listened and everything, you didn't try to distort, it was cool to talk, we felt that there was no judgment and that we could speak freely and say what we thought and everything. [*... *] There were no questions that implied the answer or were biased. [*... *] I didn't feel guided or influenced to say answers that weren't my own.*

Reflexivity and critical thinking emerged while talking about how the setting could be improved:

*Maybe to have a bigger group of adolescents, a slightly more varied group, because we were all very similar in many ways [*... *] and it would have been nice to get a few other teenagers that could add their perspectives, I guess.*

*I think it would be interesting to ask others in the class like what parts of their identities are important 'cause every single one of us in the group is LGBTQ* ... *[Laughs] Yeah, we need an opinion from a straight person!*

*Maybe not do it at the end of the year, because at the end we were too much in a hurry [*... *] do it at a time when we can help work on it.*

Some of the feedback also indicated that the project allowed the participant to gain more perspective and understanding of who they were:

*It was like, it's not scary to talk about it, and people are, in fact, nice. Wow. I didn't know that was possible in school [*... *] And also, I've always been super interested in psychology and stuff. And also like social justice, activism, and kind of putting those together, and into a study. I don't know, it felt like [*... *]. I've always had this question What is me? What makes me me? Is it my brain, my consciousness, is it my body, is it* ... *I don't know* ... *[*... *] I have all those questions. This has begun to answer a few things, organized a few things in my head, and kind of made a start somewhere of what makes us us, what makes us an individual.*

3.3.4. Outcomes: Lessons Learned

The adolescent co-researchers acknowledged that this project helped them to become more aware of their social environment, more reflexive, and more open to others.

(student) *Identities are also a pretty vast subject; it doesn't stop where we defined it; there's still a lot more to talk about. [*... *] I always knew that it [identity] was much more than my gender and what I look like, identity; identity is much more than that.*

(adult researcher) *Did it change during this process?*

(student) *Yeah, it kind of expanded.*

They felt that they could share their views on identity more freely, both with their parents/friends and themselves and even publicly, at a conference or in the media. One of them mentioned at the end of the academic year that talking openly within the group allowed him to better identify his feelings as an LGBTQ+ adolescent.

*Until I was 12, I knew the words LGBTQ+, but I didn't know what they meant. So I couldn't put what I was feeling into words, so obviously, it was a bit complicated for me, with my parents, and with regard to myself.*

The data they collected helped them learn about themselves through others and had positive effects on their social well-being:

*Something I realized, reasonably major, is that I am not alone. People often say to me, yeah, you're not the only adolescent asking yourself this kind of question, there are millions of teenagers asking themselves this question, but it's all very well to talk about it. Still, when you realize that all the people took part in the survey, you say to yourself, "well, yeah, I'm not the only human being on Earth asking myself these kinds of questions", you feel less alone. There it was concrete; you see the answers of the people.*

Gaining self-confidence through the research project seems partly related to the fact that this study enforced the co-researcher's ability to pay attention to themselves and others at the same time, without depreciating any of them through comparisons:

*I think [what I learned the most is] diversity. The different stages of development we're at, just how we all navigate our identities completely differently, even if we are at the exact same age. So just like, looking at the responses, it was just really interesting to see that some people had part of their identities that were way more developed than mine, but other parts that were less developed. It was really interesting to see what parts were the most important.*

Acknowledging individual differences also fostered empathy towards their peers:

*[The aim of such co-research] is not just getting to know ourselves better but to understand others better, to see others' perspectives. I think that's really what I gained out of this, other people's perspectives, and just trying to understand how people do that because I know myself. I know how I do things, and I think it's really beneficial to gain empathy and compassion to understand someone differently.*

Participating in this project was considered stimulating and made the students proud, as the number of participants in the survey exceeded their expectations:

*Look at that, I made that, all those people, most of them I don't even know!*

Finally, concrete propositions for new rules inside the school community arose beyond exchanging ideas during the project. Although democratic participation and openness to differences are already part of the Lab School Paris' culture, the students contributed to making the school more inclusive by officially acknowledging and welcoming gender diversity by asking all their classmates by which pronoun(s) and the name they wished to be addressed:

*Introducing yourself with your pronouns, yes, I think it's very important! [*... *] It would be nice to do an introduction sheet with your name, the name you'd like to be called by, it's safe to be called in class, with your parents* ... *The pronouns you'd like to be used in class*... *(*... *) Yeah, it's starting to become the norm.* [Laughing at people from "old generations" identifying as girls or boys.]

This practice will be introduced at the beginning of each year among middle school students at Lab School Paris.

These results will be discussed along with study 2 results.

#### **4. Study 2: Construction of Identity in Adolescence**

As mentioned previously, this research was initiated when students started meeting in school to discuss gender and sexual orientation issues that they were confronted with, and that caused discomfort to some of them. With time, the discussion topic enlarged to identity formation, and the research group designed a survey that questioned dimensions of identity that the adolescents perceived as most important. This section presents the results of these questions from the survey.

The survey also included questions that go beyond the scope of the present paper and will be presented in a subsequent article, such as the Consciousness of one's responsibility scale (Hagège et al. 2021) and an adapted version of the Cantril ladder of satisfaction with life (Levin and Currie 2014).

All co-researcher students identified as LGBTQ+ and were most interested in the topic of gender identity and diversity, although such an interest is growing in society and research (Perry et al. 2019; Rubin et al. 2020).

Gaining a better knowledge of gender identity is particularly important in the case of adolescents who identify as non-binary, a-gender, or genderqueer, as little is known about them (Jones et al. 2016), or their experiences of schooling (Paechter et al. 2021). Most studies about non-binary adolescents focus on social background and mental health and indicate that they are particularly vulnerable, with high rates of depression, anxiety, and suicidal ideation, and risk of experiencing more abuse and victimization than cisgender people (Chew et al. 2020; Jackson et al. 2022; Van der Vaart et al. 2022; Pullen Sansfaçon et al. 2020; Richards et al. 2016). On the other hand, gender self-acceptance (i.e., being satisfied with one's self-defined gender identity) is negatively associated with stress and positively associated with life satisfaction and perceived academic achievement, which confirms the importance of the recognition of gender diversity and of cultivating gender-identity safe school environments (Day et al. 2018; Watson et al. 2021).

#### *4.1. Participants*

The participants included 1210 middle school and high school pupils, aged 11 to 18 (M = 15.54, SD = 1.71). Participants' self-identified gender was female (60.2%), male (32%), non-binary (5.1%), and subjects indecisive about their gender (2.3%). Individuals were described as non-binary when they did not self-categorize as exclusively female or male but as either the combination of the two or as something else, following Galupo et al. (2017) and Hyde et al. (2019). Five subjects chose not to answer the question relative to gender identity and were excluded from further analyses, including the gender identity variable.

#### *4.2. Measures*

The online survey assessed several sets of information:

Social and demographic information: The participants indicated their age, living environment (small to medium city/large city), school grade, parents' occupation, nationality, and religion. The coders defined family socioeconomic status (SES) based on the participants' description of their parent's occupations. Then they assigned values to the rank of the occupation type resulting in lower, middle, and upper SES, following the 2020 INSEE categories and through a procedure similar to Lignier and Pagis (2017).

Dimensions important to identity were measured from six questions about how important the following dimensions were about their identity: (1) leisure activities, (2) religion, (3) politics/activism, (4) cultural origin, (5) gender, and (6) sexual orientation. The questions were rated on a 7-point Likert scale from unimportant to very important.

#### *4.3. Procedure*

The survey was administered online. The first author contacted middle and high schools across France. All the co-researchers also disseminated information using e-mails, newsletters, and social media. Respondents were informed about the research aims and data confidentiality and provided informed consent.

Ethical approval procedures are not yet systematically required in educational science in France for non-interventional studies such as surveys (Claudot et al. 2009). New approval procedures are gradually implemented, but not all institutions have the adequate infrastructure to apply for formal approval before any research (Carvallo 2019). We, therefore, submitted the present research project to two researchers from Swiss institutions (the Haute École pédagogique du Valais and the Centre interfacultaire en Droits de l'enfant, université de Genève) as well as a deontologist/ethic officer from the French Agence de biomédecine, who gave their approval for the study.

#### *4.4. Data Analysis*

Here we report the analysis of the six dimensions important for identity. How important leisure activities, politics, religion, cultural origin, sexual orientation, and gender are important to identity formation was analyzed as a function of gender, age, living environment, and SES.

These dimensions were analyzed by a 4 × 3 × 4 × 2 repeated measures ANOVA using SPSS statistics (SPSS Inc. Chicago, Illinois, United States of America), with gender identity (male/female/non-binary/indecisive), SES (upper/middle/low), age (11–12/13–14/15– 16/17–18) and living environment (small to medium city/large city) as intrasubject factors. Greenhouse-Geisser correction was applied to *p* values associated with multiple degrees of freedom. Paired *t*-tests were used for 2 × 2 comparisons.

#### *4.5. Results*

Table 1 reports the sociodemographic characteristics of the 1210 participants. Of those who reported their parents' occupation (97.4%), 47.3% came from upper SES, 32.1% from lower SES, and 18.1% from middle SES. The sample lived in small to medium cities (64.3%) or large cities (35.5%).

All the dimensions of identity did not receive the same ratings of importance, F(5, 5480) = 18.51, *p* < .001. As described below, the effect of dimension interacted with gender, age, living environment, and SES.

Gender identity. Dimension of importance interacted with gender identity, F(15, 5480) = 3.66, *p* < .001. Gender identity had a significant effect on the dimensions of leisure activities (F(3, 1096) = 2.86, *p* = .03), politics (F(3, 1096) = 5.67, *p* < .001), sexual orientation (F(3, 1096) = 7.72, *p* < .001) and gender (F(3, 1096) = 5.67, *p* < .001), whereas there was no effect of gender identity on the dimensions of religion and cultural origin (Figure 3). Post-hoc comparisons showed that the dimensions of sexual orientation and

gender were more important to non-binary than male and female subjects (all *p* < .001). Sexual orientation was also more important for indecisive than for female (*p* = .02).

**Table 1.** Sociodemographic characteristics.


**Figure 3.** Mean (± SEM) importance for leisure activities, politics, religion, cultural origin, sexual orientation, and gender as a function of gender identity.

The dimension of politics was more important to non-binary than to male and female (respectively, *p* < .001 and *p* = .003) as well as more important to female than male (*p* = .016). Leisure activities were more important to male than female and non-binary (respectively, *p* < .001 and *p* = .01).

The most important dimensions for indecisive, male and female, were leisure activities (see Table 2 for the associated *p* values), whereas non-binary rated gender, politics, sexual orientation, and leisure activities as the most important dimensions. Religion was the least important dimension for all groups.

Importance interacted significantly with SES (F(10, 5480) = 2.07, *p* = .023). SES impacted the dimension of religion (F(2, 1096) = 3.30, *p* = .03) more importance given to religion in lower SES than upper SES (*p* = .01). There was no effect of SES on any other dimension.


**Table 2.** *p* values associated with paired *t*-tests resulting from within-dimension comparisons for each gender identity.

Age interacted with Dimension, F(15, 5480) = 2.12, *p* = .01, with a significant effect of age for sexual orientation (F(3, 1096) = 3.13 *p* = .02) and gender (F(3, 1096) = 3.04, *p* = .028) dimensions only. The youngest group (aged 11–12) rated both sexual orientation and gender less important than the 13–14 years old (respectively, *p* = .01 and *p* = .006), the 15–16 (respectively, *p* = .003 and *p* = .006) and the 17–18 years old (respectively, *p* = .01 and *p* = .04).

Dimension interacted with living environment F(5, 5480) = 2.74, *p* = .028, with the effect of living environment significant for religion (F(1, 1096) = 4.26, *p* = .04) and leisure activities dimensions (F(1, 1096) = 3.93, *p* = .048) only. Religion was more important in large cities than in small to medium cities. Leisure activities were more important in small to medium cities than in large cities.

#### **5. Discussion**

The current study primarily investigated new ways of fostering the development of 21st-century skills among adolescents through collaborative action research focusing on identity formation. In this section, we will discuss the process of action research, the role of school in fostering identity formation, and skills related to a harmonious and prosperous development into adulthood (study 1). In discussing the various dimensions that adolescents considered most important in defining their identity (study 2), we will focus on the sociodemographic factors related to these dimensions and on elements that are directly relevant to the primary object of our study. We will conclude with the limitations and recommendations for further research.

#### *5.1. Looking Back at the Action Research Process*

Our research process shared all the characteristic features of action research: immersion of the researchers in the situation; work unfolding in response to a specific situation and not to the researcher's requirements; questions and problems emerging from the local context; building of descriptions and theoretical frameworks within the context; iteration and tests within the situation; close collaboration between researchers and actors (Holwell 2004). However, it went further than usual action research, classically defined as "a form of selfreflective inquiry undertaken by participants in social situations to improve the rationality and justice of their own practices, their understanding of these practices, and the situations in which the practices are carried out" (Carr and Kemmis 1986, p. 162).

A specific feature of our research is the coexistence of two intertwined levels: the students' needs that called for intervention and the decision of the educational team to launch a collaborative action research, not only as an answer to those needs but also as a means to reach an even higher target. Indeed, although the project originated from the students' concerns about gender identity, the implemented setting made it possible to work beyond the initial problem of developing 21st-century skills, as if feeding two birds with one seed.

However, the level of implication differed depending on the stakeholders' statuses. Resorting to the stakeholders groups model proposed by Stringer and Ortiz Aragon (2021), it can be said that the primary stakeholders' goal—the students—was to get a better understanding of their own identities as adolescents, whereas the secondary stakeholders' goal—the researchers and the educational team—was to explore new ways of fostering 21st-century skills among adolescents through the research process. As represented in Figure 4, the primary stakeholders' goal is embedded in the second goal: having a better understanding of adolescents' identities can indeed help find ways to foster their 21stcentury skills.

**Figure 4.** Level of involvement of the participants in the two studies (based on Stringer and Ortiz Aragon's stakeholders groups model, 2021).

Action research is a dynamic process involving recurring cycles of activity, sometimes also called self-reflective spiral of cycles—e.g., plan, act, observe, reflect (Kemmis et al. 1988, 2014; Nazari 2021); thinking, planning, doing, and evaluating (Bilorusky 2021)—, and characterized by principles of participation, iteration, inventiveness, and emergence (Burns and McPherson 2017). Figure 5 describes the two cycles that made up our journey: the first one, from October 2021 to January 2022, paved the way for the actual action research cycle, from February to July 2022.

**Figure 5.** Summary of the activity cycles in our action research, inspired by Kemmis and McTaggert's cycle (2014).

Facilitating identity exploration is one of the roles of the school (Denney 2022; Flum and Kaplan 2012; Roeser et al. 2000). While all the participants were not equally involved in the discussions, each one expressed their appreciation of the framework provided by the project. However, it sometimes became complicated to distinguish clearly between the inputs arising from the initial "Gender and Society" group and what specifically came up from the research process. However, the discussions and the research design provided opportunities to reflect and learn, find significance in their and others' contributions, and feel more empowered or more aware than before the beginning of the project. Using accurate data to work on real-world problems that concern them personally has successfully fostered motivation and engagement in young people as co-researchers (Jacquez et al. 2020). Throughout the process, we followed Kaplan and colleagues' four design-based principles to guide teachers in facilitating student identity exploration (2014): (1) promoting personally relevant topics and issues concerning students' daily lives; (2) triggering identity exploration through personal reflection; (3) maintaining a safe school environment; (4) scaffolding exploratory activities to facilitate students through their identity's exploration.

All the adults who took part in the process were also aware that they could, as role models, play a critical role in providing templates for young people to develop their identities (Harter 2012; Schwartz and Petrova 2018). Lab School Paris' pedagogical approach seeks to promote the students' social and emotional development in various ways (Haag and Martin 2021). This pilot project aimed to support further the students' selfdetermined motivation, self-efficacy, and engagement. This is coherent with other studies showing that educational contexts encourage positive civic outcomes, which promote supportive environments for identity exploration while offering critical and analytic awareness of societal issues (Adams and Fitch 1983; Denney 2022; Kaplan et al. 2014; Manganelli et al. 2015). The role of schools as "arenas for exploration and socialization where young people experiment with different roles, values, and relationships" is crucial in the case of "adolescents living in poor and working-class urban communities and deprived of enough opportunities for exploration outside schools" (Abbasi 2016, p. 106).

#### *5.2. Gender Identity and Important Dimensions for Identity*

The results of study 2 informed on dimensions important for adolescents' identity, as defined by the adolescent co-researchers themselves: leisure activities, politics/activism, religion, cultural origin, sexual orientation, and gender. The importance of these dimensions varied according to gender identity, SES, and living environment.

Age had little impact on the importance of identity dimensions. However, the 11– 12 years old rated sexual orientation and gender as less critical than older age groups, whereas other dimensions did not vary across age. This can be understood because sexual feelings mostly emerge in adolescence, prompting less interest in sexual orientation in late childhood (Diamond and Savin-Williams 2009).

Religion was more important in lower than upper SES and large cities than in small to medium cities. The interpretation of these results calls for caution. How religion is related to development depends on the cultural context, which also depends on various factors such as sociodemographic status and living environments (urban vs. rural) (Good and Willoughby 2007). In further analyses of some variables from our survey that were not considered in the present paper, religiosity should be regarded given its relationship with SES, living environment, and religion (Trommsdorff 2012a). Still, some studies demonstrate considerable variance in adolescents' religious practices and experiences (Smith and Lundquist-Denton 2005). While religion is assumed to be important in adolescent development, no simple generalizations are possible from the literature results (Trommsdorff 2012b).

Leisure activities were important to identity development (Leverson et al. 2012a), although leisure activities bring together a vast set of activities, some having beneficial or detrimental effects (Freire 2013; Shaw et al. 1995; Stattin et al. 2005). Our questionnaire asked how important leisure activities were to identity, with no possibility of explaining which activity was considered. Male participants rated leisure activities more important than nonbinary and female respondents. Previous research has demonstrated substantial differences between male and female in leisure activity choices, with some researchers pointing out that the presentation of leisure activities may be gender stereotypical (Leverson et al. 2012b). Moreover, non-binary young people report barriers in accessing sports practice, resulting in a lower rate of engagement in sports activity (Herrick and Duncan 2018).

The dimension of politics and activism was more important to non-binary than female and male participants, a well-documented effect in the adult non-binary population. For example, more than three-quarters of non-binary adults U.S. citizens reported being registered to vote in 2014 compared to 65% of the U.S. population (James et al. 2016). According to Arnold-Renicker et al. (2020), activism is embraced by non-binary communities to establish their rights and protections. Research has also found an increased interest in political issues among young women in the last 20 years (Briggs 2008), decreasing the gap between male and female. Our results follow this trend, with young women aged 11 to 19 more interested in politics than male participants.

Not only did non-binary participants find the politics and activism dimension more important to identity than male and female participants, but sexual orientation and gender dimensions were also more important to non-binary teenagers than the other groups. Being able to put labels or having words to describe their identity constitutes a turning point for non-binary adolescents (Rankin and Beemyn 2012), who then engage in essential processes of self-reflection and self-education (Bragg et al. 2018). This was evidenced in study one, with co-researchers all identifying as LGBTQ+, but also by study 2 showing that identity dimensions of sexual orientation and gender were significantly more important to non-binary adolescents than other gender groups.

A portion of the participants was indecisive about their gender identity (2.3%). This is not an isolated phenomenon: respondents in an extensive survey of more than 2000 participants, primarily LGBTQ from 15 years old, included about 9% of individuals who did not know how to self-characterize their gender (Richard 2019). Not only can self-categorization be an ongoing process, but its stability can also vary across individuals (Jackson et al. 2022). Self-categorization refers to the capacity to state, describe and articulate one's gender and includes several processes: an internal sense of gendered self, gendered attributes, other

people's perception, and knowledge of gender in the world (Jackson et al. 2022). Gender self-categorization is a dynamic process across the lifespan.

The results of these two studies offer complementary insights into the question of identity formation in adolescence. They show that identity is determined by different factors that are inextricably connected and the product of both individual characteristics and the context in which they evolve (Lazzeri 2013). They also show how, through the whole research process—especially the construction of the survey and the analysis of the results, some of which are presented in study 2—adolescents became more aware of those various dimensions, thereby getting a better understanding of who they were. Moreover, the discussions allowed them to reflect critically on social norms and explicit or implicit expectations of the various groups to which they belong (family, friends, school, etc.), giving them tools to analyze complex social situations and to become more assertive in those around them. Finally, the research setting also allowed both adults and young coresearchers to experiment with new pedagogical models and build more horizontal and collaborative relationships. Research indicates that identity exploration in school has been associated with motivation, engagement, positive coping, openness to change, flexible cognition, and meaningful learning (Kaplan et al. 2014); in our research, the participants' attitudes were clearly in line with those observations. It is worth noting that our project took place throughout the school year, allowing a progressive integration of new skills for each participant according to their needs and pace. In the long run, whether this experiment will benefit the participants remains to be investigated.

#### *5.3. Limitations*

A significant limitation of this study was the limited scope of the research setup: it was prompted by the demand of a group of students with very homogeneous characteristics in the sense that all of them identified as LGBTQ+. They all come from privileged social backgrounds, primarily associated with high levels of cultural capital (e.g., teachers, researchers, company directors); most of them have had the opportunity to live in or visit multiple countries, thus opening up internationally. Three of the five adolescent co-researchers had a natural exposure to scientific research, one of their parents or both working in academia. Likewise, in our sample, parents' occupations revealed that the upper classes were overrepresented in the collected data. Adapting this setting to a traditional school environment to achieve generalizable results would require substantial adjustments.

Another limitation lies in the short time frame within which the research has been conducted, as pointed out by the participants themselves, which restricted opportunities to work with the adolescent co-researchers on outcomes and dissemination of the project's result.

#### *5.4. Recommendations for Future Research*

One way to strengthen our findings will be to replicate this action research in various school contexts to build surveys that reflect the identity-related concerns of more diverse social backgrounds and reach more diverse participants. Improvements suggested by the co-researchers should be considered, such as integrating the whole process into the curricular activities and starting earlier in the school year.

Contrasting with action research in single situations, Holwell (2004) insists on the concept of iterability: to address criticisms made to research-action for its lack of generalization, such methodologies should be possible to adapt to different situations. It may not be feasible to replicate this action research on other sites by bringing in several researchers each time. However, a well-planned and rigorous methodology of co-research, with detailed guidelines for the teachers and student co-researchers, could realistically be implemented under the supervision of a research coordinator in the framework of a collaborative project.

Future research should include a valid assessment of the efficiency of action research with adolescent co-researchers in fostering social and emotional skills and engagement and reflexivity, combining both quantitative and qualitative approaches.

Scholars working on identity and adolescence have pointed out since the late 1990s that identity construction is challenging in our society (Baumeister and Muraven 1996)2. Adolescents are vulnerable to risk factors, including emotional, relational, and behavioral problems (Aldam et al. 2019). With adolescents representing a significant proportion of the global population (16% in 2022), understanding and describing identity development during adolescence remains an essential objective for research (Lannegrand-Willems 2012). In the current context, which is particularly anxiogenic due to climate change (Marks et al. 2021; Salomon et al. 2017), special attention must be paid to interventions that can improve young people's personal resources and skills and build resilience for coping with life's adversities and challenges (Taylor 2020). Intervention programs should consider the identity processes of exploration and commitment mobilized by individuals in the investigation of the self, relationships with others, and the social world, to accompany and support the various dimensions of identity construction in adolescence (Lannegrand-Willems 2017), which corresponds to what has been broadly defined earlier as 21st-century skills.

Educational teams must carefully monitor the implementation of interventions. Indeed, there is no such thing as 'one size fits all' (Pressman and Cross 2018). Even protocols generally considered the most rigorous—large-scale randomized controlled trials—are not always conclusive (Lortie-Forgues and Inglis 2019). Moreover, interventions' effects can differ substantially depending on some social and environmental characteristics of their targets. For instance, interventions designed to improve psychological health may not only be ineffective but may even produce detrimental effects in some children, notably the most vulnerable ones, such as deterioration in well-being or increased scores on anxiety or depression scales (Das et al. 2016; Montero-Marin et al. 2022). In the specific case of identity, Lannegrand-Willems and Bosma (2006) point out that in a comparison between three high schools, students' exploration and commitment were higher in the school with students from higher socioeconomic backgrounds. In contrast, Kroger and Marcia (2011) note that differential intervention strategies must be targeted at individuals according to their identity statuses to be efficient. Above all, each child should be supported in a way that respects and fosters their needs and opens a range of possibilities, allowing them to explore various facets of their identity in a secure environment and harmoniously develop both their academic and non-academic skills.

#### **6. Conclusions**

This school intervention fostered engagement and motivation based on a co-research process with adolescents. It led to more comfort, a better understanding of their identities, and, more generally, identity formation in a group of five adolescents. This action research resulted in a survey administered to 1210 adolescents that informed on dimensions important to identity formation. Mostly gender diversity modulated the relative importance of dimensions to identity formation, pointing out the relevance of educational contexts in promoting a supportive environment for identity exploration.

As our world faces environmental and social problems that current solutions cannot address, there is a growing demand in the field of education to explore new ways to address these increasingly complex challenges. Addressing significant issues for the students and opening up opportunities for them to make their voices heard and take responsibility is beneficial in terms of academic success and the development of their social, emotional, and civic skills. Beyond that, fostering 21st-century skills ultimately aims to enable young people to play their role as active citizens in society fully.

**Author Contributions:** Conceptualization, P.H.; methodology, P.H. and T.F.; investigation, P.H. and T.F.; data curation, S.D.; writing—original draft preparation, P.H., T.F. and S.D.; writing—review and editing, P.H., T.F. and S.D.; supervision, P.H. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was partly funded by the Erasmus+ program line for Strategic Partnerships for Higher Education *LabSchoolsEurope* (grant no. 2019-1-DE01-KA203-005003).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data that support the findings of this study are available on request from the corresponding author, (P.A.) The data are not publicly available due to the fact that only part of the whole dataset was analyzed for this paper.

**Acknowledgments:** We wish to thank the co-researchers who participated in this action research: Zero Arrabal, Ulysse Berra, Raphaël Disdet, Lieve Potters and Clementine Shelsky, as well as Léo Ragazzi, who facilitated the working group "Gender and society" and Margot Le Lepvrier for the design work on some figures. We also wish to thank Ulysse Berra and Aurélien Berra for their comments on earlier versions of this paper and for proofreading.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Notes**


#### **References**


Briggs, Jacqueline Ellen. 2008. Young Women and Politics: An Oxymoron? *Journal of Youth Studies* 11: 579–92. [CrossRef]

Burns, Anne, and Pamela McPherson. 2017. Action Research as Iterative Design: Implications for English Language Education Research. In *Reflections on Qualitative Research in Language and Literacy Education*. Educational Linguistics. Edited by Seyyed-Abdolhamid Mirhosseini. Cham: Springer International Publishing, pp. 105–20. [CrossRef]


Harter, Susan. 2012. *The Construction of the Self: Developmental and Sociocultural Perspectives*. New York: Guilford Press.

Heckman, James J., and Tim Kautz. 2012. Hard Evidence on Soft Skills. *Labour Economics* 19: 451–64. [CrossRef] [PubMed]


Levin, K., and C. Currie. 2014. Reliability and Validity of an Adapted Version of the Cantril Ladder for Use with Adolescent Samples. *Social Indicators Research* 119: 1047–63. [CrossRef]

Lignier, Wilfried, and Julie Pagis. 2017. *L'enfance de l'ordre: Comment Les Enfants Perçoivent Le Monde Social*. Liber. Paris: Seuil.

Lortie-Forgues, Hugues, and Matthew Inglis. 2019. Rigorous Large-Scale Educational RCTs Are Often Uninformative: Should We Be Concerned? *Educational Researcher* 48: 158–66. [CrossRef]


## *Article* **Are STEM Students Creative Thinkers?**

**Christabel Borg Preca 1,\*, Leonie Baldacchino 2, Marie Briguglio <sup>3</sup> and Margaret Mangion <sup>2</sup>**


**Abstract:** Scholarly research has increasingly examined the role of STEM (Science, Technology, Engineering, and Mathematics) education, and that of creativity as a transversal skill. However, far fewer studies have investigated the relationship between the two, particularly in secondary-school contexts, and they have obtained inconsistent results. This paper contributes to the literature by asking: *To what extent is studying STEM associated with higher levels of creativity in a secondary-school context?* The study utilises a pre-existing dataset gathered in Malta (EU) from some 400 students aged between 11 and 16 years old. It yields information on both the engagement in STEM (measured by exposure to STEM chosen by students as optional subjects, and the enjoyment of STEM considered by students to be their favourite subjects), as well as creativity levels (measured by Divergent Thinking performance on Alternate Uses Tests). Correlation analysis revealed a strong positive link between the two phenomena, lending support to the notion that STEM students tend to be more creative than other students. Using regression analysis, a model is estimated to identify the possible effects of engaging in STEM subjects on creativity, once the other co-determinants of creativity are controlled. The results indicate that both the exposure to STEM subject/s and enjoyment thereof significantly and positively predict creativity, even after controlling for the other possible determinants of creativity (such as age, gender, parental education, and participation in creative activities). These findings offer encouraging insights into 21st century education and for curriculum development as they suggest that, in addition to having value in their own right, STEM subjects can contribute to the development of creativity in young people.

**Keywords:** STEM; creativity; divergent thinking; 21st century skills; secondary education

#### **1. Introduction**

In recent years, there have been considerable efforts among educators and policymakers to promote science, technology, engineering, and mathematics (STEM) subjects and careers, as they are considered by many to play a key role in boosting economic competitiveness, productivity, and innovation (Collard and Looney 2014; European Commission 2009; Runco 2010). STEM education is often linked to economic competitiveness, as it is believed to contribute to workforce-ready individuals (Bentley et al. 2022). Indeed, STEM-related jobs are among the fastest-growing jobs in the labour market. Within the European Union (EU), the number of people working in STEM-related fields increased by 12% between 2003 and 2013. To date, STEM accounts for 7% of the total occupations within the EU. STEM occupations are also among the highest-paying sectors (Melguizo and Wolniak 2012) and are associated with improved economic, social, and personal wellbeing (Beede et al. 2011).

In parallel, there has been an increasing understanding of the importance of creativity as a transversal 21st century skill (Partnership for 21st Century Learning 2015; Voogt and Roblin 2012). Creativity typically refers to the generation of ideas that are novel, original, or unique, as well useful, appropriate, or contextually relevant (Amabile et al. 1996; El Murad and West 2004; Runco and Jaeger 2012). Creativity is increasingly considered to be a crucial

**Citation:** Borg Preca, Christabel, Leonie Baldacchino, Marie Briguglio, and Margaret Mangion. 2023. Are STEM Students Creative Thinkers? *Journal of Intelligence* 11: 106. https://doi.org/10.3390/ jintelligence11060106

Received: 31 December 2022 Revised: 25 May 2023 Accepted: 27 May 2023 Published: 1 June 2023

**Copyright:** © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

competency (Marrone et al. 2022) for economic and social development. Yet, while STEM and creativity are both observed as important skill sets, a question remains as to whether one comes at the cost of the other.

Traditionally, the Arts and Humanities were considered to be linked to creativity and, in particular, divergent thinking (DT), which involves the generation of multiple ideas or solutions related to a task or situation (Cropley 2006). The Sciences, on the other hand, tend to be associated more with convergent thinking, in the sense of tackling problems by providing a single optimal solution (Cropley 2006). Indeed, during his influential 'Rede Lecture', titled 'The Two Cultures', in 1959, scientist and novelist C.P. Snow highlighted a division between the Humanities and Sciences. This has since triggered scholarly curiosity as to whether there is a difference in thinking styles across domains (Furnham et al. 2011).

A number of researchers have since sought to examine the links between STEM subjects and creativity, with inconsistent results. Studies by Furnham et al. (2011), van Broekhoven et al. (2020), and Williamson (2011) all indicate that there are no differences in creative thinking abilities between students from different disciplines. Stylianidou et al. (2018), on the other hand, suggest that science education in the early years may enhance creativity. There therefore remains an unresolved question regarding whether there is any relationship, positive or negative, between engaging in STEM education and creativity.

#### **2. Theoretical Background**

#### *2.1. STEM Education*

STEM education is a complex and poorly defined field (Li et al. 2020) that has been the focus of an increasing body of research since 2000 (Kayan-Fadlelmula et al. 2022; Li et al. 2020; Li and Xiao 2022). What started as an acronym, bringing together the four core subjects of Science, Technology, Engineering, and Mathematics, has changed considerably over the years (Dare et al. 2019; Martín-Páez et al. 2019), to the extent that Breiner et al. (2012) argue that STEM education holds different meanings to different stakeholders. Tytler (2020) notes that the STEM acronym is underpinned by the notion that it collectively comprises "a coherent package of subjects that cover the knowledge and skills around the sciences, applied sciences, and the digital world that constitute the driving force towards a post-industrial global future and the future wealth of countries" (p. 22).

Of particular interest to this study is the teaching of STEM in secondary-school contexts, where different approaches to STEM education have been observed. Kennedy and Odell (2014) identify a number of STEM initiatives within the US education system, targeting students from secondary-level education to high-school-level (14–18 years old) students. These include the T-STEM programme that targets students from 6th to 12th grades, providing a hands-on approach to STEM education and linking students with industry. This programme has not only been associated with greater academic success, but also with fewer reports of disruptive behaviour. Reiss and Mujtaba (2017) highlight that students who are exposed to careers education in STEM, such as increasing their awareness of the transferability of STEM skills or the higher income associated with STEM careers, have an increased likelihood that they will choose STEM subjects at a post-secondary-education level. However, Kennedy et al. (2020) show that students' attitudes towards STEM subjects, and their likelihood of pursuing STEM subjects at a post-secondary level, vary greatly between the different subjects, depending on their perceived relevance, personal usefulness, enjoyability, and self-efficacy.

By any definition, STEM education is widely believed to hold great potential for teaching students how to tackle complex problems that dominate our world, such as climate change, the depletion of natural resources, and public health crises (Merrill and Daugherty 2010). It enables students to develop skills in communication, critical thinking, problem-solving, and the analysis of data, while developing an understanding of processes and provision of solutions based on knowledge (Froschauer 2015; Jang 2016). Kennedy and Odell (2014) highlight how engineering can add a dynamic element to the teaching of Science and Mathematics that promotes STEM literacy, defined as the ability to engage with

and communicate complex problems with audiences. Consequently, STEM is considered a critical domain of knowledge that is linked to the success of students and graduates in the 21st century (Sanders 2009).

#### *2.2. Creativity Education*

At the same time, there is broad agreement in academia and industry about the need for skills that are transversal, multidimensional, and associated with higher-order cognitive processes, in order to deal with problems that are complex and diverse (Brown et al. 2010). Foremost, among such skills is creativity (Partnership for 21st Century Learning 2015). A number of studies focus on nurturing creativity in educational contexts. For instance, Briguglio et al. (2022) identify the impact of a school-based educational programme in secondaryschool contexts, finding that, while creativity levels generally decreased between the start and end of the school year, students exposed to a programme with creative practitioners fared better. The same study also found that parental education has a positive effect on creativity.

A key theme in the literature is the criticism of educational systems for inadequately nurturing transversal skills, particularly creativity (Beghetto and Madison 2022; Darbellay 2022). Corazza et al. (2021) refer to the "Traditional Education System (TES)" where creativity is not only omitted as an explicit objective, but students even tend to be penalised for behaviour associated with creativity, such as mind-wandering, slow responses, or unexpected answers. The authors argue that "creativity ... has today become a democratic necessity" and that "there is today an urgent call for the formal introduction of creativity inside the education system" (p. 72). Yet, it seems safe to say that the introduction of creativity to the education system is generally perceived as an unwelcome disruption and met with resistance. In view of this, Beghetto and Madison (2022) maintain that "it is not students who need to get smarter before they address complex challenges, rather, it is schools that need to get smarter in the kinds of curricular opportunities they provide young people" (p. 11).

#### *2.3. The Relationship between STEM and Creativity*

While there has been considerable scholarly attention paid to both STEM and creativity separately, there is a wide scope to examine the link between the two. Learning methods and thinking styles that prevail in the Sciences have traditionally been believed to differ from the ones in the Arts and Humanities. Specifically, individuals who are attracted to the Sciences are considered to prefer convergent thinking, while those who gravitate towards the Arts and Humanities are assumed to prefer creativity and DT (Furnham et al. 2011). However, empirical studies have contradicted these assumptions. For example, Williamson (2011) found only minor differences in learning styles and no significant differences in problemsolving behaviour among undergraduate students from different domains. Furnham et al. (2011) also found no significant differences in DT fluency, creative judgement, or achievement between Arts and Science undergraduate students. Similar results were obtained by van Broekhoven et al. (2020) who found no significant differences in creativity between university students specialising in the Arts or STEM-related fields.

Furthermore, some authors (e.g., Root-Bernstein et al. 2008) suggest that there is actually a positive link between scientific success and ability within artistic domains, as noted in polymaths who excel in a variety of disciplines, including the Sciences and Arts. Indeed, it may also be argued that creativity can be developed through STEM subjects (Zhbanova 2019). Sánchez et al. (2022) consider creativity to emerge from processes that are essential in Science and Mathematics activities. This suggests that STEM may play an indirect role in the development of creativity by facilitating these processes. In support of this, Schoevers et al. (2020) found that mathematical ability and the ability to generate ideas influenced mathematical creativity among fourth graders. Stylianidou et al. (2018) also found that the focus on problem-solving activity and agency in STEM has a positive impact on student creativity.

Conversely, some authors focus on the effect of creativity on STEM. Henriksen (2014) argues that the link between creativity and the Arts is evident in cases of successful leaders and innovators. Lubart et al. (2022) argue that creativity is fundamentally important in Science, and note that many scientific breakthroughs are the product of creative thinking. Peppler and Wohlwend (2018) note that art and creativity can facilitate STEM engagement by making these subjects appear more approachable and relatable.

It is worth noting that much of the research on the relationship between STEM and creativity in educational settings has been at a tertiary level (van Broekhoven et al. 2020; Furnham et al. 2011; Williamson 2011), possibly because of a greater distinction of the subjects studied and greater clarity concerning the choice of specialisation at this level.

#### *2.4. Synthesis, Research Question, and Hypotheses*

In summary, there seems to be little disagreement in the literature that both STEM subjects and creativity are important skills to be nurtured in an educational context. However, the results are inconsistent as to whether there is a positive effect of nurturing creativity for STEM performance and whether there is a positive effect from engaging in STEM on creativity itself. Moreover, the extant research on the relationship between STEM and creativity in educational settings has largely occurred at post-secondary-, tertiary-, and, to a lesser extent, early school settings, with far less evidence being obtained from a secondary-school level (pre-college age).

Against this backdrop, this study focuses on the first relationship and asks the following research question: To what extent is studying STEM associated with higher levels of creativity in a secondary-school context? More specifically, we set out to assess, against the null hypothesis (H0) of zero association, the alternative hypothesis that students who learn STEM subjects demonstrate a higher level of creativity, as measured by DT. We test this hypothesis in two ways: firstly by assessing whether students who choose STEM subjects demonstrate greater creativity, thereby capturing the effect of exposure to STEM subjects (H1a), and secondly, by assessing whether students who consider STEM subjects to be their favourite subjects exhibit higher levels of creativity, thereby capturing the effect of the enjoyment of STEM subjects (H1b). We test our hypotheses in the milieu of a Maltese secondary-school environment. This context and the method adopted are described in the following section.

#### **3. Materials and Methods**

#### *3.1. Context of the Study*

The context of this study was Malta, a European Union (EU) small island state with a population of approximately 500,000 people. Compulsory schooling in Malta is between the ages of 5 and 16 years old, and students may attend one of three types of schools, namely, state, church, and independent schools. State schools attract a particular geographic catchment, as stipulated by the boundaries set for the different districts, while church and independent schools attract students from all over the island. Although there are variations in the management and teaching policies across the types of schools, they are all required to follow the same curriculum with a generally homogeneous teaching practice, as regulated by the National Education Act (CAP 327).

At the end of Year 8, students (aged 11–13 years) choose a number of optional academic, vocational, or applied subjects that would lead them to a Matriculation and Secondary Education Certificate (MATSEC)—an entry requirement for post-secondary education. At this time, they continue to study core subjects (English, Maltese, Mathematics, Religious Studies/Ethics, and Personal, Social, and Career Development), and they supplement these with optional subjects. These optional subjects include a range of academic, vocational, and applied subjects. Among them, and of particular interest to this study, were those identified as STEM subjects by the Maltese authorities (European Schoolnet 2018), namely, Health and Social Care, Engineering, Technology, Information Technology, Computing, Design

and Technology, Biology, Chemistry, Physics, Mathematics, Graphical Communication, and Agribusiness. All schools in Malta offer STEM-related subjects at the secondary level.

#### *3.2. Data Collection*

To address the research question, this study performed a statistical analysis of preexisting survey data that were gathered in 2016 as part of a larger research project commissioned by the Arts Council Malta (see also Briguglio et al. 2022). Although a few years have elapsed since the data were gathered, there have been no major changes to the curriculum related to STEM or creativity since then. The topics remain highly relevant to stakeholders in Malta, including educators, researchers, and policymakers. A subset of the data collected was suitable to test this study's hypotheses and was shared with the present authors, following approval by the Arts Council Malta. An overview of the original data collection procedure is provided below for the sake of completeness.

The research sample was that of 400 students, broadly and evenly distributed by gender and aged between 11 and 16 years. The students were recruited from nine secondary schools in Malta (five state, two church, and two independent schools), all of which were originally identified by the Arts Council on the basis that they were beneficiaries of a statefunded creativity programme, they represented different school types, and they agreed to participate in the research. Participation by students in the study was voluntary and entailed the signing of consent forms by their legal guardians. Teachers were provided with written and verbal instructions to ensure that the data collection instruments were administered in a consistent manner. The original data collection was covered by ethics and data protection clearances, and permission to use the unpublished secondary data as well as ethics and data protection clearances were obtained by the present authors.

The data were gathered by means of a paper-based questionnaire, distributed by teachers to students in their classrooms at the start of the school year, in October 2016. It is pertinent to note that this constituted an initial round of data collection prior to the implementation of the above-mentioned creativity funding programme. It took students an average of 20 min to complete the entire questionnaire. The content of the questionnaire included, inter alia, i. measures of creativity, ii. measures of engagement in STEM, iii. measures of engagement in creative activities, and iv. demographic and other data.

As in other studies, creativity was operationalised by measuring the capacity to produce new ideas through DT—a key indicator of creativity (Doron 2016). In this regard, Guilford et al.'s (1960) Alternate Uses Test (AUT) was employed, involving the generation of multiple ideas or solutions to a given task or problem. Two such tests were assigned at the very beginning of the questionnaire, where participants were asked to think of as many uses as they could for a sock and a box, within a time limit of three minutes per object.

The data related to STEM studies were gathered in the original survey by asking students the following questions: 'Which optional subjects have you chosen at school? (circle any)' (to measure exposure to STEM subjects) and 'Think of ALL your school subjects. Which is your favourite?' (to measure enjoyment of STEM subjects). The questionnaire also gathered data related to the participants' background, such as age, gender, and parental education, as these were identified in the previous research as relevant co-determinants of creative outcomes (Briguglio et al. 2022; Shah and Gustafsson 2020). As a further control to understand the diversity in creativity, the questionnaire also asked students about their participation in school-based and extra-curricular creative activities. This was measured by means of an adaptation of Batey's (2007) Biographical Inventory of Creative Behaviours (BICB), namely, limiting the options to 15 items known to be relevant in Malta (e.g., writing a short story, producing a picture, or creating a sculpture). Students were asked to mark any activities that they had been involved in. Composite scores of creative activities were then created by adding up the number of items that participants reported engaging in during school hours and during their free time in the preceding month. The number of activities chosen were subsequently collapsed into two indices—one for creative activities conducted at school and one for creative activities conducted in free time.

#### *3.3. Data Preparation and Analysis*

Responses to the AUT DT test were scored by two tenured academics at a highereducation institution, knowledgeable in creativity theory, research, and practice. Indices of creativity were created, including ideational fluency (the number of ideas generated), flexibility (the number of conceptual categories into which the ideas may fit), elaboration (the amount of detail provided to communicate the ideas), originality (the uniqueness of the ideas) (e.g., Gu et al. 2019; Runco 2010; Yi et al. 2015), relevance, appropriateness or feasibility (Baruah et al. 2021), and novelty (Amabile et al. 1996; El Murad and West 2004; Runco and Jaeger 2012). Inter-rater reliability, which was estimated by Pearson's Bivariate Correlations, was strong (*r* > .7; *p* < .001) or moderate (*r* = .5–.7, *p* < .01), and all points of divergence between the raters were resolved.

On the basis of the data available, the researchers were able to create an index of exposure to STEM subjects and another of enjoyment of STEM subjects. There being no a priori reason to weight the STEM subjects, all of them were provided with equal weighting. To compress and simplify the data by extracting the most important information from a data table (Abdi and Williams 2010), Principal Component Analysis (PCA) was performed, thereby deriving an overall score for creativity (i.e., dependent variable) that encompassed all the above-mentioned creativity indices for both items in the DT test (sock and box).

Table 1 presents the dataset that was used for this study. As can be observed, the 400 students were aged between 11 and 16 years with a mean age of just over 13.5 years. Just over half the sample students were female, and just over half had parents whose highest level of education was tertiary (university). Students undertook an average of 4 to 5 out of 14 creative activities at school and around 5 out of 14 possible activities in their free time. Their creativity scores ranged from 0 to 1, with a mean of 0.5. Just over two-thirds of the students chose at least one STEM subject as an optional subject with responses ranging from zero to five; however, only around one-third of the students named a STEM subject as their favourite.


**Table 1.** Descriptive statistics.

The Results Section starts with a simple description of the data, assessing the students' engagement in STEM subjects, followed by a basic analysis by gender. A correlation analysis then followed to test the links between engagement in STEM subjects and creative activities or creativity scores. Recognising that correlation does not indicate causality (other variables may have influenced the relationship), Ordinary Least Squares (OLS) regressions were performed in order to estimate the strength of the associations between variables (Hutcheson 1999), parsing out the effect of co-determinants. There are three components of an OLS, namely, "a random component for the response variable, which is assumed

to be normally distributed, a systematic component representing the fixed values of the explanatory variables in terms of a linear function, and finally, a link function which maps the systematic component onto the random component" (Hutcheson 1999, p. 2).

In this study, the response (dependent) variable is the PCA creativity score, the explanatory (independent) variables are the STEM subjects, and the link function includes the important correlates of creativity measures (including age, gender, education level of parents, and creative activity during school hours or free time). Prior to running the regressions, the normality of the dependent variables was tested using the Kolmogorov–Smirnov test. This revealed that the measures of creativity were non-parametric. Logarithmic transformations were therefore conducted prior to using them in the regression models. The statistical analysis was performed using International Business Machines Statistical Package for Social Sciences (IBM SPSS) software.

#### **4. Results**

#### *4.1. Preliminary Analysis*

The data analysis revealed that most students in the sample had some exposure to STEM, as they chose at least one STEM subject as an optional subject (Table 2). The three most popular STEM subjects are Biology (*n* = 111), Physics (*n* = 87), and Graphical Communication (*n* = 80), whereas the least popular are Engineering (*n* = 3), Technology (*n* = 5), and Agribusiness (*n* = 6).

**Table 2.** Students' exposure to STEM subjects.


Note: *N* = 400.

While most students had some exposure to STEM, the same cannot be said regarding the enjoyment of STEM, as 64.8% of the sample (*n* = 259) did not name any STEM subjects as being their favourite subject. One third of the sample (33.7%, *n* = 135) named one STEM subject as their favourite, while only 2.5% of the sample (*n* = 6) named two or three STEM subjects as their favourite (Table 3). The most common favourite STEM subjects were Biology (*n* = 40), Mathematics (*n* = 39), and Computing (*n* = 21), while the least popular were Technology and Engineering (*n* = 0) Health and Social Care (*n* = 1), and Design and Technology (*n* = 1).

**Table 3.** Students' enjoyment of STEM subjects.


Note: *N* = 400.

When comparing students' gender and exposure to STEM, it was evident that STEM subjects were more popular among boys than girls in our sample (Figure 1). Notably, many more girls (38%, *n* = 84) than boys (13.6%, *n* = 24) chose no STEM subjects as optional subjects. Similarly, more girls (27.1%, *n* = 60) than boys (26.1%, *n* = 46) chose only one optional STEM subject. Conversely, more boys (40.9%, *n* = 72; 16.5%, *n* = 29; 2.8%, *n* = 5) than girls (26.2%, *n* = 58; 7.2%, *n* = 16; 0.9%, *n* = 2) chose two, three, or four optional STEM

subjects. A chi-squared test (10, *N* = 399) = 39.31, *p* < .001 confirmed that the association between gender and exposure to STEM subjects was statistically significant.

**Figure 1.** Students' exposure to STEM subjects, distributed by gender. Note: *N* = 400.

When comparing students' gender and their enjoyment of STEM subjects, it was again evident that STEM subjects were appreciated more by boys than girls (Figure 2). Most female students (77.8%, *n* = 172) named no STEM subjects as their favourite, in contrast to half the male students in the sample (49.9%, *n* = 87) who reported that their favourite subject was from the STEM category. A chi-squared test (6, *N* = 399) = 39.93, *p* < .001 confirmed that the association between gender and enjoyment of STEM subjects was statistically significant.

**Figure 2.** Students' enjoyment of STEM subjects, distributed by gender. Note: *N* = 400.

#### *4.2. Correlation Analysis*

Since many of the variables were non-parametric, a Spearman's Rank Order Correlation was performed. As shown in Table 4, a significant positive correlation can be noted between the engagement of students in STEM subjects (be it as a choice of optional subject or as their favourite subject) and creativity scores (measured as a PCA score). There was some support for both H1a and H1b that a positive link existed between the exposure to STEM subjects and creativity (*r* = .264, *p* < .01) and between the enjoyment of STEM subjects and creativity (*r* = .159, *p* < .01).


**Table 4.** Spearman's rank order correlation matrix.

Note: *N* = 400; \* *p* < .05, \*\* *p* < .01.

The correlation results, however, also indicate that there is a significant link between creativity and gender (*r* = −.185, *p* < .01), creativity and parental education (*r* = .239, *p* < .01), and creativity and participation in creative activities performed at school (*r* = .160, *p* < .01) and during free time (*r* = .211, *p* < .01). Therefore, it was difficult to determine whether the link between STEM and creativity was driven by, perhaps, a gender effect (more boys chose STEM, and boys scored better for creativity) or a parental-education effect (highly educated parents were linked to greater STEM exposure and creativity levels). For this purpose, we turned to a regression analysis.

#### *4.3. Regression Analysis*

OLS regressions were performed to identify whether changes in creativity scores may be attributed to the study of STEM subjects. The variables included in the OLS models were selected on the basis of past research, including Briguglio et al. (2022). The multiple regression employed the creativity (PCA scores) as the main dependent variable in order to provide a general view of the effect of STEM subjects on overall creativity. Three models were run. The first employed a base model to forecast the creative outcomes (PCA scores). This model included three key demographic variables (age, gender, and parental education), as well as engagement in creative activities (based on the indices constructed) as predictor variables. The second model added 'STEM Exposure' as an independent variable while the third model replaced this with 'STEM Enjoyment'. Table 5 shows the results of these three models. All Tolerance values were above .10, and the Variance Inflation Factor (VIF) values were less than 10, indicating that there were no multicollinearity issues.

The results displayed in column 1 show that the base model is significant (*R*<sup>2</sup> = 0.120, *F* = 10.486, *p* < .001). Gender was found to be negatively associated with creativity (*B* = −0.041, *p* < .01), which implies that male students outperformed their female counterparts on the DT tests, once all other factors were controlled for. As expected from the previous literature (e.g., Briguglio et al. 2022), the level of education of the students' parents had a positive effect on their creativity (*B* = 0.032, *p* < .001). Participation in creative activities during the students' free time also had a significant positive effect on creativity (*B* = 0.007, *p* < .01), while participation in creative activities at school had no additional significant effect.

The regression was repeated with the addition of 'STEM Exposure' as an independent variable to test H1a. This model, shown in column 2, was significant (*R*<sup>2</sup> = 0.135, *F* = 9.895, *p* < .001) and was improved as a result of adding the independent variable to the base model. This independent variable was observed to have a positive effect on creativity (*B* = 0.017, *p* < .05), which indicates that the more optional STEM subjects students chose, the more likely they were to perform better in terms of creativity. This offers support for H1a.


**Table 5.** OLS regression: predicting individual creativity scores (PCA).

Note: coefficients are shown with standard errors in parenthesis. \*\*\* *p* < .001; \*\* *p* < .01; \* *p* < .05; two-tailed. *N* = 400.

To further test our expectations, the regression was conducted again, this time with 'STEM Enjoyment' as an independent variable to test H1b. As shown in column 3, the model significance was slightly stronger than the previous model (*R*<sup>2</sup> = 0.135, *F* = 9.914, *p* < .001), and it was once again an improvement on the base model. The impact of STEM as a favourite subject/s was positive and significant and twice as large as that of STEM as optional subject/s (*B* = 0.038, *p* < .05). This shows that students who indicated more STEM subjects as their favourites obtained higher creativity scores, thereby supporting H1b.

Taken together, these findings suggest a rejection of the null hypothesis (H0) that stipulates that there is no association between the engagement in STEM learning and creativity. On the contrary, whether measured by the exposure to or enjoyment of STEM subjects, the engagement in STEM subjects was positively associated with creativity, as measured by DT. Furthermore, the results suggest that, among students who considered STEM subjects to be their favourite subjects, creativity scores are even higher. These results are derived after controlling for co-determinants of DT, including age, gender, and parental-education level, as well as engagement in creative activities at school and in their free time.

To test for the robustness of our results, regression analyses were repeated with all the individual DT indices as dependent variables. The analysis confirmed our findings: the propensity to indicate STEM subjects as favourite subjects is positively associated with all the indices. Choosing STEM subjects as optional subjects was positively linked to elaboration (the amount of detail), originality (uniqueness), and novelty, but not significantly related to fluency (the number of ideas generated) or flexibility (the number of conceptual categories into which the ideas may fit). For ease of reference, the results are summarised in Table 6 by displaying the coefficients and respective standard errors of the main variables.


**Table 6.** OLS regression abridged results: predicting creativity by STEM engagement.

Note: † *p* < .10; \* *p* < .05; \*\* *p* < .01; \*\*\* *p* < .001; two-tailed; *N* = 400. Full models are available from the authors upon request.

#### **5. Discussion**

The main aim of this study was to explore the extent to which studying STEM subjects in secondary-school was associated with creativity levels. Although previous research has investigated the relationship between creativity and different domains of knowledge, including the Arts and STEM subjects (e.g., Furnham et al. 2011; Stylianidou et al. 2018; van Broekhoven et al. 2020; Williamson 2011), the results are inconsistent, leaving an unresolved issue regarding the relationship between STEM and creativity. Moreover, previous research on this relationship has largely focused on post-secondary- (Li and Xiao 2022), tertiary- (van Broekhoven et al. 2020; Furnham et al. 2011; Williamson 2011), and, to a lesser extent, early education levels (Stylianidou et al. 2018), leaving a gap in the knowledge concerning this question in the context of secondary-school (pre-college-age students). This study attempted to address the gap in the research by investigating the relationship between students' engagement in STEM at secondary-school and their creativity levels (measured by DT scores). It did this by exploring both the impact of exposure to STEM (choosing STEM subjects as optional subjects/s) and the enjoyment of STEM (considering STEM subjects as a favourite subject/s).

Guided by the extant literature, a parsimonious model was employed that sought to predict creativity by age, gender, parental education, and involvement in creative activities at school and during free time. Subsequently, the model was re-estimated with additional test variables that captured students' exposure to STEM and their enjoyment of STEM subjects. Our key finding is that both exposure to STEM as optional subject/s and enjoyment of STEM as a favourite subject/s significantly and positively predict creativity outcomes, thus offering more support for our hypotheses. We believe that these findings contribute to the literature on the links between STEM and the development of transversal skills (e.g., Froschauer 2015; Sánchez et al. 2022; Stylianidou et al. 2018; Zhbanova 2019). Furthermore, they support the previous research on the complementarity of STEM and creativity (e.g., Root-Bernstein et al. 2008; Sánchez et al. 2022; Schoevers et al. 2020; Stylianidou et al. 2018; Zhbanova 2019).

Further detailed regression analyses revealed that exposure to STEM as optional subjects was significantly associated with overall creativity, elaboration, originality, and novelty, but not with fluency, flexibility, or relevance. On the other hand, students who enjoyed STEM subjects—so much that they considered them their favourite subject/s demonstrated significantly higher creativity levels for all of the six indices (fluency, flexibility, elaboration, originality, novelty, and relevance). Intrinsic motivation—a factor linked with creativity (Amabile 1996) could be at play. The finding hints at the importance of schools offering relevant and motivating STEM experiences that not only entice students to

choose STEM subjects, but also result in their enjoyment of these subjects. Initiatives, such as co-curricular STEM clubs within schools, allow students the flexibility to experiment, explore, and apply STEM to projects they are personally invested in. Initiatives can promote the application of STEM skills, such as science fairs and competitions. As Peppler and Wohlwend (2018) argue, creativity itself can facilitate the engagement in these subjects as it makes subjects intrinsically more approachable and relatable.

This latter notion of engagement is also important in the discourse relating to equity. It is noteworthy that our findings reveal that girls are considerably less likely to study STEM subjects as optional or favourite subject/s in Malta. Many countries have attempted to target this gender gap in STEM education and employment. Common strategies include informal programmes targeting underrepresented groups and industry-led campaigns in formal- and informal-education settings. Initiatives to promote equitable STEM education could look at engaging females through a holistic approach, which includes different stakeholders from the public, private, academic, and tertiary sectors.

Although the relationship between studying STEM subjects and creativity clearly emerges from this study, it is important to consider some of the limitations in the method. The main limitations relate to the identification of the key variables (STEM engagement and creativity), the sampling frame, and the cross-sectional nature of the data.

With the data available, it was not possible to control for students' prior STEM education, nor for any differences in STEM experiences outside school. It was also not possible to control for the quality of the STEM experience that they received—the dataset did not contain any information related to the pedagogical approach in schools, nor was it possible to estimate the models for individual schools. Such issues could be tackled in future research where data collection can permit detailed questions to be asked. On the other hand, we believe that the study offers a novel contribution to distinguishing between the choice of STEM subjects as an optional subject and the enjoyment of STEM subjects as a favourite topic.

With regards to the measurement of creativity, the study employed data that were generated using Guilford et al.'s (1960) AUT as a creativity measure. Although they are not perfectly analogous (Runco 2010), DT tests are widely used by creativity researchers (e.g., Gu et al. 2019) as they provide estimates of creative potential, ideation, and everyday problem-solving tasks. It is well known that the test does not necessarily account for all the skills related to creativity, but that it mainly reveals DT skills. The fact that the test involves a systematic enquiry might resonate more closely with the systematic thinking approach used in scientific enquiry, thereby favouring students of scientific subjects. In subsequent studies, this could be investigated by using different tests and measures of creativity.

A different set of limitations pertain to the sample. The dataset was gathered from schools that opted into a government-funded creativity programme. This suggests that they were already inclined to include topics related to creativity within their curricula or pedagogy, and that students' exposure to creativity may be greater than the exposure of students in other schools. With this said, there was no reason for this to bias the results in relation to the differences observed between STEM students and others. A related limitation pertains to the cross-sectional nature of the data. Future research could adopt a longitudinal approach to explore the causal effects of STEM engagement on creativity over time. This, together with a more nuanced measure of STEM engagement, a wider set of measures for creativity, and a broader range of schools would constitute an improvement to this study.

#### **6. Conclusions**

This study provided evidence that the study of STEM is linked with higher levels of divergent thinking. This was determined through the OLS regression analysis using a sample of 400 students aged between 11 and 16 years. Considering that schools have generally invested in developing their STEM capacity, but are often criticised for inadequately nurturing transversal skills, these findings offer encouraging insights into 21st

century education, as they suggest that, in addition to having value in their own right, STEM subjects may also contribute to the development of creativity in young people.

**Author Contributions:** Conceptualization, C.B.P., L.B., M.B. and M.M.; Methodology, C.B.P., L.B., M.B. and M.M.; Formal analysis, C.B.P., L.B., M.B. and M.M.; Investigation, C.B.P., L.B., M.B. and M.M.; Writing—original draft, C.B.P. and L.B.; Writing—review & editing, L.B., M.B. and M.M.; Supervision, L.B.; Funding acquisition, C.B.P. and L.B. All authors contributed equally to this study. All authors have read and agreed to the published version of the manuscript.

**Funding:** The data utilized in this paper were funded by the Arts Council Malta (project number E16LG94-01). The study was also partially funded by the Endeavour Scholarship Scheme (Malta), which was part-financed by the European Union–European Social Fund (ESF)–Operational Programme II–Cohesion Policy 2014–2020 "Investing in human capital to create more opportunities and promote the well-being of society". Neither of these funding bodies had any role in the analysis or interpretation of the data, in the writing of the paper, or in the decision to submit it for publication.

**Institutional Review Board Statement:** The study utilised data that were gathered in accordance with the University of Malta's Research Ethics Review Procedures, and was approved by the Faculty of Economics, Management and Accountancy Research Ethics Committee (protocol code 05371, approved on the 26 May 2020). Given that the original data collection involved children in schools, ethics clearance was also requested and obtained from the Malta Ministry for Education and Employment.

**Informed Consent Statement:** All the students (minors) involved in this study gave their assent to participate, after full informed consent was obtained from their legal guardians. Permission from the schools was also obtained prior to the study.

**Data Availability Statement:** The study utilised secondary data Requests for access to the original data may be made via the corresponding author.

**Acknowledgments:** Gratitude is due to Liberato Camilleri, Gilmore Camilleri and Melchior Vella for assisting in the data analysis.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

Abdi, Herve, and Lynne J. Williams. 2010. Principal component analysis. *Wiley Interdisciplinary Reviews: Computational Statistics* 2: 433–59. [CrossRef]

Amabile, Teresa M. 1996. *Creativity in Context*. Boulder: Westview.


Collard, Paul, and Janet Looney. 2014. Nurturing creativity in education. *European Journal of Education* 49: 348–64. [CrossRef]

Corazza, Giovanni Emanuele, Frederic Darbellay, Todd Lubart, and Chiara Panciroli. 2021. Developing intelligence and creativity in education: Insights from the space–time continuum. In *Creativity and Learning*. Edited by Soila Lemmetty, Kaija Collin, Vlad P. Glăveanu and Panu Forsman. Cham: Palgrave Macmillan, pp. 69–87.

Cropley, Arthur. 2006. In praise of convergent thinking. *Creativity Research Journal* 18: 391–404. [CrossRef]


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

## *Article* **Effect of Reverse Engineering Pedagogy on Primary School Students' Computational Thinking Skills in STEM Learning Activities**

**Xiaohong Liu 1, Xiao Wang 1, Kexue Xu <sup>2</sup> and Xiaoyong Hu 2,3,\***


**Abstract:** Computational thinking (CT) is important for students because it is one of the 21st century's skills. Reverse engineering pedagogy (REP) can improve students' CT due to its ability to develop students' cooperativity, algorithmic thinking, creativity, and problem-solving in discipline education. Thus, this study aimed to explore the effect of REP on primary school students' CT skills in STEM learning activities. A total of 101 fifth graders in a primary school participated in the study for one semester (16 weeks), including 51 students in the experimental group (EG) with REP, and 50 students in the control group (CG) with the demonstration method (DM). The computational thinking scale (CTS) was used to measure the CT skills of students in the pretest and posttest. The Wilcoxon signed-rank test and the Mann-Whitney *U* test were used to analyze the data. The results verified that REP had a fine effect on the improvement of students' CT skills compared to the DM. The findings can provide some ideas for researchers to develop students' CT skills in STEM learning activities. Teachers can use different teaching methods to reasonably arrange teaching activities to develop primary school students' CT skills.

**Keywords:** computational thinking skills; reverse engineering pedagogy; STEM learning activity; primary school student; a quasi-experimental study

#### **1. Introduction**

Computational thinking (CT) is the thinking process which can formulate problems and their solutions (Wing 2011). CT training can improve the flexibility of students' thinking. This process involves several related cognitive skills, including abstraction, decomposition, debugging, creativity, cooperativity, heuristic reasoning, algorithmic thinking, recursive thinking, critical thinking, problem-solving, and data analysis (Barr and Stephenson 2011; Brennan and Resnick 2012; Korkmaz et al. 2017; Wing 2006). However, not all sub-CT skills are specific and measurable. Therefore, to make the development of CT more specific and detailed, it is necessary to identify a set of CT concepts, skills, and/or practices that are specifically defined and measurable (Weintrop et al. 2016). According to the International Society for Technology in Education (ISTE 2015), there are five CT skills: namely, creativity, critical thinking, cooperativity, problem-solving, and algorithmic thinking. The Computational Thinking Scale developed by Korkmaz et al. (2017) also covers the CT skills of these five acceptable psychometric measures. CT is considered as a form of higher-order thinking; thus, CT skills are essential for every student (Grover and Pea 2019). In this study, CT skills as defined by ISTE (2015) were adopted.

CT is an indispensable part of the core skills of STEM (science, technology, engineering, and mathematics) education (Arık and Topçu 2021; Sun et al. 2020; Tan et al. 2019). They can constantly cultivate students' ability to meet challenges in the future. How to effectively foster students' CT skills has become a key point in educational research in recent years.

**Citation:** Liu, Xiaohong, Xiao Wang, Kexue Xu, and Xiaoyong Hu. 2023. Effect of Reverse Engineering Pedagogy on Primary School Students' Computational Thinking Skills in STEM Learning Activities. *Journal of Intelligence* 11: 36. https://doi.org/10.3390/ jintelligence11020036

Received: 23 December 2022 Revised: 10 February 2023 Accepted: 13 February 2023 Published: 15 February 2023

**Copyright:** © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**<sup>\*</sup>** Correspondence: huxiaoy@scnu.edu.cn

Visual programming is one of the common tools for fostering K-12 students' CT skills; it is helpful for training students' mathematical thinking, critical thinking, creativity, and algorithmic thinking (Liu et al. 2021; Luo et al. 2020; Rodríguez-Martínez et al. 2019; Wong and Cheung 2020). Robot programming activities in STEM education are an effective teaching strategy, as they can deepen students' comprehension of scientific concepts, improve students' learning interest, and cultivate their creativity, critical thinking, communication, and collaboration skills (Boya-Lara et al. 2022; Jaipal-Jamani and Angeli 2017; Üçgül and Altıok 2022). However, some studies have found that visual programming learning did not have a positive influence on all higher-order thinking (e.g., Chang 2014; Falloon 2016). For example, Scratch did not affect problem solving and algorithmic thinking (Jiang and Li 2021). Thus, the way to cultivate CT skills in visual programming teaching environments should be further discussed.

Interdisciplinary approaches can foster students' interest in learning, which can in turn cultivate their creativity and problem-solving skills (Bernstein et al. 2022). Reverse engineering pedagogy (REP), which was developed for engineering courses, involves knowledge in the field of engineering, mathematics, science, and computers. REP can instruct students to analyze existing works, deduce design parameters and implementation methods, and realize interaction between work groups (Zhong et al. 2022). The general teaching process is as follows: starting from a complete work, which can be called a "black box" (Otto and Wood 1998), students discover the design parameters and schemes of the existing work under the guidance of the teacher, and then improve or innovate the work according to the learning objectives. Different from the demonstrative method (DM), REP has the following advantages: (1) it helps students to deepen their understanding of scientific concepts and enhance their design ability in practice (Hess 2000); (2) it can improve students' learning enthusiasm (Barr et al. 2000); and (3) it can enhance students' learning abilities (e.g., creativity, insight, and hands-on skills) (Grantham et al. 2010; Zhong et al. 2022). Therefore, it has good applicability in robot education (West et al. 2015; Zhong et al. 2020). Some researchers have explored the effect of REP on students' skills. For example, Ladachart et al. (2022) explored the role of REP in deepening students' understanding of scientific concepts compared with design-based learning. Moreover, previous studies found that REP could develop students' algorithmic thinking, problemsolving, and creativity (Abdüsselam et al. 2022; Grantham et al. 2010; Klimek et al. 2011; Tan et al. 2021). According to the definition of ISTE (2015), CT is a subset of skills including creativity, cooperativity, algorithmic thinking, critical thinking, and problem-solving. Thus, REP has the underlying ability to promote students' CT skills. Quasi-experimental research refers to the research method that does not need to randomly arrange the subjects, but uses an original population to carry out experimental treatment under relatively natural conditions (Heath 2018). Quasi-experimental research has the basic form of experimental research, including a causal hypothesis and some types of operation that compare two (or more) conditions (Tharenou et al. 2007). Therefore, this was a quasi-experimental study which explored the effect of REP on the CT skills of primary school students in STEM visual programming robot projects.

#### **2. Literature Review**

#### *2.1. Computational Thinking*

The term CT was first proposed by Papert (1980) in his book, *Mindstorms: Children, computers, and powerful ideas.* Wing (2006) defined CT as "solving problems, designing systems, and understanding human behavior, by drawing on the concepts fundamental to computer science" (Wing 2006, p. 33). Two classification methods of CT definitions were proposed by Tang et al. (2020). The first category emphasized that CT belongs to a domainspecific field, which only covered programming and computing concepts (Denner et al. 2012; Weintrop et al. 2016; Zhang and Nouri 2019). For example, Brennan and Resnick (2012) proposed that CT included three aspects, namely, computational concepts, practices, and perspectives. The other category emphasized that CT was not limited to computer science

(e.g., Guzdial 2008; Lai et al. 2021). For example, Selby and Woollard (2013) developed a CT framework including five aspects: (1) abstraction, which focuses on basic information to solve problems; (2) decomposition, which means the ability to decompose big problems into small ones; (3) algorithmic thinking, which refers to the ability to use flow charts or refine steps of problem-solving; (4) evaluation, which refers to the tendency to find the best solution to a problem; and (5) generalization, which refers to the learning transfer ability. Some researchers believe that CT is a kind of comprehensive thinking, which includes mathematical thinking, scientific thinking, and engineering thinking (Doleck et al. 2017; Korkmaz and Bai 2019). The International Society for Technology in Education (ISTE 2015) stated that CT is a problem-solving process that includes (but is not limited to) the following components: (1) data abstraction; (2) logical reasoning and data analysis; (3) the algorithm idea of automatic solution; (4) using computer-related tools to design solutions to problems; (5) efficient problem solving; and (6) learning transfer. Therefore, ISTE defined CT as comprehensive thinking and a key component of interdisciplinary teaching, which is closely related to using science, technology, and mathematical logic in hands-on operations to solve problems. Creativity, critical thinking, communication, and collaboration, which are included in CT, are seen as key skills that will help students succeed in the future (Üçgül and Altıok 2022). Therefore, this study regarded CT as a kind of higher-order thinking and explored the CT progress of primary school students.

Considering the importance of CT, CT skills should be cultivated and developed in children from an early age (Lindberg et al. 2018; Manches and Plowman 2015). STEM education provides an effective physical environment for fostering students' higher-order thinking (Waterman et al. 2020). CT can be integrated into STEM education science courses use physical models (Arık and Topçu 2021). Visual programming learning could develop students' CT skills (Chou 2020; Tang et al. 2020). Robot education in STEM education provides a good physical environment for visual programming, which helps to enhance students' interest in learning; develops their creativity, critical thinking, communication, and collaboration; deepens their comprehension of scientific concepts; and improves their CT skills in practical activities (Boya-Lara et al. 2022; Jaipal-Jamani and Angeli 2017). However, some studies have found that visual programming learning did not have a positive influence on all higher-order thinking. For example, Jiang and Li (2021) discovered that Scratch programming learning did not effectively improve students' algorithmic thinking and problem-solving ability. Problem-solving and algorithmic thinking also tend to be the weakest among students' CT skills (Korkmaz and Bai 2019). Therefore, the influence of teaching methods on improving students' CT skills remains to be explored in the robot visual programming environment.

#### *2.2. Reverse Engineering Pedagogy and Computational Thinking*

Reverse engineering (RE) originated in the field of engineering (Raja 2007). Contrary to forward engineering which emphasizes the process from ideas to projects, RE starts from a complete project, goes through a series of measurement and analysis processes to obtain a virtual model, and emphasizes understanding and overall grasping of projects (Zhong et al. 2020). The design process of engineering has been considered as a teaching method that can be used to improve students' problem-solving and CT skills (Ehsan et al. 2021; Ladachart et al. 2022; Zhou et al. 2017). RE can be traced back to 1992 as a teaching method. Sheppard (1992) set the teaching objectives of the "Mechanical Anatomy" course to develop students' problem-solving skills, and encouraged students to be hands-on, namely, in the "anatomy" process—disassembly and reconstruction. RE can be used in the process of developing different products based on existing components or products, namely redesign (Lee and Woo 1998).

REP conducts teaching according to certain steps. Wood et al. (2013) summarized 10-step RE and redesign approaches. The structure was divided into three phases: reverse engineering, modeling and analysis, and redesign. The first stage began with studying, forecasting, and making assumptions about the project to reduce the influence of learners'

psychological biases on learning. Then learners disassembled the project to deepen their understanding of components and projects. The second stage was the analysis and modeling. The main task was understanding the structure of projects, analyzing the existing problems, and thinking about the optimal solutions. The third stage was the redesign. Three improvement methods were proposed: namely, the parametric, adaptive, and primitive methods. According to the stages of Piaget's cognitive development, children's thinking develops through four stages (Piaget 1972): the sensori-motor level, the pre-operational level, the stage of concrete operations, and the formal operational stage. The development of K-12 students' thinking involves the latter three stages. Students' abstract thinking and logical thinking also develop from generation to maturity. Thus, teachers must set appropriate teaching objectives to guide the redesign process. Therefore, REP adopted in this study did not fully follow the 10-step reverse engineering and redesign methods proposed by Wood et al. (2013). This experiment started with the analysis and disassembly stage to learn the basics and redesign the product.

REP has formed a specific educational model during its development. Zhong et al. (2020) summarized previous studies and proposed "the Latent Model", which included four RE instructional models, namely, (1) "Deconstruction and recovery" which means the dismantling and recovery of the project; (2) "Troubleshooting and recovery" which means solving problems in the project and restoring the structure of the project; (3) "Element minitrim", which means deconstructing and adjusting some elements of the project; and (4) "Structural innovation" which means dismantling and rebuilding the project. Troubleshooting can effectively improve students' ability to solve problems (Zhong and Li 2019). It is easier to tweak certain elements of a project than to innovate the structure in the teaching process. Therefore, the second and third RE instructional models were used in this experiment according to whether there were problems with the projects.

In the process of REP and the redesign approach, students can develop innovation over a "hands-on" project (Otto and Wood 1998). REP is a project-based learning strategy which could cultivate students' communication and collaboration abilities in mechanical engineering education (Barr et al. 2000). In computer courses and engineering courses, REP could enhance students' problem-solving skills by solving problems that arise in specific projects (Dempere 2009). REP could improve K-12 students' creativity and self-efficacy in STEM visual programming projects (Leonard et al. 2016). REP has advantages over forward project-based pedagogy (FPP) in terms of promoting K-12 students' creativity (Zhong et al. 2020). Concrete instructional design in STEM courses is used to improve students' mathematical thinking, CT, and problem-solving skills (Sung and Black 2020). The International Science Education Conference 2021 (ISEC 2021) used REP to incorporate design into the physics curriculum to address the problem of unfocused goals, which greatly improved students' problem-solving efficiency. In programming activities, it is valid to use REP to cultivate students' logical thinking, algorithmic thinking, critical thinking, and problem-solving skills (Abdüsselam et al. 2022; Griffin et al. 2012; Rogers-Chapman 2014). Therefore, it can be inferred that REP can develop students' CT skills in STEM learning activities, but this still needs to be confirmed in future studies. Thus, this study explored the effect of REP on CT skills of primary school students in STEM learning activities.

#### *2.3. Research Question*

Nowadays, interdisciplinary skills are conducive to students' success in global competition (ISTE 2015). Thus, CT training in STEM education is very important. Existing studies have found that visual programming cannot effectively improve each dimension of higher-order thinking, such as problem-solving and algorithmic thinking, but it can effectively cultivate students' creativity, cooperativity, and critical thinking. Previous studies (Abdüsselam et al. 2022; Barr et al. 2000; Rogers-Chapman 2014; Sung and Black 2020; Zhong et al. 2020) indicated that REP can improve students' creativity, communication and collaboration abilities, mathematical thinking, problem-solving ability, algorithmic thinking, and critical thinking. There is less research exploring the effect of REP on primary school students' CT skills in STEM learning activities. Thus, this research adopted a two-group pretest-posttest quasi-experimental study to explore whether REP can promote the CT skills of primary school students in STEM visual programming and robot projects. Therefore, the following question was raised.

RQ: can REP effectively improve the CT skills of primary school students in STEM learning activities?

#### **3. Methodology**

#### *3.1. Participants*

This research was launched as part of STEM learning of a primary school in Xiamen, China, from March to June 2022. The research subjects were 101 fifth graders aged 10–11 years old, who were taught by the same teacher. A quasi-experimental study was conducted. Two classes were selected as the control group (CG, 25 girls and 25 boys) and the experimental group (EG, 22 girls and 29 boys) in this primary school. Since the classes of the primary school were divided according to a random principle and students started to learn visual programming and robotics from the fifth grade, the two groups had a similar starting level in visual programming learning.

#### *3.2. Learning Materials*

Six topics of fifth graders' STEM learning activities related to AI were selected in this experiment: namely, Publicity Board, Noise Detector Design, Sound and Light Control Switch, Gesture Interaction, Alarm Line, and Mine. A brief introduction of each topic is shown in Appendix A. The programming platform used in this study was uKit Explore, professional visual programming software provided by UBTECH for the competition. It uses an Arduino-compatible open-source platform master controller to meet the programming needs of learners at different levels with rich learning resources. The software is compatible with the uKit servo and several structural parts specifications, and supports many programming languages. Students' programming projects can be retained in the form of projects for communicating and displaying among students and teachers. Figure 1 shows the home page of the uKit Explore software. There are different colors in the leftmost stage which represent different functions. For example, the yellow block represents the "Sensor" function.

The general process of each project is to code and build the project according to the existing materials, carry out continuous debugging and operation, and finally finish the teaching goal. The construction of the robot project involves engineering and science knowledge, and the process of coding and debugging involves the knowledge of computer and mathematical logic, which are adapted to the content of various disciplines in STEM learning activities. One typical project was the Noise Detector Design (see Figure 2); the teaching objective of this project was that students could use the color of the tiny flashlight LED to detect the volume of the sound. To achieve this goal, sound sensors, tiny flashlight LEDs, switches, and several parts were used for construction, as shown in Table 1: Firstly, the detection department and handle were built, then the whole of the project was assembled. The project construction process involved engineering and physics knowledge, and mathematical logical thinking. Secondly, students used conditional statements (showing a blue light if the volume of the sound was less than 40 dB or showing a green light if the volume of the sound was less than 45 dB), controlling the color of the lights to indicate the volume of the sound (see Figure 3). In this process, setting code parameters and programming processes involved knowledge of computer and mathematics disciplines.


**Figure 1.** uKit Explore home page interface.

**Figure 2.** Noise detector design models.

**Table 1.** Material bar of noise detector design.



**Table 1.** *Cont.*

**Figure 3.** The code of noise detector design.

#### *3.3. Procedures*

The experimental process is shown in Figure 4. The same STEM course teacher taught the EG and CG, and the materials for construction and the textbooks used were the same. The teacher taught each class once a week for 16 weeks. In the pretest and posttest, questionnaires containing the CT scale were sent to the EG and CG students. Students in both groups completed the CT skills pretest and posttest. During the first week, they completed the pretest. From week 2 to week 3, the teacher taught the basic knowledge of STEM visual programming and robot projects, and taught students how to use the programming software uKit Explore. From weeks 4 to 15, the teacher assigned six visual programming and robot projects for the two classes and asked the students to complete one project every two weeks. The teacher divided each class into 10 learning groups, numbered 1–10 with five to six students in each group. In each project, the teacher's job involved analyzing cases, providing teaching tasks, and answering questions proposed by students. The students' learning tasks were reviewing the basic knowledge, building the project, and running it. However, the EG and CG adopted different teaching procedures. The teaching procedures of the CG were reviewing, constructing the project, demonstrating, and reporting. The teaching procedures of the EG were analyzing projects, troubleshooting and dismantling the project, rebuilding the project, demonstrating, and reporting. The CG was taught first every week. Each group in the CG constructed projects referring to the theme and experimental equipment provided by the teacher and then presented their robot projects and programming codes on the stage. Then the EG was taught, and the programming projects built by the CG were distributed to each group in the EG according to the number of the group. The teacher provided the task lists of the EG as follows: (1) debug whether the project is working properly; (2) if the programming project can run, disassemble and rebuild it according to the teaching objectives; (3) if there is a fault in the programming project, try to solve the problem, disassemble, and rebuild. After that, all the groups reported the existing problems and solutions of the original projects, the innovative points of the new projects, and the flow charts of design thinking in turn. Finally, students showed and ran the new projects, and reported the new knowledge learned. The teacher commented on the project of each group and invited representatives from other groups to make comments. In the final week, the EG and CG completed the posttest of their CT skills.

**Figure 4.** Experiment design and procedure.

#### *3.4. Instrument*

ISTE (2015) stated that CT is a reflection of higher-order thinking, and divided CT skills into the following five sub-dimensions: cooperativity, creativity, algorithmic thinking, critical thinking, and problem-solving. Scholars have developed several computational thinking scales (CTSs) to evaluate the CT skills of adolescents. For example, Korkmaz et al. (2017) put forward a CTS to evaluate undergraduates in Turkish. Realizing that high school students' CT skills in China could be better measured, Korkmaz and Bai (2019) revised the scale proposed by Korkmaz et al. (2017). The sub-dimensions of this CTS were consistent with the CT skills measured in this study. Therefore, the CTS developed by Korkmaz and Bai (2019) was translated into Chinese for this study, then adapted and simplified for the understanding and application of primary school students in China. In this study, the CTS was a 5-point Likert scale with 20 single-choice items which were divided into the following 5 sub-dimensions: (1) Creativity (3 items); an example of this subscale is: "I believe I can solve the problems that might arise when I encounter new situations". (2) Cooperativity (4 items); an example of this subscale is: "More ideas are emerging in collaborative learning". (3) Algorithmic thinking (4 items); an example of this subscale is: "I can immediately establish a thought process that can solve the problem". (4) Critical thinking (4 items); an example of this subscale is: "I can use a systematic approach when comparing the options at hand and making a decision". (5) Problem-solving (5 items); an example of this subscale is: "I can apply my planned solutions step by step".

To further verify the applicability of this CTS to this study, 100 fifth graders from another two classes of this primary school were selected for the pilot study before the beginning of this study (McNeill et al. 2016). A total of 73 valid data were collected, and the CTS was tested for reliability and validity. The results showed that the Kaiser–Meyer–Olkin (KMO) was 0.853 > 0.800 (*p* < 0.01), the explanatory degree of cumulative variance was 70.832% > 70%, and the Cronbach's alpha was 0.895 > 0.600. The Cronbach's alpha of each dimension of CT skills is shown in Table 2, all of which were higher than the threshold (Alwin 1989; Alwin and Beattie 2016). Although the first figure is low, the CTS could be used (Tran 2018).


**Table 2.** Reliability of the revised CT scale.

#### *3.5. Data Analysis*

This study used the software SPSS 26.0 to analyze the data on students' CT skills in the two classes. Firstly, the means (*M*) and standard deviations (*SD*) of the EG and CG data were calculated using descriptive statistical methods. Secondly, this study used the Kolmogorov– Smirnov Z-test to test whether the data of the two groups conformed to normal distribution. If the result conformed to normal distribution, this study used the independent samples *t* test to verify the difference in the starting and ending levels of students' CT skills between the two classes, and used the paired samples *t* test to test the difference in the development of students' CT skills between the two classes. If the result did not conform to normal distribution, the Mann–Whitney *U* test was performed to test the difference in the students' CT skills' starting and ending levels in the two classes. The Wilcoxon signed-rank test was used to verify the differences in the improvement of CT skills by students in the two classes.

#### **4. Results**

The aim of this research was to test the effect of REP on fifth graders' CT skills, so the standard deviations (*SD*) and means (*M*) of each class were counted in the pretest and posttest. Cronbach's α of the pretest and posttest were 0.850 and 0.948. If the data from each dimension of the two tests were normally distributed, this study would use the paired sample *t* test and independent sample *t* test. The results are shown in Table 3 (Dereli ˙ Iman et al. 2017), and only the KS-Z of cooperativity in the pretest of EG was 0.176 > 0.05. The result did not conform to normal distribution. Therefore, non-parametric tests were used in this study.

**Table 3.** Descriptive statistics of the experimental group (EG) and the control group (CG) and normality test results with Kolmogorov–Smirnov Z.


Note: EG = the experimental group; CG = the control group.

This study used the Mann–Whitney *U* test to verify the difference in the CT skills' starting and ending levels of students in the two classes, as shown in Table 4. The results of the pretest (creativity: *U\** = 0.997 > 0.05; cooperativity: *U\** = 0.278 > 0.05; algorithmic thinking: *U\** = 0.652 > 0.05; critical thinking: *U\** = 0.964 > 0.05; problem-solving: *U\** = 0.066 > 0.05) verified that there were no significant differences in the starting levels of CT skills in the two classes, which further indicated that the two selected classes were suitable for this study. The results of the posttest (creativity: *U\** = 0.000 < 0.05; cooperativity: *U\** = 0.000 < 0.05; algorithmic thinking: *U\** = 0.000 < 0.05; critical thinking: *U\** = 0.000 < 0.05; problemsolving: *U\** = 0.000 < 0.05) showed that there were significant differences in the ending levels of CT skills in the two classes. The results verified that the development of CT skills in the EG and CG was inconsistent.

**Table 4.** Mann–Whitney *U* test results from the pretest and posttest scores of the experimental and control groups.


Note: EG = the experimental group; CG = the control group; \* Statistical significance level has been adopted as .05/5 = .01 for this analysis using Bonferroni correction.

This study used the Wilcoxon signed-rank test to compare the CT skill differences between the paired samples of the two classes, as shown in Table 5. The results showed that the five sub-dimensions of the CT skills in the two classes significantly improved, namely, creativity (CG: *Z\** = −5.401, *p* < 0.001; EG: *Z\** = −6.171, *p* < 0.001), cooperativity (CG: *Z\** = −5.535, *p* < 0.001; EG: *Z\** = −6.168, *p* < 0.001), algorithmic thinking (CG: *Z\** = −5.530, *p* < 0.001; EG: *Z\** = −6.230, *p* < 0.001), critical thinking (CG: *Z\** = −5.996, *p* < 0.001; EG: *Z\** = −6.228, *p* < 0.001), and problem-solving (CG: *Z\** = −5.669, *p* < 0.001; EG: *Z\** = −6.230, *p* < 0.001). The improvement of the EG's CT skills was greater than that of the CG, which indicated that REP played a more positive role in developing primary school students' CT skills than the DM did.

**Table 5. The** Wilcoxon signed-rank test on the CT skills' five sub-dimensions of the two classes.


Note: EG = the experimental group; CG = the control group; \* Statistical significance level has been adopted as .05/5 = .01 for this analysis using Bonferroni correction.

#### **5. Discussion**

CT is a key element in developing STEM learning activities (Weintrop et al. 2016; Yin et al. 2020). To cultivate K-12 students' CT skills, the education community will continue to develop ways to develop CT skills (Waterman et al. 2020). This research used a quasi-experimental study to explore the effect of REP on CT skills of primary school students in STEM learning activities. Compared with the DM, REP is more in line with the characteristics of STEM learning activities, as it can fully mobilize knowledge in various discipline areas and can better cultivate students' problem-solving abilities (Dempere 2009).

The research results showed that the improvement of each dimension of CT skills in the EG was significantly better than that of the CG, indicating that REP played a positive role in developing students' CT skills in STEM learning activities, which was consistent with the previous hypothesis. Previous researchers have used diverse teaching activities in REP to explore its effect on the five sub-dimensions of CT skills. For example, in the course of mechanical engineering, REP and the learning method of group cooperation were used to construct the three-dimensional solid model, and freehand sketches and notes were used to record the mechanical decomposition process of RE during team communication, which greatly improved the students' cooperativity (Barr et al. 2000). REP can help students solve practical problems better (Calderón 2010; Lur et al. 2022). In computer and science courses, Klimek et al. (2011) fully introduced the teaching model and strategy of creative thinking by investigating the usage scenarios of REP and listing the methods to solve practical problems, which cultivated students' creativity and algorithmic thinking. Zhong et al. (2020) compared the different influences of FPP and REP on problem-solving and studied the Latent Model, which involved four models of REP to foster students' problem-solving ability and creativity in a variety of teaching activities. Griffin et al. (2012) took Deconstruction Kits in REP as a tool to attract learners' attention and to develop their critical thinking and problem-solving ability, and then improved students' creativity through decomposition and debugging. Zhong et al. (2022) built a blue smart car produced by CFunWorld using REP. In the process of perceiving, observing, breaking down, summarizing, drawing program flow charts, restoring works, redesigning, revising and adjusting, and reflecting, students can develop hands-on skills, algorithmic thinking, critical thinking, creativity, and problem solving through cooperative learning. The experimental process of this study also followed the REP and redesign methods to develop students' CT skills to the maximum extent. Two teaching models from the Latent Model proposed by Zhong et al. (2020) were used in this study. One concrete example in this experiment was "the Noise Detector Design project". In the process of troubleshooting, the EG students learned the design ideas of projects and the principle of the components by analyzing or debugging the codes and the projects, while the CG students built the project using the project code directly provided by the teacher. Therefore, the problem-solving skills and critical thinking of EG students were better developed than those in the CG. In the process of disassembling and adjusting their projects, the students in the EG were asked to record the design thinking of the new project and show their ideas on stage according to the learning objectives. The students applied the design ideas of the original projects to the new ones and made innovations through learning transfer to cultivate creativity and algorithmic thinking. Therefore, when students in the EG encounter similar problems, it is easy to apply the knowledge and skills they have learned.

#### **6. Conclusions and Limitations**

This research was carried out in STEM visual programming and robot projects, and explored the effect of REP on primary school students' CT skills. The results verified that REP can develop students' CT skills better than the DM can. Specifically, REP could develop the five sub-dimensions of CT skills.

This study has theoretical and practical significance. Theoretically, REP originally belonged to the field of engineering, but the application of REP in STEM education has expanded it to interdisciplinary fields. This study verified the applicability of REP to

elementary school robot education, which is consistent with the findings of Zhong et al. (2020) and Israel-Fishelson and Hershkovitz (2022). This study verified the effect of REP in primary school and provided ideas for future research on CT training methods. Teachers can consider using different teaching methods to foster students' CT skills, which has certain reference significance for future research. In a practical sense, this study considered the effect of teaching methods, teachers, teaching time, the starting level of students' CT skills, and other factors on the experimental results, and conducted effective control to prevent irrelevant variables from having a significant influence on the experimental results. This study applied REP to STEM teaching activities in primary schools, provided an example of the use of REP in primary school robot education, and cultivated students' hands-on operation and learning transfer ability.

Some limitations should be considered when representing the findings. Firstly, the sample was limited to 101 fifth graders from a primary school in China. Further research can expand the sample to include other grades and regions. Secondly, the course type was limited. This study was conducted in STEM visual programming and robot projects, which mainly relied on programming and computers. Future research can extend the approach to other courses, such as unplugged projects, electronic reading, etc. Finally, the research method adopted was a quasi-experimental study, and two existing classes in a primary school were selected as the EG and CG. There was therefore no way to expand the sample size of the two groups, resulting in a small sample size. The problem of small sample size often occurs in quasi-experimental studies (e.g., Cheng et al. 2020; Cutumisu et al. 2020; Hsiao et al. 2021; Nicolaidou et al. 2021; Tang and Hew 2022; Yalçın and Erden 2021; Zhao et al. 2022). Therefore, it is necessary to expand the sample size to ensure that the experimental results are consistent with this study in future studies and to further prove the reliability of this study. Finally, this study only assessed participants' feelings or confidence about their CT skills via the CT scale, not their actual CT skills. Future research could use different measurements to evaluate students' CT skills.

**Author Contributions:** Conceptualization, X.L. and X.H.; methodology, X.L.; formal analysis, X.L. and X.W.; investigation, X.W.; resources, X.W.; data curation, X.W. and K.X.; writing—original draft preparation, X.L. and X.W.; writing—review and editing, X.L. and X.H.; supervision, X.H. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the National Social Science Foundation, "Teacher Portrait and Application Research from the Perspective of Artificial Intelligence", grant number BCA220206.

**Institutional Review Board Statement:** The study was approved by the Ethics Committee of South China Normal University (approval code: SCNU-AIE-2022-004).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The original contributions presented in the study are included in the article/Appendix A, further inquiries can be directed to the corresponding author.

**Acknowledgments:** We would like to acknowledge the facilitators' and students' participation and valuable feedback.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

**Weeks Project' Name Teaching Objectives Works and Codes** 4–5 Publicity Board Learning objectives: (1) Students learn to use related components, brightness sensors, and tiny flashlight LEDs. (2) Students understand the meaning and usage of the function blocks: "if . . . . . . so . . . " and "Otherwise". Learning content: (1) Students build a propaganda window, propaganda support frame, and operation platform. (2) Students perform visual programming to realize the function that the light changes with the intensity of light outside. 6–7 Noise Detector Design Learning objectives: (1) Students learn to use related components—sound sensors and tiny flashlight LEDs. (2) Students understand the meaning and use methods of the function block: "if . . . . . . perform . . . otherwise if . . . . . . perform . . . ". Learning content: (1) Students build the test section, handle (handheld part), and assemble the whole project. (2) Students perform visual programming to light the color of the lamp flap according to the volume of the sound. 8–9 Sound and Light Control Switch Learning objectives: (1) Students learn to use relevant components, sound sensors, and brightness sensors. (2) Students understand the meaning and usage of the logic function block: "And". Learning content: (1) Students build the first layer including the fixed brightness sensor, sound sensor, and tiny flashlight LEDs, and place the motherboard and battery in the second layer. (2) Students perform visual programming to realize that the light will be on for 10 s when the light outside is dimmed or there is sound.

**Table A1.** Information on the control group's six items.



**Table A2.** Information on the experimental group's six items.

**Table A2.** *Cont.*


#### **References**

Abdüsselam, Mustafa Serkan, Ebru Turan-Güntepe, and Ümmü Gülsüm Durukan. 2022. Programming education in the frameworks of reverse engineering and theory of didactical situations. *Education and Information Technologies* 27: 6513–32. [CrossRef]

Alwin, Duane F. 1989. Problems in the Estimation and Interpretation of the Reliability of Survey Data. *Quality and Quantity* 23: 277–331. [CrossRef]

Alwin, Duane F., and Brett A. Beattie. 2016. The KISS Principle in Survey Design. *Sociological Methodology* 46: 121–52. [CrossRef] Arık, Merve, and Mustafa Sami Topçu. 2021. Computational thinking integration into science classrooms: Example of digestive system.

*Journal of Science Education and Technology* 31: 99–115. [CrossRef]

Barr, Ronald E., Philip S. Schmidt, Thomas J. Krueger, and Chu-Yun Twu. 2000. An introduction to engineering through an integrated reverse engineering and design graphics project. *Journal of Engineering Education* 89: 413–18. [CrossRef]

Barr, Valerie, and Chris Stephenson. 2011. Bringing computational thinking to K-12. *ACM Inroads* 2: 48–54. [CrossRef]


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

## *Article* **Young Creators: Perceptions of Creativity by Primary School Students in Malta**

**Margaret Mangion \* and Jasmin Antonia Riebel**

Edward de Bono Institute for Creative Thinking and Innovation, University of Malta, MSD 2080 Msida, Malta **\*** Correspondence: margaret.mangion@um.edu.mt

**Abstract:** Policymakers and employers insist that educational institutions prepare workforce-ready candidates fluent in the application of 21st Century Skills such as creativity. So far, only a few studies have explored the self-perceived creativity of students. This paper addresses this gap in literature by providing an understanding of how young students in upper primary feel about themselves as creative beings. Data for the present study was collected through an anonymous online survey that 561 students, aged 9–11 years and residing in Malta (EU), completed. In-depth responses were collected from a subset of the original sample made up of 101 students through an anonymous online form containing a set of questions. Data was analysed using regression analysis for the quantitative component and through thematic analysis for the qualitative part. Results indicate that, overall, students in Year 6 felt less creative than students in Year 5. Furthermore, findings show that the type of school attended impacted the students' perception of creativity. From a qualitative perspective, findings led to insights into (i) the interpretation of the term creativity and (ii) the impact of the school environment and how timetabling impacted students' creativity. The findings suggest that the student's perceived creative personal identity and the concrete manifestations that they engage in are influenced by environmental factors.

**Keywords:** creativity; self-perceived creativity; creative personal identity; education; primary school; creator; context; environment

#### **1. Introduction**

Creativity as an area of study has attracted increasing attention over the last few decades. Empirical research and theories have moved in a multitude of directions apart from the initial study of the person as a creative being. The value of creativity also found its way through practical paths of application through methods like creative problem-solving (Isaksen et al. 2011). Creativity and the desire to optimise it have come to the fore in occupational scenarios too (Kennedy and Sundberg 2020; Partnership for 21st Century Learning 2015) since it became a required characteristic to have on the job and also on a personal level (Beers 2016). With creativity featuring in the top places on the list, the 21st century skills have been identified with the intention to equip students and therefore future citizens of the world with a set of transferable skills that could be useful to address changing situations (Chu et al. 2017; Partnership for 21st Century Learning 2015; Voogt and Roblin 2012). What are referred to as 'applied skills' have been classified as very important for success at work (Casner-Lotto and Barrington 2006). Against this backdrop, the role of creativity and its development becomes imperative. This justifies the call by the late Anna Craft (2003) who urged scholars to consider creativity as an important part of education, a notion that is also supported by Gormley (2020). Linking creativity to education, Karwowski (2015) claims that the former "is among the most valuable human qualities achieved during the process of learning and teaching" (p. 165).

The aim of this study is threefold. Firstly, since there is an expectation that the future workforce becomes more creative and demands are made on students to enhance this

**Citation:** Mangion, Margaret, and Jasmin Antonia Riebel. 2023. Young Creators: Perceptions of Creativity by Primary School Students in Malta. *Journal of Intelligence* 11: 53. https://doi.org/10.3390/ jintelligence11030053

Received: 27 January 2023 Revised: 24 February 2023 Accepted: 10 March 2023 Published: 13 March 2023

**Copyright:** © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

skill, we thought that it is important to ask the students themselves what they think that creativity is. Secondly, it would be interesting to see if the students identify themselves with characteristics that are present in creative persons. Finally, how the environment impacts individuals in their engagement with creative practices is also of interest. Limited research has been conducted on implicit theories leading to how young students themselves interpret the term 'creativity' and their creative personal identity. This study hopes to build on the existing body of literature about implicit theories of creativity by exploring these research gaps also identified by Karwowski (2009).

#### **2. Theoretical Background**

#### *2.1. Defining Creativity*

Defining creativity has been an elusive task for many authors for decades (Plucker et al. 2004; Runco and Jaeger 2012). Although variations exist, there seems to be some consensus towards definitions with the inclusion of 'novelty', 'originality', 'usefulness' and 'appropriateness' (Amabile 1996; Runco and Jaeger 2012). Moving in a more action-oriented direction, Karwowski (2015) refers to creativity as a specific interplay between imagination, improvisation, and innovation. Corazza (2016) wrote about a concern with some definitions of creativity, primarily with Runco and Jaeger (2012) claiming that perceptions may be at play and may influence how an artifact is interpreted in terms of its originality and effectiveness. Mindful of the context within which this study took place, the definition of creativity offered by Plucker et al. (2004) is considered to be highly relevant, since they address the individual, the creative action and the context within which the creative activity takes place defining it as "the interaction among aptitude, process, and the environment by which an individual or group produces a perceptible product that is both novel and useful within a social context" (p. 90). This definition places a focus on the individual and his surroundings allowing a dynamic shift to the relationship between the creator, creation, and the context (Lubart 2017) within which such activity takes place.

Most of these explicit definitions emerged from studies following the work of academics and experts in the field mostly concerning adult participants (Plucker et al. 2004; Runco and Acar 2012). Runco and Bahleda (1986) claim that explicit theories are predominantly psychometric in genre however, as Lim and Plucker (2001) state, they do not explain how individuals perceive creativity while they are engaging in everyday activities. On the other hand, implicit theories of creativity are constructed by untrained individuals from their experiences. Their understanding of creativity is emergent as a result of their perception of the term (Niu and Sternberg 2002). To this end, implicit theories could also be perceived as subjective in nature. However, Gralewski and Karwowski (2018) state that implicit theories of creativity could be used to define traits linked to creativity including the judging of behaviour, while from a conceptual point of view Runco and Bahleda (1986) state that implicit theories are more easily tested in general. Following the aims of this paper, our objective is to formulate an understanding of what creativity means for the targeted sample population. Understanding differences that lie between explicit and implicit definitions of creativity may arguably provide grounds to enhance interventions that may facilitate human agency leading to creative behaviour. This may lead to an increased awareness of one's creative personal identity.

In the literature creativity has been categorised in different ways. Rhodes (1961) categorised existing definitions at the time and coined the 4Ps framework. He included the *person* referring to individual characteristics related to creativity, product constituting of features that are appropriate and useful while being novel, process referring to cognitive activities in the creative endeavours and press referring to environmental elements that may lead to creative behaviour. Expanding on the 4Ps, the 7Cs of creativity were identified by Lubart (2017). Lubart captures a comprehensive illustration of the different dimensions through which creativity may be observed. Briefly, the 7Cs refer to Creators, referring to those who are engaged in the creative activity. Creating, that reflects the process related to the creative activity. Collaboration refers to third parties that may be involved in the creative

activity thus adding an interpersonal dimension. Contexts refer to the physical environment where the activity takes place, while creations entail the product resulting from the creative activity. Consumption refers to the adoption of the creative product, and finally curricula is concerned with the education and development of creativity. This paper will make explicit reference to 'creators' and 'context' although it is not excluded that other facets of creativity might find their way in the discussion that ensues. It might be useful to observe that the 4C model of creativity (Kaufman and Beghetto 2009) could arguably be employed to explore the level at which the 7Cs might occur. Ranging from eminent creativity (Big-C) which is not a common occurrence in the population, and Pro-C which relates to more professional approaches to little-c creativity which involves everyday, practical creativity, and mini-c which is manifested through a sense of analysis and problem-solving this model categorises different levels of creativity effort. Little-c creativity is similar to everyday creativity in that it relates to activities that address issues such as mixing ingredients into a recipe (Richards 2010) that is different from what was previously available. In the case of mini-c creativity, Beghetto and Kaufman (2007) refer to an interpretive and transformative that links the process of leanring and creativity. By interpreting and manipulating information received new transformations manifested as creative ideas may emerge. Since the creative self-concept, which includes creative personal identity, refers to one's understanding of how able they are to think and act creatively (Karwowski and Beghetto 2019), one may suppose that the rate at which creators engage with the contexts they find themselves in may impinge on the quantity and quality of activities related to little-c/everyday creativity and mini-c creativity. Supporting this argument is a recent study by Li et al. (2022). The authors investigated the link between the creative self-concept, and creative behaviour amongst others and found a positive relationship between the two.

#### *2.2. Children's Psychological Development and Their Understanding of Creativity*

Over the last few years, the creative self has acquired considerable importance especially through the work of Karwowski (Karwowski et al. 2013; Karwowski 2015). Flammer (1995) states that self-beliefs are shaped over time as one goes through different phases. Following the argument by Flammer (1995), one may assume that the creative personal identity of young individuals emerges as they acquire life experiences. In Eysenck (2022) Piaget's theory of cognitive development is outlined referring to schemata that are linked to different developmental stages of the child. Piaget claimed that between 7 and 11 years of age children typically reach the concrete developmental stage whereby they can start to solve problems in a logical way. The development of abstract thought follows this phase. At the age around 8 children also begin to innovate instead of imitating (Stricker and Sobel 2020). By contrast from a sociocultural perspective, Vygotsky (1962) viewed social interaction as the primary source of cognition and behaviour. Such interactions facilitate continuous processes rather than stages. The social and cultural aspects are given considerable importance in the shaping of personality through scaffolding. According to Kohlberg's (1984) theory of moral development, by age 9 children start to acquire conventional morality where acceptance of social rules about what is right and what is wrong is predominant.

Some studies have explored how young students interpret creativity (Eshun et al. 2013; Karwowski 2015). Others have explored the self-perceptions of creativity by students (Deveci 2021; Falconer et al. 2018; Stricker and Sobel 2020). While Kupers et al. (2019) claim that understanding children's creativity is very important for teachers, it may be argued that before we can start to understand children's creativity, the children themselves need to know what it is and how they can nurture it. This may facilitate the cultivation of creative behaviour of young students.

Stricker and Sobel (2020) conducted three studies to understand how children aged 5–10 reflect on and understand their creativity as well as others' creative actions, and the role that novelty and utility play. The first study consisted of semi-structured interviews with open-ended questions asking the participants what it means for them to be creative.

Interestingly, the children's examples of their own creativity showed to be more specific and relevant than those of others. In the next experiment, children and adults were asked to compare two different actions in terms of creativity and to justify their answers. According to this second study, the age around 7–8 years seems to be a transition point where the children's perception of creativity and their appreciation of the novelty aspect did not differ significantly from adults anymore. The third study focused on utility instead of novelty as the second cornerstone of creativity, according to the standard definition of creativity (Runco and Jaeger 2012), with the result that here, older children underrated utility and perceived actions which were novel but not useful as more creative.

A further study exploring creativity in primary school children was conducted by Falconer et al. (2018). This research investigated the infamous 'fourth grade slump' which represents a drop in scores on creativity tests using four measuring scales. Although these studies are useful to understand a general picture when it comes to 'quantifying' creativity, they miss out on the possibility of delving deeper into the 'how' and 'why' elements of empirical findings. This further justifies the need for the current study since these qualitative aspects are addressed together with a more generic interpretation through quantitative means.

#### *2.3. Creative Personal Identity*

The self-concept is made up of the belief that a person may have about oneself. Rogers (1959) was one of the first psychologists to explore the self-concept and attributed three different components to it; the self-image, self-esteem, and the ideal self. More recently in her work related to everyday creativity, Richards (2010) addressed issues related to the development of the self-concept. In modern creativity literature, the role of creative self-beliefs, which represent a group of beliefs that make up the creative self of individuals, has come to the fore (Karwowski 2016) in relation to the important role it could play in the manifestation of everyday creativity (Richards 2010) and creative potential (Runco and Acar 2012). In relation to the creative self-concept, typically we find reference to creative self-efficacy (CSE) (Beghetto 2006; Karwowski et al. 2013; Tierney and Farmer 2002), creative-personal identity (CPI) (Karwowski et al. 2013; Jaussi et al. 2007), creative metacognition (Kaufman and Beghetto 2013) and self-rated creativity (Furnham et al. 2008).

In his work Erik Erikson (1968) established his theory of identity. According to the psychologist, identity is shaped throughout life and is categorised into eight psychosocial stages. Essentially, identity provides the individual with perceptions in relation to the self and the interaction with others. In more recent studies Randel and Jaussi (2003) differentiated between social identity and personal identity with the latter showing an increased stability overall (Brewer and Gardner 1996). The CPI of individuals entails the attributed importance of the construct in relation to the person's identity (Jaussi et al. 2007). In a study conducted by Karwowski and colleagues in 2013, CPI was found to have a strong positive relation with Openness to Experience as a psychological construct. In keeping with the focus of this study, since Karwowski and Barbot (2016) highlighted that the creative self-concept is relatively well developed by the age of 10 and since Brewer and Gardner (1996) referred to the stability of personal identity, the pertinence in exploring how young students feel about their CPI seems to gain traction. Previous studies have focused on adult populations involving university students (Kaufman and Beghetto 2009) and working individuals (Tierney and Farmer 2002). It is only more recently that studies started to shift the attention to younger populations (Karwowski 2015). This allows for an enhanced scope for the current paper.

#### *2.4. Schools and Creativity*

There have been contradictory accounts of how school and the school environment impact student creativity. The TED talk by the late Sir Ken Robinson about how schools kill creativity (Robinson 2006) and his writings attracted considerable attention over the years. Focusing on the creativity gap, Runco et al. (2017) studied a population of undergraduate

students and found that they were less creative at school than when in other contexts. They found that creative potential was evident outside of school but was not used once on the school premises. This could be possibly linked to the lack of autonomy offered in school settings. Their sample also indicated that students were more creative when they were alone rather than being with other students. In the meantime, studies and scales have been put forward to establish the creative climate in schools (Richardson and Mishra 2018). Studies like the ones highlighting the fourth grade slump (Kim 2011) still attract considerable attention although more care should be taken about the contextual elements that may lead to the significance of statistical results. Torrance (1968), noticed the fourth grade slump when exploring fluency, flexibility, originality and elaboration in a sample of third and fourth graders over a number of years. Kim (2011) found that the production of ideas increased until the third grade but remained constant between the fourth and the fifth grade followed by a decrease. Kim associates this decrease with the possibility that young students might self-impose judgment and fear inaccuracies or potential issues with the appropriateness of their responses. Karwowski (2022) reviewed literature outside the field of creativity in search for factual references about how school may kill creativity. His search revealed that this claim does not depict the true nature of the phenomenon under discussion.

Davies et al. (2013) reviewed 210 documents commissioned by the Learning and Teaching Scotland. Their search led to the emergence of a number of factors that concern the context the students were receiving their education in thus adding relevance to the environment as an important factor to take note of when promoting creativity. Exploring the environment from different perspectives, namely physical, pedagogical and external involvement, among these factors one finds reference to the flexible use of space and time, availability of resources, working in different environments outside the formal classroom, and the inclusion of elements of playfulness and gamified activities. Moreover, the review found that soft skills like the formation of respectful relationships between the young students, their teachers and opportunities for group work, and the involvement of external entities were also beneficial to the development of skills related to creativity. The role of the teachers is highly important when establishing environments that are conducive to the creativity of young students while at school Al-Dababneh et al. (2019). Gormley (2020) explored creative environments in relation to risk-taking, autonomy, novelty and flexibility from the perspective of teachers. He found that creative environments are sometimes influenced by examinations and expectations. He goes on to note that these findings are significant in view of the emphasis currently placed on education and the need to foster creativity. Apart from these conclusions, students' perception of how their teachers relate to them greatly influences how ththeir classroom experience unfolds. As a consequence, teachers can make a difference by supporting their students' creative expression through practices and procedures in the classroom that encourage taking intellectual risks such as expressing novel ideas (Beghetto 2006). Exploring the primary physical education perspective on education, Konstantinidou et al. (2014) found that only 36.5% of their sample agreed that primary school students are offered a lot of opportunities to manifest their creativity in school. The authors claim that sports including basketball, volleyball, gymnastics and sports skill practice offer the possibility for creative activity.

Of particular interest is how these influences may be catalysed by the school environment in relation to theoretical concepts of creativity. Taking into consideration the 4C Model by Kaufman and Beghetto (2009), it is pertinent to consider how the environment may play a role in the development of the different 'Cs' in the model. It is assumed that one can build on the engagement and proficiency experienced through different 'Cs' starting with the development of mini-c creativity all the way up to Pro-C Creativity (Karwowski et al. 2022). Abilities to solve problems and the combination of different types of information could arguably be addressed as early as pre-school age and developed throughout the years of formal schooling. The facilitation of the above-mentioned aspects related to the environment may provide fertile grounds where mini-c and little-c could be developed

and nurtured during the primary years through creative learning (Beghetto 2016, 2019) with mini-c creativity being critical to creative learning due to its potential to develop ideation and problem-solving skills (Karwowski et al. 2022). The authors go on to claim that "making creativity more salient in the school environment not only requires enhancing students' creative abilities but also building their creative selves." (p. 244) Grohman (2019) conducted an experiment where she worked on cultivating little-c and mini-c creativity in students in higher education by teaching for creativity resulting in journals and a creative project. These hands-on tasks provided space where the students could practice skills related to problem solving and the application of knowledge as for little-c and mini-c creativity. This section offered an overview of some of the research that has been exploring different facets of creativity in schools over the years. The current study is interested in how young students perceive the school environment to facilitate or inhibit their level of creativity during the school day.

#### *2.5. Research Questions and Hypothesis*

Following the review presented above, one might expect that students may have an idea of what creativity may mean and for them to have a general conceptualisation of themselves and their creativity through their creative personal identity. The following research questions which will be explored through a qualitative study and hypothesis which will be tested on a larger sample are proposed:

RQ1: How do Maltese primary school students perceive creativity?

RQ2: How do students aged 9–11 perceive their creative personal identity?

RQ3: To what extent do students aged 9–11 acknowledge their creative personal identity?

**H1:** *Students aged 9–11 have a positive perception of their own creative personal identity.*

#### **3. Materials and Methods**

#### *3.1. Context and Samples*

The present study took place in Malta, a small island state in the European Union. The total population on the island at the time of data collection was around 516,100 (NSO 2021). In Malta, school is compulsory between the ages of 5 and 16 and students may attend one of three types of schools, state schools run by the state, church schools run by the Secretariat for Catholic Education, and independent schools which are privately run. Primary school education covers Years 1 through to Year 6 where students sit for a national benchmarking assessment. Fifteen schools were invited to take part in the study with eight schools accepting the invitation. The selection criteria used during the recruitment of participants were related to the aim of the study, primarily for participating students to be between the ages of 9 and 11, which in general refers to children in their fifth and sixth year of primary education in the loval context respectively. Here grade five is equivalent to grade four in the American school system.

This research project is made up of two studies. For both studies, following adherence to all the necessary ethics procedures, clearance was obtained from the University of Malta's Research Ethics Committee and the Directorate for research, lifelong learning and employability within the Ministry of Education, Secretariat for Catholic Education and Independent schools. The headmasters of the selected schools agreed to act as intermediaries in the dissemination of the information letters and consent forms. Signed consent was obtained from the parents or legal guardians and sent directly to the researcher. All data was collected online between April and June 2021 since access to schools was not allowed due to the mitigation measures that were in place to control the spread of the COVID-19 virus. Participants were presented with an online form that included three sections. Section one aimed to collect demographic data, section two was aimed to collect responses related to CPI adapted from Karwowski et al. (2018), and section three contained open and close-ended questions. A copy of the questionnaire can be found in the Supplementary

Materials. The primary researcher accessed each class virtually and read out each question to all groups to avoid potential misinterpretations.

#### *3.2. Study 1*

For Study 1, the data obtained anonymously through submission via Google Forms was analysed using the Statistical Package for Social Science (SPSS), following a data cleaning process. 561 participants aged between 9 to 11 attending primary schools in Malta took part. The students attended either Year 5 (n = 298; 53%) or 6 (n = 263; 47%) of either a state school (n = 84; 15%), a church school (n = 260; 46.3%), or an independent school (n = 217; 38.7%). The schools of which two were state schools, three church schools and another two independent schools were selected through convenience sampling. This sample can be seen as representative for the Maltese population. The results, however, are not generalizable beyond primary aged Maltese school students. The Google Form comprised two main sections. In Section A respondents answered questions related to demographic data while in Section B the questions about CPI was presented. The questions for CPI are presented in Table 1.

**Table 1.** Items used from the Short Scale of Creative Self (adapted from Karwowski et al. 2018).


The questions relating to the CPI in the Short Scale of Creative Self (SSCS) questionnaire by Karwowski et al. (2018) were used to collect data. The instrument has properties of high internal consistency as well as high reliability of measurement over time. Five items, including "I think I am a creative person" or "Creativity is an important part of myself" (Table 1), with a Likert-scale ranging from 1 (no way) to 5 (yes), were used to measure the students' trait-like Creative Personal Identity. Students were also asked to give some demographic details and to answer additional questions such as what their favourite subject in class is.

#### *3.3. Study 2*

Study 2 involved 101 students, all of whom also participated in study 1, and who attended two church schools. The sample comprised of Year 5 (n = 48; 45.7%) or Year 6 (n = 57; 54.3%). In this study, female students were in the majority with 57 (56.4%) female participants and 44 (43.6%) male participants.

The researchers developed a set of questions (Table 2) related to creator and context as part of the 7Cs framework by Lubart (2017) to explore the students' perceptions of creativity on its own as well as related to the school environment. We outlined five open-ended questions to give respondents the opportunity to express themselves and two close-ended questions. The latter included 'are you creative?' and 'is the school a place where you can be creative?' These questions were purposely asked since we were hoping to first obtain a yes/no answer followed by an elaboration in the question that followed.


**Table 2.** Open and Close-ended questions used.

#### *3.4. Data Analyses*

Data collected for Study 1 was analysed using the Statistical Package for Social Science (SPSS) using correlations and regression analysis. According to Miller and Acton (2009) a regression analysis is suitable and congruent with linear models. This appeared to be suitable in relation to the current study. Data collected from Study 2 was analysed through thematic analysis using the method by Braun and Clarke (2006). The authors define thematic analysis as "a method for identifying, analysing, and reporting patterns (themes) within data" (p. 6).

The thematic analysis for the present project is informed by an essentialist/realist world-view, and has been analyst-driven, following a theoretical approach as opposed to an inductive analysis, since the researchers were especially interested in two areas: *Creator* and *context*. This led to a number of themes explicitly related to these two areas, and enabled the researchers to gain insight into the predetermined research questions.

Following Braun and Clarke's (2006) guide to performing thematic analysis in six phases (Table 3), a semantic approach was employed where the researchers stayed close to the explicit meaning of the data without "looking for anything beyond what a participant has said or what has been written" (Braun and Clarke 2006, p. 13).

**Table 3.** Six-Phases Approach by Braun and Clarke (2006).


Coding of the qualitative data was carried out using NVivo 12. With the two overarching themes 'creator' and 'context' already set, the researchers coded the whole data set twice: First, in relation to the participating students as creators, and a second time, with emphasis on the context in which creativity took place (or did not). In both instances, the process involved progressing from the descriptive level of the initially developed codes to summarizing the detected patterns into categories and, furthermore, interpretation of these patterns by defining and naming sub-themes. During this process, a short description for each code was developed (Table 4) before ideas to cluster the codes were brainstormed. The results were then transferred to a spreadsheet which was vital for the process of reviewing and refining the sub-themes. Phase four and five of Braun and Clarke's (2006) approach happened collaboratively and interactively which gave the researchers the opportunity to bring together their unique understandings of the obtained results which will be described in the following section.


**Table 4.** Examples for descriptions developed for each generated code.

#### **4. Results**

*4.1. Study 1*

4.1.1. Descriptive Statistics and Preliminary Findings

Over half the participants were male (n = 325; 57.9%), while almost half of the sample had one sibling (n = 273; 48.7%) while 16.6% of the participants had two siblings, and only 5.8% lived in families with three or more children. 16.9% said that they did not have any brothers or sisters. When being asked about their favourite subject, around 39% of the participants listed Mathematics as their first favourite subject. 15.9% of the students liked Physical Education (P.E.) the most, while 9.1% named Science and 9.4% English as their favourite lessons at school. Around 22% of the participants had either one or both of their parents working in Public Administration, Education or the Health Sector.

When looking at the students' Creative Personal Identity, all items from the SSCS scored very high with the most dominant item being "Being a creative person is important to me" with a mean score of 4.47 out of 5. "Finding a solution is a characteristic which is important to me" had the weakest mean score amongst all five items but was still exceptionally high with a mean score of 4.26. When looking at the Creative Personal Identity scale as a whole, the average is 4.37 (Table 5).

**Table 5.** Creative Personal Identity.


#### 4.1.2. Data Analysis

A correlation analysis that was initially carried out may be found in the supplementary material accompanying this article. This analysis did not indicate any relation between the type of school, the mode of delivery during the school term, or the family situation (e.g., the number of siblings) and any item of the CPI. However, it seems more likely for only-children to attend an independent school (*p* < 0.05). Participants who named English as a favourite subject were likely to not name Mathematics as a favourite subject (*p* < 0.05). Another negative correlation was found between Science as a favourite subject and Mathematics (*p* < 0.05). Several correlations were found between the five items of the CPI scale of the SSCS (Table 6). These inter-item correlations are in line with a study by Karwowski et al. (2018) that confirmed high internal consistency of the SSCS after its development.


**Table 6.** Regressions (State Schools and Church Schools; dependent variable = CPI average).

\*\*\* *p* < 0.001; \*\* *p* < 0.01; \* *p* < 0.05 (2-tailed).

Subsequently, regression analysis was performed for different models. To see overall effects, a regression was run with all three types of school entered in the model as independent variables. This decision is based on findings by Briguglio et al. (2022) who looked into creativity in relation to different school types in the Maltese school context as well and found church schools to be negatively related to students' creativity.

#### 4.1.3. Summary of the Findings

Through the analysis of our findings it emerged that Year 6 students feel less creative than Year 5 students (B = −0.159, *p* < 0.05). Moreover, state schools are significantly and positively associated with CPI (B = 0.223, *p* < 0.05), while no significant effect was found for church schools (Table 6). In this model, the Independent School variable was excluded by SPSS due to multicollinearity with the tolerance being 0. The students' gender and number of siblings did not show any impact.

#### *4.2. Study 2*

The emerging findings were divided into two overarching themes; creators and context. Sub-themes have also emerged as indicated in the findings below (Tables 7 and 8).




**Table 8.** Summary of emerging findings—Overarching Theme Context.

#### 4.2.1. Theme: Creators

The creators represented by students aged 9–11 emerged as a strong overarching theme in the study. An illustration of the subsequent sub-themes is displayed in Table 8. Overall, participants showed a very nuanced understanding of creativity even at their young age, ranging from providing implicit definitions of creativity to perceptions of creativity related to the academic definition of the term, covering aspects such as novelty, originality, and usefulness as illustrated in the extracts below:

*"When you think of an idea which is different from the ideas of others, ideas which are unique, and when you let your imagination run wild."* —Participant 2 (Female, Year 6)

*"Something that not everyone can think of"* —Participant 107 (Female, Year 5)

*"The ability to think unique thoughts and to do what no one else has done"* —Participant 15 (Female, Year 6)

*"To do something in your own special way"* —Participant 125 (Female, Year 5)

As seen above (Participant 15), originality was also associated with 'uniqueness' in a number of cases and in one case 'special'. The terms 'imagination' or 'invention' were also often mentioned when the students were asked to define creativity. Imagination and other cognitive functions including 'thinking', 'use your brain', 'dreaming', were attributed to creativity when participants defined creativity on a number of occasions. These comments were often linked to action-oriented tasks which involved thinking and doing such as building and creating objects. Although there appeared to be a predominant link between creativity and the arts and crafts, creating as a process within a constrained environment and involving problem-solving skills also emerged as a result of the process of imagination and creation when linked to creativity as indicated in the extracts below:

*"I made it using my imagination and from things I found on the ground"* —Participant 142 (Female, Year 5)

*"I think it is a way to help people or fix situations"* —Participant 22 (Female, Year 6)

*"I think creativity is when someone invents something from scratch"* —Participant 639 (Male, Year 6)

Creativity was also referred to as "something important to have" and "useful in life", "fun" and as a form of self-expression to externalise feelings and ideas and in order to develop and in relation to personal growth:

*"It helped me believe in myself more"* —Participant 6117 (Male, Year 6)

The importance of creativity appears to be linked to the consequential psychological development of the students such as building of self-confidence for instance through the praise and acknowledgement the students received for being creative, for example, when their teachers, peers or families liked and approved of what the respondents created. Moreover from a more social perspective, some students remarked that creative ideas and actions can impact our surroundings though problem-solving:

*"leads the world to be a better place"*. —Participant 119 (Female Year 5)

Besides exploring the meaning that young students give to creativity, it was also useful to find out when they were creative most and under which conditions. It was interesting to note that students seem to sometimes confuse creativity with creative writing (essay writing) which takes place in languages. This could be due to the fact that on day-to-day occasions, creativity or everyday creativity are not explicitly referred to other than during art lessons referring to artistic creativity and when creative writing tasks are set during language lessons. Moreover, most respondents associated creativity with activities which are extracurricular in nature most of the time.

*"When I do a new trick of football"* —Participant (Jayden Vella (Male, Year 5)

"*During my free time*" —Participant 636 (Male, Year 6)

*"I am creative most when I'm playing, drawing, doing crafts and when I'm day dreaming."* —Participant 104 (Female, Year 5)

*"When I am doing creative writings, during breaks when I play with my friends, when I do a craft and when I do the steam lesson work."* —Participant 636 (Male, Year 6)

Moreover, overlapping with the overarching theme represented by 'context', students seem to attribute conditions that facilitated creativity when asked 'when are you creative most?'. It was evident that they placed considerable value on autonomy, support and feedback in the form of input and the concept of time namely in making the time to think and to play. A request for quiet spaces and silence also emerged strongly in the responses while an element of intrinsic motivation emerged through responses that explicated excitement and anticipation.

*"When I am in a valley and there would be silence"* —Participant 6326 (Male, Year 6) *"I am mostly creative when eager to do something I'm really excited for"* —Participant 617 (Male, Year 6)

In relation to their own creations, when asked 'what is the most creative thing you ever did at school?' and 'why do you think it was creative?' examples of self-perceptions and manifestations of self-belief in the ability to be creative emerged as can be observed below:

*"Think and finish before the rest, coming up with new ideas. I think because not everyone can come up with new ideas"* —Participant 535 (Male, Year 5)

*"My social studies project book in year 4. Mine was the only one like it"* —Participant 5321 (Male, Year 5)

*"An object for the STEAM lesson where I had to create something without spending a lot of money. I made a Christmas tree out of old school plastic bottles, cloth and handmade decorations using reusable things. Because I built it all by myself without looking at any pictures or examples. I also helped in reducing waste"* —Participant 636 (Male, Year 6)

*"When I read extracts from George's Marvellous Medicine and Cinderella to the class during drama lesson. Because I used my own expressions and pacing and I felt proud of doing well"* —Participant 15 (Female, Year 6)

*"I think the most creative thing I did at school was invent a one-word story game to pass the time while we waited for school to start. I think it was creative because no one thought of it an because everyone thought it was fun"* —Participant 2 (Female, Year 6)

#### 4.2.2. Theme: Context

This overarching theme also emerged prominently in our findings. It was split into two sub-themes; 'different places where the students are creative', and 'stimulation through the environment that fosters the student's creativity'.

When asked 'What would help you to be more creative while at school?' the respones varied however there was a strong predominance for 'more time'. More time was linked with thinking—"more time to think", more time to interact with others, more time for craft, drama and music, and longer breaks. Some responses where more concrete in nature:

*"To discuss topics with friends"* —Participant 133 (Female, Year 5)

*"When we're in a lesson that we make things"* —Participant 128 (Female, Year 5)

*"To be given more constructive free-time"* —Participant 538 (Male, Year 5)

*"I think if there are more 'games' related to the subject. Ex. In social studies you have to act a scene"* —Participant 6124 (Male, Year 6)

*"More science experiments"* —Participant 102 (Female, Year 5)

It was noticeable how the element of noise appeared to be important for the participants. Responses included requests for more quiet spaces, "when the class is really quiet". Other responeses included:

*"It will help me be more creative if there wouldn't be as much noise."* —Participant 27 (Female, Year 6)

*"When around me is very quiet and calm"* —Participant 97 (Female, Year 6)

The physical environment and its relationship with their creative activities, appear to bear considerable importance for the young respondents. The conditions appear to bring together a set of criteria that could make the environment more conducive to creative possibilities which often fall under the guise of social form of context. Clear distinctions were noticed in the responses registered when students referred to creativity while at school or at home. At home, the environment is less rigid and time is not controlled by the timetable or a bell. This appeared to allow the students more opportunities for engagement with creative activities. At school, specific subjects like Science and Physical Education were named often besides classes that are creative by nature, such as Creative Writing, Drama, Art or Music lessons. In general, the students wished for more creative activities during school ("*Creativity lessons would help me be more creative*", Participant 6312, Year 6, male).

To conclude our analysis we found that the response below captures beautifully the student's perception of creativity, his self-belief about creativity and an acknowledgement for the need of creativity in everyday life:

*"When you use your brain and come up with something creative On a scale from 1–10, I give myself an 8.5 (I do think I am creative) I am creative most when I have spaces to attack in the midfield in football*

*Yes—Without creativity you cannot do a lot of the subject"* —Participant 6124 (Male, Year 6)

#### **5. Discussion**

Studies about the definition of creativity and CPI in adults are abundant in literature. In view of the importance attributed to creativity in the list of 21st Century Skills, it was thought that gaining a better understanding of how a future generation conceives the concept and how they feel about themselves as creative persons was a timely exercise to execute.

The aim of this study was to gain an understanding of what meaning young students attribute to creativity, to understand if they identify themselves as creative individuals and to explore the impact of the school environment on their creativity. A review of the literature led to the realisation that creativity is an important part of education (Craft 2003; Gormley 2020) and that it can be achieved during the learning process (Karwowski 2015). This further enhances the need to understand creativity as a construct from the point of view of young students as recipients and contributors to processes that may lead to creative outcomes.

In line with explicit definitions of creativity (Amabile 1996; Runco and Jaeger 2012), the respondents attributed originality and novelty to creativity. Moreover, there was an emphasis on the uniqueness of creative activities. From a more personal perspective it was interesting to observe more action-oriented definitions related to thinking and doing thus supporting the view expressed by Plucker et al. (2004). Thinking as a process also emerged noting the fact that the young participants acknowledge cognitive processes in some way. This is also indicative of their psychological development (Stricker and Sobel 2020). Focusing on the creators, namely young students in primary school and the context; of their respective schools, we tried to better understand how the participants identified with creativity in an effort to find elements that might be assumed or overlooked by adults who run the education facilities. Overall, the findings indicate that Year 6 students feel less creative than Year 5 students. A number of factors might account for this including the stress that the national benchmark examinations may impose on the students. Among the key findings we found that the students in general seem to believe that they are creative individuals and that creativity is important for them. Interestingly, many of the students who identified themselves as creative seem to have Mathematics as their favourite subject, but not science which at that age is still experiment based and very generic. Although the students in general felt that creativity is important for them, solution finding was not as strongly correlated.

Looking at Study 2 which involved students who also took part in Study 1, it was striking to see how clear the notion of time is to the respondents. Time was identified as one of the main sub-themes in relation to requests for more time to think and to engage with activities that promote the creative process including problem-solving. This is in contrast with findings from Konstantinidou et al. (2014). Echoing Davies et al. (2013), a review of pedagogical approaches that may enhance the school environment to make it more conducive to creative activity is highly relevant to promote little-c creativity (Kaufman and Beghetto 2009). The findings of the current research indicate that students seem not to have enough time for trial-and-error activities due to the highly structured curriculum. Therefore, in support of the claims by Craft (2003) and Gormley (2020), creativity may still find it difficult to penetrate the thick walls of the curriculum and rigid syllabi that might adopt traditional teaching.

The psychological development of the respondents had an important role in this study. As outlined by Karwowski (2015) the self-concept is fully visible by 10. It was insightful for us as reaserchers to observe the level of understanding and appreciation of the self-concept of the students. The findings in fact seem to indicate the need to invest further in self-perceptions through the provision of activities and reflective practices that the students can engage with. Carrying on with issues related to 'time' and moving away from academic performance and the influence that it may have on the self-perception of individuals, it was clear that the students do not have many opportunities to think about their own self-development.

When exploring the context, it was noticeable, how action-oriented the responses were. It felt like the students had 'recommendations' that could be implemented if they had the opportunity to voice their opinions and concerns.

Exploring the influence that the context may have on the development of creativity, the current study shows some support for findings by Davies et al. (2013) since respondents referred to physical, pedagogical and also the external environment and the subsequent factors referred to by the authors. Respondents appeared to place some emphasis on their preference for calm surroundings where more greenery could be enjoyed perhaps outside the physical classroom. In relation to the context in school, of particular importance are emerging findings relating to the need for students to have access to down-time and quiet spaces and a wish for more autonomy.

Although we are pleased with the outcome of the study, some limitations need to be acknowledged. These limitations were related to the extent of data that was collected and the data collection method. Due to restricted access in schools induced by the COVID-19 pandemic at the time of data collection, online methods were used. It would have been interesting to access the school and follow up with interviews of focus groups. Moreover, psychological factors including openness to experience, conscientiousness and agreeableness were not included in this study. It would have been interesting to see how these constructs could relate to CPI in a sample population so young. Moreover, it would be of interest to see what the results of a longitudinal study would be as the young students grow into adolescents and young adults. Exploring the role of the creators and how it evolves in changing contexts could inform how creativity can best be harnessed to enhance the capabilities of young people in an effort to further develop their 21st Century Skills.

#### **6. Conclusions**

Findings from the two studies indicate that young students have adequate interpretations of the term 'creativity' while statistical tests revealed that the sample population scored highly on their self-perceived CPI. Schools are often referred to as places where creativity fades, however, plenty of opportunities exist to enhance engagement with skills that promote it in its different aspects.

**Supplementary Materials:** The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/jintelligence11030053/s1, Table S1 Correlation Analysis, Table S2 Questionnaire.

**Author Contributions:** Conceptualization, M.M.; Methodology, M.M.; Validation, J.A.R.; Formal analysis, M.M. and J.A.R.; Investigation, M.M. and J.A.R.; Resources, M.M. and J.A.R.; Data curation, J.A.R.; Writing—original draft, M.M.; Writing—review & editing, J.A.R.; Visualization, J.A.R.; Supervision, M.M.; Project administration, M.M.; Funding acquisition, M.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received a seed fund from the University of Malta-IOTRP02.

**Institutional Review Board Statement:** Ethics clearance was obtained from the following entities: University of Malta, Faculty of Education reference code: 7908\_12042021. Directorate for research, life long learning and employability, Ministry of Education, Secretariat for Catholic Education.

**Informed Consent Statement:** The parents or legal guardians of all the students (minors) involved in this study gave their assent to participate. Permission from the schools was also obtained prior to the study.

**Data Availability Statement:** The data that support the findings of this study are available from the corresponding author upon reasonable request.

**Acknowledgments:** We wish to experess our gratitude to Leonie Baldacchino for her support.

**Conflicts of Interest:** The authors declare no conflict of interest. The University of Malta had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

#### **References**

Al-Dababneh, Kholoud Adeeb, Eman K. Al-Zboon, and Jamal Ahmad. 2019. The creative environment: Teachers' perceptions, self-efficacy, and teaching experience for fostering children's creativity. *Early Child Development and Care* 189: 1620–37. [CrossRef] Amabile, Theresa M. 1996. *Creativity in Context: Update to "The Social Psychology of Creativity"*. Boulder: Westview Press.

Beers, Sue. 2016. 21st Century Skills: Preparing Students for THEIR Future. 21st Century Skills, 1–6. ASCD. Available online: https://cosee.umaine.edu/files/coseeos/21st\_century\_skills.pdf (accessed on 26 January 2023).

Beghetto, Ronald A. 2006. Creative self-efficacy: Correlates in middle and secondary students. *Creativity Research Journal* 18: 447–57. [CrossRef]

Beghetto, Ronald A. 2016. Creative learning: A fresh look. *Journal of Cognitive Education and Psychology* 15: 6–23. [CrossRef]

Beghetto, Ronald A. 2019. *Beautiful Risks: Having the Courage to Teach and Learn Creatively*. G–Reference, Information and Interdisciplinary Subjects Series. Lanham: Rowman & Littlefield.

Beghetto, Ronald A., and James C. Kaufman. 2007. Toward a broader conception of creativity: A case for "mini-c" creativity. *Psychology of Aesthetics, Creativity, and the Arts* 1: 73–79. [CrossRef]


Craft, Anna. 2003. Creative thinking in the early years of education. *Early Years* 23: 143–54. [CrossRef]


Erikson, Erik Homburger. 1968. *Identity, Youth and Crisis*. New York: W. W. Norton & Company, Inc.


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

## *Case Report* **Bored, Distracted, and Confused: Emotions That Promote Creativity and Learning in a 28-Month-Old Child Using an iPad**

**Shiva Khalaf, Hechmi Kilani, Melissa B. Razo and Elena L. Grigorenko \***

Department of Psychology, University of Houston, Houston, TX 77004, USA **\*** Correspondence: elena.grigorenko@times.uh.edu

**Abstract:** Digital technology is increasingly becoming a part of daily life, including the lives of children. Portable digital devices are omnipresent and integrated into activities that did not previously require them. The related skills are often referred to as 21st-century skills, constituting a new type of literacy: digital literacy. These devices and skills bring unique, innovative elements to the learning experience; yet, we do not know the extent to which behavior, emotion, and socialization are affected by such experience. For preschool-aged children, interactions with digital devices and games for the purposes of learning can lead to a state of confusion and boredom, an emotional driving force that may generate mind-wandering and exploration, which, in turn, may facilitate learning. Our interdisciplinary observational case study examined the behavioral patterns linked to digital gamebased learning (DGBL) by observing how a child's mind-wandering contributed to iPad use when they were allowed to freely engage with the device and explore independently during the learning process. Building on a previous case study of a 28-month-old boy, "Ryan", we evaluated the effects of bouts of mind-wandering as he played various DGBL applications (apps) by examining the length of time that Ryan exhibited relevant affective and behavioral states, iPad manipulations, and social interaction during the playtime. Ryan's interactions with the iPad were video recorded for five weeks, and the video footage was coded using a detailed rubric. The results indicated that negative emotions, such as boredom, distraction, and confusion, if coupled with attentiveness and persistence, led to positive mind-wandering and positive learning outcomes. However, when boredom was coupled with frustration, it led to negative mind-wandering and a lack of learning outcomes. In conclusion, our study presents evidence that DGBL apps may improve learning by capitalizing on positive and avoiding negative mind-wandering.

**Keywords:** digital game-based learning; mind-wandering; preschool children; creativity; creative problem solving; learning

#### **1. Introduction**

The American Academy of Pediatrics (AAP) has several recommendations regarding screen time for children aged five and below. These recommendations encompass an array of media devices: televisions, computers, and mobile devices. It is advised that parents not allow a child between the ages of two and five more than one hour per day in front of a screen. This one hour should consist of high-quality programming—which generally means the programming should be educational in nature—and should be consumed while in the company of a caregiver (Council on Communications and Media 2016). The AAP has also acknowledged that new evidence is coming forth which suggests that interactive applications (apps) used on touchscreen devices may be effective educational tools for children aged 24 months or older. However, this research was carried out with unique, experimental apps not available to the public (Council on Communications and Media 2016). The COVID-19 pandemic introduced a situation within education that was not foreseen, and its duration has brought to the attention of educators and parents how needed digital literacy, a 21st-century skill, and effective digital education opportunities are for children

**Citation:** Khalaf, Shiva, Hechmi Kilani, Melissa B. Razo, and Elena L. Grigorenko. 2022. Bored, Distracted, and Confused: Emotions That Promote Creativity and Learning in a 28-Month-Old Child Using an iPad. *Journal of Intelligence* 10: 118. https://doi.org/10.3390/ jintelligence10040118

Received: 28 July 2022 Accepted: 22 November 2022 Published: 2 December 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

(UNESCO 2021). In light of this recent experience and the AAP's finding regarding interactive apps, we wanted to investigate whether digital game-based learning (DGBL) via the currently available interactive iPad apps exhibits the same efficacy as the experimental apps. In studying this, we hoped to uncover that DGBL offers a learning benefit to children in today's changing, 21st-century technological environment and that this benefit should be capitalized on in preparation for the new natural and societal challenges.

The use of digital media has been reported to have both negative and positive outcomes. With regard to the former, excessive media use has been associated with poor motor skills and increased physical inactivity (Felix et al. 2020), decreased attention (Vigil 2019), unfavorable psychological outcomes with regard to mental health, cognitive functioning, and academic achievement, and poor language development (Duch et al. 2013; Oswald et al. 2020). Such findings possibly stem from a lack of child–parent interaction and reduced quantity and quality of playtime for these children, especially since a child's indirect interactions with an adult caregiver, such as praise, indirect commands, and questions from the caregiver, have been shown to boost their self-esteem, exploration, and creativity (Bonnette et al. 2001).

Regarding the latter, one possible benefit from DGBL could be the freedom to engage in mind-wandering—those moments when attention shifts away from the task at hand and onto something unrelated. Mind-wandering may provide a child time and space to engage in creative problem solving, also referred to as a time of creative incubation (Baird et al. 2012), that enhances their ability to learn. Behnamnia et al. (2020a) conducted a review of 20 published articles to determine if the studies presented evidence of positive outcomes in children ages three to six who engaged with DGBL. They found several positive learning outcomes such as creativity (defined in various ways, including finding creative solutions to problems and creative play), self-efficacy with the technology, increased critical thinking skills, and better learning performance (Behnamnia et al. 2020a). The authors observed that the use of digital games, particularly on a mobile device, could increase learning in multiple disciplines and academic subject areas from science/math to the liberal arts by using techniques that allow children to visualize the concepts in everyday life, such as the ability to "see" invisible abstracts, or engaging children through attractive means like storytelling (ibid.). Another study demonstrated that mind-wandering provided an incubation period for the participants by having them attempt to solve the same problem set before and after allowing the mind to wander (Baird et al. 2012). Among the four experimental conditions, it was shown that the group who engaged in an undemanding task—in this case, " ... a choice reaction time task (0-back) requiring infrequent responses" (ibid.)—scored higher in the second round of testing than the other groups, providing evidence that when a person's attention is moved away from the task at hand and toward something that allows them to mind-wander, the person is prone to creative incubation (ibid.).

In an attempt to elucidate contradictory findings, the helpfulness and adaptability of mind-wandering and its derived costs and benefits were reviewed by Mooneyham and Schooler (2013). While they found that mind-wandering has been observed to have negative effects on mood, reading comprehension, sustained attention, and certain other cognitive functions, they observed that it has also had positive outcomes (ibid.). Based on the compiled literature, it appeared that while mind-wandering can detract from certain performance factors such as the ones listed above, it benefits creative thinking and dishabituation from the task. It has been stated that these processes, in turn, can lead to enhanced learning.

According to this synthesis of the literature, it is possible that there are many benefits to be derived for young children's learning while using DGBL apps on a mobile device when these apps allow a child to engage in mind-wandering that leads to creative problem solving. However, no child under the age of three was tested or observed by any of them. Additionally, only one of the studies pointed to self-efficacy with digital technology as a measure of learning while using numerous academic or cognitive measures (Behnamnia

et al. 2020a). The ultimate goal of learning is to obtain and increase skills (academic as well as others), but if a child is to learn those skills on a digital platform, it is important that they are able to also learn how to navigate the platform itself. The creative incubation triggered or facilitated by moments of mind-wandering may help a child to learn, not only the academic material being presented, but how to effectively navigate the digital environment in which the material is being presented. To our knowledge, there have not been any empirical studies attempting to combine the two conditions, i.e., allowing the mind to wander while using a DGBL app. As stated above, the COVID-19 pandemic exemplifies one of the challenges of the 21st century, and we believe that it will greatly benefit children to become proficient in using DGBL apps. Correspondingly, the field of cognitive psychology and related fields need to understand the balance of benefits and obstacles related to this form of learning. Therefore, there is a concern about the lack of studies on mind-wandering and its relationship to preschool-aged children's use of DGBL apps.

Our current study is the second in a series of case studies with a participant named "Ryan." At the time of the data collection, Ryan was a 28-month-old child for whom this was the first systematic iPad exposure. In the first study (Zhukova et al. 2020), Ryan was given several opportunities over the course of five weeks to play DGBL apps of his choice in the presence of one or more caregivers. These play sessions were video recorded, and the recordings were coded for affects, behaviors, verbalizations, and iPad manipulations displayed by Ryan. These data were combined with his performance outcomes on the app he chose to play most often, *Doodle Dots*, to identify relationships between his affects and learning performance, defined as his acquired digital literacy and academic performance. The recordings were also coded for caregiver verbalizations to explore the roles of a caregiver and of social interactions in digital literacy acquisition and learning. The findings indicated that Ryan increased his speed and manipulation proficiency in using the iPad over the period of five weeks. The frustration he displayed during use correlated to a negative mind-wandering outcome—an increase in the number of errors he made—while displays of attentiveness, help-seeking, and persistence correlated to a positive mind-wandering outcome: his generation of novel situations and increased creativity.

Our study further investigated how the use of a DGBL app can contribute to a child's learning, defined as the development of app navigation skills and effective iPad manipulations, and how mind-wandering is a positive contribution to that learning. While mind-wandering is an abstract mental activity, in psychological research, it has been operationalized as task-unrelated thought (Murray and Krasich 2022). In the current study, this construct was quantified as moments when Ryan shifted his attention away from the task at hand and onto something unrelated as a result of his boredom, distraction, and/or confusion. Past studies have shown that if such distractions and task-unrelated thoughts are coupled with attentiveness and persistence, they could lead to positive mind-wandering that facilitates creative problem solving and greater learning (Belton and Priyadharshini 2007; Bonnette et al. 2001; Kort et al. 2001; Pachai et al. 2016; Zhukova et al. 2020). Conversely, if the tendency to shift attention from the here-and-now is combined with frustration, it could lead to negative mind-wandering and a lack of positive learning outcomes (Kort et al. 2001; Maloney et al. 2015; Pachai et al. 2016; Zhukova et al. 2020). Because past studies have used self-reporting by the participants to measure the amount of mind-wandering, and because a child at this age cannot effectively carry that out, we gauged his bouts of mind-wandering as he used the game apps by recording the moments of boredom or distraction as per an affect coding rubric we developed. These affects were coded for frequency during the first study, but during the second study, we coded for the duration of time the child exhibited such behavior. Additionally, in this rendition of the data, Ryan was observed as he played several different apps on the iPad, not only *Doodle Dots*. Our hypothesis was that, as he played these interactive, educational game apps on the iPad, he would engage in mind-wandering, and this mind-wandering would lead to periods of creative incubation that would mediate positive learning outcomes.

#### **2. Materials and Methods**

#### *2.1. Participant*

Ryan is a 28-month-old Caucasian male raised in a bilingual, Russian–English household. He began attending preschool at 18 months of age. Our study gathered observational data on Ryan while he was in the care of a childcare provider (hereafter, caregiver) three times per week for five weeks, which resulted in 15 timepoints of data collection. Ryan is a typically developing child; for more information regarding Ryan's developmental skills, please see Zhukova et al. (2020). The study was approved by the Yale IRB.

#### *2.2. iPad Apps Used*

*Doodle Dots* (PBS KIDS Sprout 2011) is a game targeted at preschool children. In this application, children respond to verbal prompts to connect dots and complete a picture. The "dots" the child is required to connect are either numbers, colors, geometric shapes, or fruit. A child can select their choice of dots from icons at the top of the screen and can select the picture puzzle they wish to complete from a gallery of images, which includes animals, toys, foods, vehicles, etc. If the child selects the wrong dot after a prompt, they will receive corrective feedback (e.g., "That's not a green dot!"). There is no time limit or limit to the number of incorrect responses a child can receive.

*ABC Go* (Peapod Labs 2010) focuses on exposing children to a variety of vehicles and methods of transportation by exploring the alphabet and linking letters to words. This application teaches children new words through sight, sound, and touch. After choosing a letter icon, a child is exposed to a means of transportation that begins with that specific letter. The child can also hear a description of the vehicle, see a video of the vehicle in use, or play a puzzle in which they must "find" the vehicle by filling in its outline as delineated by a dashed line.

*Memory King* (Innovative Mobile Apps 2011) is a card-matching game that seeks to reinforce memory skills and develop vocabulary. The child chooses the number of cards to be matched, which can range from 2 to 32 pairs. The categories of cards utilized in this study include animals, numbers, letters, shapes, and colors.

*Monkey Preschool Lunchbox* (THUP Games 2009) features seven minigames within the context of a monkey trying to fill his lunch box. After every few rounds of the game, a child is rewarded with a virtual sticker that they can drag onto a bulletin board. None of the games have time limits, and the monkey in the corner provides positive and negative feedback with respect to how the child responds to a given prompt.

*Memory Train* (Piikea St. LLC 2011) presents its memory development activities in an adaptive play environment. It uses a variety of objects that test the memory of colors, facial expressions, and specific aspects of larger pictures.

*Agnitus'* (Agnitus 2012) multitude of activities covers 22 different academic skills based on the Common Core State Standards. This game allows a child to master a variety of skills, such as the recognition of colors and shapes, basic counting and sorting, and matching objects and letters.

*Counting Ants Lite* (Playtend Apps LLP 2009) helps young children learn the numbers one through ten. The app cycles through a series of scenes and minigames in which a gradually increasing number of cars encounters obstacles along its drive. The app seeks to introduce children to the concept of quantity in a variety of ways. There are no auditory instructions, and while there is text onscreen indicating what to do at each point in the game, a child need not know how to read to follow the intuitive style of play.

#### *2.3. Study Design*

Ryan was presented with an iPad. Without receiving any instruction on how to operate the device, he was allowed to play the DGBL apps available on it. Each play session lasted an average of 25 min, and Ryan's behavioral interactions with the iPad and his caretaker were video recorded. This occurred over the course of 5 weeks for a total of 6:27:33 hours of footage. The video recordings were then observed and coded for Ryan's affects, behaviors, and iPad manipulations, as well as for the verbal interactions that occurred between him and the caregiver.

#### 2.3.1. Experimental Setting

The experimental setting was a naturalistic setting, in the home of Ryan's day caregiver. Both he and the experimental caregiver sat at a quiet table and interacted with the iPad. The setting was not controlled or altered in any way besides minimizing interactions with members of the household during play. Experimental priority was placed on the dyad, Ryan and his caregiver.

#### 2.3.2. Coding Scheme

Behavioral coding of the video footage was conducted by seven undergraduate volunteers working under the supervision of a graduate student for a total of eight coders. The undergraduate coders were trained by the supervisor for approximately 2 months on the coding process and proper use of the coding rubric. During this time, the coders met one or two times per week to compare their practice codes, calibrate them, and refine the rubric.

After training, the 15 recordings were divided into two groups, and four coders were randomly assigned to each group. Each recording was randomly assigned a "main coder" to view and code its entirety, while the other coders within its group were assigned a randomly chosen portion of the video consisting of 20% of its length to quadruple-code.

To conduct this coding, the free and opensource Behavioral Observation Research Interactive Software (BORIS) was used. BORIS was chosen because of its accessibility and its ability to code for both the frequency of behaviors and the duration of time for which behavior was observed. The list of key behaviors was defined in the software's Ethogram. Coders were able to specify a keyboard key assigned to each behavior to record the corresponding behavior. The recorded events were exported as behavioral binary data with a time interval set as 1 s. That is, all variables were coded as 1 for the presence of that behavior or 0 for lack thereof for each second of the video footage. The descriptions of all behaviors are listed in Table 1.


**Table 1.** Child coding rubric of behavior, affect, verbalization, and iPad manipulation states.




*Note.* \* Empty reference cells indicate behavioral states that were defined by the authors of this study.

#### *2.4. Inter-Rater Reliability*

The total video footage during the 15 observation sessions was 6:27:33 hours, with a mean duration of 25:50 min (min = 5:31 min, max = 49:31 min). As mentioned, the coding was conducted by eight independent student coders who quadruple-coded 20% of each recording. Inter-rater reliability was computed as an intraclass correlation coefficient (ICC) for multiple raters, as discussed in Shrout and Fleiss (1979), for each category of the analysis. For child affect, child behavior, child and caregiver verbalizations, and iPad manipulations, an ICC of 0.88, 0.75, 0.58, and 0.71, *p* < 0.001 for all, was achieved, respectively. Such values are indicative of moderate–good reliability estimates (Koo and Li 2016).

#### **3. Results**

#### *3.1. Descriptive Statistics*

Table 2 presents a tetrachoric correlation matrix that depicts the associations between the observed behavioral and affective states of Ryan while he played on the iPad. Key assumptions for the use of the tetrachoric correlation were met. Namely, (a) the underlying distribution of the data was bivariate normal, (b) all variables were linearly related, and (c) the error terms were normally distributed. Ryan's boredom, defined as his lack of interest in an activity or a slow, unmotivated response to the task at hand, negatively correlated to both his attentiveness (r = −.55) and frustration (r = −.86), and positively correlated to his persistence and exploring (r = .28 and .21, respectively). His attentiveness was moderately associated with his sense of delight (r = −.55) and smiling (r = −.47). The negative correlation indicates that when Ryan was engaged in responding to the app, he did not display any emotion. His frustration positively correlated to his persistence in playing with the iPad (r = .60). While he did not display delight when in a confused affective state (r = −.99), he did seek help from his caregiver (r = .88). There was no significant association between the states of boredom and confusion.


**Table 2.** Correlation coefficients depicting the degree of association among observed behavior and affect states.

*Note*. Bold entries denoted are not statistically significant at *p* < .05; all other elements are statistically significant at *p* < .01.

Figure 1 depicts the duration of time (measured as the ratio of observed behavior to recorded footage of each occasion) that Ryan exhibited various behavioral and affective states across the 15 timepoints. Notably, Figure 1 is provided only for descriptive purposes; to test for statistical significance, logistic regressions were computed to examine relative change across time. We discovered that as the days passed, Ryan became more attentive, began to explore new games, was less distracted, and had fewer occurrences of eye contact between himself and the caregiver.

**Figure 1.** Changes in behavior and affect states over time. Duration of time is measured as the ratio of observed behavior to the recorded footage of each occasion.

The frequency of Ryan's several iPad manipulations and social interactions during our study are presented in Table 3. During the first few sessions in which Ryan was unfamiliar with iPad navigation, he repeatedly chose to play the same app. He randomly tapped the screen and showed signs of ineffective iPad manipulation (gestures such as hitting the iPad with a palm or fist). He sought attention from the caregiver via verbalizations or eye contact and/or pointed at the source of the problem. As the study progressed, Ryan's

help-seeking behaviors lessened in frequency as his displays of frustration diminished. Furthermore, as his skills developed (e.g., he learned to manipulate the iPad effectively), he began switching between apps more frequently, had fewer questions for his caregiver, and used fewer inappropriate and ineffective manipulations. We believe the increase in his ability to switch between apps is a sign that, as he became more proficient, he got bored. He lost the desire to play on a single app and became interested in exploring the other games that were available to him. As boredom was one of the chosen means to identify Ryan's bouts of mind-wandering, we believe Ryan engaged in creative problem solving when he grew tired of playing an app, leading to the learned ability to navigate away from his current game to something novel.


**Table 3.** Total occurrences of behaviors across all timepoints.

*Note.* For a description of each behavioral category, please see Table 1.

Ryan's caregiver provided direct and indirect commands and raised questions to guide Ryan throughout his play sessions. The caregiver also used verbal praise to encourage Ryan's exploration and participation in the gaming apps.

#### *3.2. Analyses of Observational Data*

Affects and behaviors observed in the video footage were coded such that a code of 1 indicated the presence of behavior for each second of the recording. In order to examine the statistical significance of a relative change in Ryan's behavioral states, logistic regressions were computed and are presented in Table 4. The results of these analyses indicate that Ryan's attentiveness, frustration, and exploration increased significantly over time. While Ryan's boredom and persistence did not significantly change across time, his states of confusion, distraction, and help-seeking behaviors decreased. Smiling and delight

also decreased across time. We believe this is evidence that Ryan became more engaged with the DGBL apps since, as previously stated, he did not display signs of positive affect while he was attentive.


**Table 4.** Changes in behavior and affect states over time.

*Note*. Standard errors appear in parentheses. Bold entries denoted are not statistically significant at *p* < .05; all other elements are statistically significant at *p* < .01.

Our hypothesis that mind-wandering would lead to positive learning outcomes, defined as Ryan's ability to creatively problem solve, and gain the ability to manipulate the iPad effectively and its apps, was confirmed by the analysis displayed in Table 5. When exhibited alone, the main effects of boredom and distraction (mind-wandering) both significantly predict attentiveness (β = −2.087 and β = −2.141, *p* < .01, respectively) and do not predict frustration. However, the interaction between boredom and distraction predicts neither attentiveness nor frustration (β = 1.835 and β = .798, *p* = ns, respectively). We also observed that scaffolding in the form of the caregiver's questions to Ryan predicted Ryan's attentiveness and frustration (β = .007 and β = −.066, *p* < .01).

**Table 5.** Logistic regression analysis of predictors of attentive and frustration states.


*Note*. Attentive and frustration and their related predictors were analyzed in separate models. Standard errors appear in parentheses. Bold entries denoted are not statistically significant at *p* < .05; all other elements are statistically significant at *p* < .01.

We did not examine the statistical significance of changes in Ryan's iPad manipulations across time. They were coded as the frequency of occurrences across 15 timepoints; thus, we did not have sufficient power to detect significant changes across time.

To better examine Ryan's learning as evidenced by proper handling and navigation of the device, coded iPad manipulations that were considered appropriate included occurrences of dragging, swiping, and tapping, while inappropriate manipulations included occurrences of hitting the device or using the palm of the hand and/or multiple fingers in an attempt to navigate or play a game. We expected that confusion and

attentiveness would predict an increase in navigation proficiency. A linear regression model confirmed this hypothesis (R2 = 0.017, F (3, 23,249) = 129.71, *p* < .01). Namely, confusion (β = 65.952, *p* < .01) and attentiveness (β = 36.62, *p* < .01), and their interaction (β = −145.719, *p* < .01) significantly predicted variance in navigation proficiency. Frustration (β = 81.559, *p* < .01) predicted a significant level of variance in inappropriate navigation (R2 = .008, F (3, 23,252) = 64.28, *p* < .01). Furthermore, Ryan's help-seeking behavior predicted variation in navigation proficiency (R2 = .069, F (3, 23,252) = 573.72, *p* < .01). While the main effect of help-seeking was significant (β = 69.476, *p* < .01), the interaction was not significant over time (β = −4.423, *p* = 0.164).

#### **4. Discussion**

This case study with Ryan—a child younger than 3 years of age and thus of an understudied age group in the subject of interest—examined whether mind-wandering is a positive contribution to a child's learning within the specific context of the use of digital technology when this technology is encountered, in a systematic way, for the first time. We defined learning as the development of app navigation skills and effective iPad manipulations. This study is unique in that typical DGBL studies do not consider media navigation literacy and traditionally prefer to focus on academic material instead (Behnamnia et al. 2020a; Tan et al. 2015). We defined mind-wandering as moments of boredom or distraction and concluded that mind-wandering contributed to positive learning outcomes when Ryan exhibited more attentiveness to the games he played and was able to effectively navigate to an app he liked and within the app itself.

Past studies have shown that a child's affective state is an important factor related to their learning achievement (Maloney et al. 2015; Valiente et al. 2012). Negative emotions such as boredom, distraction, and confusion, when coupled with attentiveness and persistence, could lead to positive mind-wandering that facilitates creative problem solving, mediating learning (Belton and Priyadharshini 2007; Bonnette et al. 2001; Kort et al. 2001; Pachai et al. 2016; Zhukova et al. 2020). However, the combination of frustration and boredom could lead to negative mind-wandering and a lack of positive learning outcomes (Kort et al. 2001; Maloney et al. 2015; Pachai et al. 2016; Zhukova et al. 2020). These findings are not limited to a specific age range, as experimental designs of mentioned studies included both children of school age as well as adolescents. Importantly, our study focuses on a toddler, widening the age range present in the relevant publications. Although we present a case study, the overall emotional pattern of the engagement of an iPad seems to fit the current literature.

Ryan's increased ability to switch between iPad apps is a display of the facets of creative thinking that pertain to dealing with novelty, allowing him to learn how to effectively navigate the digital device (Behnamnia et al. 2020b). The caregiver's use of verbal praise and other scaffolding encouraged Ryan to explore the available apps and continue to interact with the device (Bonnette et al. 2001). This highlights the importance of child– adult interaction while operating a digital media device. Through the course of the study, we found that Ryan's states of confusion and distraction, in addition to his help-seeking behaviors, gave way to attentiveness and increased proficiency while using the device. We believe this to be evidence that a child who is allowed to engage in mind-wandering while in the company of a caregiver will be able to engage in creative problem solving, that this will positively contribute to their learning, and that these results can be achieved using a DGBL app on an iPad. Although an iPad itself, perhaps, does not stimulate creativity, it certainly facilities the development of its facets when (1) an iPad is a novel object (i.e., as it was for Ryan at the beginning of the study); (2) skills that are called for by specific apps need to be formed in response to specific situations of dealing with novelty; and (3) a skill has been mastered, and the child gets bored with it, and mind-wandering leads the player to the formulation of novel challenges while discovering and mastering more complex capabilities of an iPad. These observations add to what we currently know about early encounters with digital technology by technology-naïve young children.

Our results also indicate that the duration of time when Ryan was confused and attentive predicted significant variation in his navigation proficiency. This finding corresponds to that of the four phases of learning proposed by Kort et al. (2001) that elucidate a child's experience of "constructive learning" and "un-learning," the processes by which they acquire new knowledge while discarding incorrect understanding, respectively.

We highlight that this case study—in times when learning from home is becoming part of the norm—is necessary to bring insights into digital literacy and online education in very early childhood (<3 years old).

#### **5. Limitations and Future Research**

The findings of this study provide support for the manifestation of positive outcomes during the use of DGBL apps by preschool-aged children. While the research findings on the general effects of media usage on young children are mixed, experts believe that the active role of adult caregivers in the form of questioning and indirect commands can increase these children's learning, exploration of novelty, critical thinking, and creativity. Thus, mobile devices used in the company of a caregiver may be an effective tool for learning. However, the results of the study should be interpreted with caution, as it was characterized by the following limitations.

First, our study was a case study of a single child, and the generalization of our findings to the population should be applied carefully. Yet, such case studies are very useful as they provide opportunities to investigate underlying principles of an occurrence within a real-life context. Moreover, they are an in-depth investigation of one particular individual, providing a glance into how a young child is handling a tall task of development during his first prolonged exposure to an iPad. A case study allows for the formation of a list of observations that can be verified in future studies. We encourage future researchers to conduct similar studies with groups of children, possibly in both home and school settings, given the changing landscape of how education is delivered.

Second, it is unclear whether or not the app-switching behavior is solely a sign of creative problem solving or whether it is merely a behavior to move away from negative affects, such as frustration and confusion, in order to self-regulate and approach the problem again at a future time. In order to bring clarity to this specific relationship, future research must specifically track task performance and the types of hardships faced in apps. Unfortunately, given the "educational" games that are available, curating a list of evidence-based games may be challenging.

Third, regarding the methods of the study, future endeavors should include a more objective method of affective and behavioral coding. Specifically, researchers interested in the field of DGBL and affective states should inquire about the growing field of affective computing. Affective computing involves systems that detect emotions of use, systems that express what a human would perceive as an emotion, and systems that actually "feel an emotion" (Wu et al. 2016). In other words, affective computing techniques seek to develop algorithms for automatic affect recognition and would involve objective measures, such as autonomic nervous system activity, voice parameters, and facial expressions, complemented with observed behavior in a video (Järvelä et al. 2020). This is the future of the field and would reduce the potential for human errors.

Fourth, it is important to acknowledge that at this young age, it is impossible to elicit a comprehensive and reflective self-report or self-assessment of either learning progress or the corresponding emotions. Therefore, it would be important to evaluate whether the observed emotions correlate with self-reported ones, working, perhaps, with older children. Yet, given the quickly changing reality of child–computer (or digital device) interactions, both with regard to when children start these interactions and for how long they are engaged in them, it might not be possible to identify an age group when children are still naïve to digital devices but already can practice self-evaluation and self-awareness.

While the primary aims of most DGBL apps for young children are to expose them to new material, stimulate their thinking, and add to their knowledge, most are neither

research-based nor psychometrically solid. That is, these apps lack empirical evidence to enhance learning and creativity in young children. Developmental benefits reaped from exposure to high-quality children's media content have been studied at length; however, this research is largely centered on evaluating the content of television programs rather than apps that are accessed on digital devices such as smartphones or tablet/iPad devices (Linebarger et al. 2017). Our case study is a step in obtaining this evidence, but more research is needed to verify that apps that are promoted as "educational" by their designers or retailers truly offer learning benefits to young users. Future educational apps designed for preschool children need to actively involve the child, encourage social interaction with caregivers, and be engaging and meaningful (Hirsh-Pasek et al. 2015).

**Author Contributions:** Conceptualization, E.L.G.; methodology, S.K. and H.K.; software, S.K., H.K. and M.B.R.; validation, S.K. and E.L.G.; formal analysis, S.K.; investigation, S.K., H.K. and M.B.R.; resources, E.L.G.; data curation, S.K. and H.K.; writing—original draft preparation, S.K., H.K. and M.B.R.; writing—review and editing, H.K., M.B.R., S.K. and E.L.G.; visualization, S.K.; supervision, S.K. and E.L.G.; project administration, H.K. and S.K.; funding acquisition, E.L.G. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the US Department of Education grant number R305A160402.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Raw data can be requested from authors due to privacy or ethical.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

Agnitus. 2012. Agnitus. Available online: http://www.agnitus.com/ (accessed on 16 July 2013).


Council on Communications and Media. 2016. Media and Young Minds. *Pediatrics* 138. [CrossRef]


**Cong Xu 1,\*, Chih-Fu Wu 2,\*, Dan-Dan Xu 1, Wen-Qian Lu <sup>1</sup> and Kai-Yi Wang <sup>2</sup>**


**Abstract:** In order to meet industrial demands, some colleges and universities have offered interdisciplinary programs that integrate design, engineering, and business. However, how many changes these programs have brought to students, and whether students participating in these programs have had better interdisciplinary ability than students involved in a single discipline study have always been questions that many researchers want to explore. In a university that offers an interdisciplinary program, we found that there is no significant difference in interdisciplinary integration ability between the students participating in the interdisciplinary program and the students involved in a single discipline study through quantitative comparisons of 91 student questionnaires and analyses of interviews with nine teachers of interdisciplinary courses and other related staff members. This may result from the students' lack of motivation, lack of prior experience, the influence of individual traits, the increase of learning pressure and academic burden, and the interference of disciplinary factors during interdisciplinary learning. The research finding is intended to improve student interdisciplinary learning effectiveness by facilitating interdisciplinary teachers' understanding of the influencing factors of student interdisciplinary learning, and by providing a reference for interdisciplinary teaching design.

**Keywords:** challenges of interdisciplinary learning; learning outcomes; interdisciplinary integration ability; student attributes; learning environment

#### **1. Introduction**

In the past 20 years, discipline-based university education has undergone a great transformation towards interdisciplinary education. Interdisciplinary education and learning have become the focus of education and teaching research today (Klaassen 2018). There has been growing interest in interdisciplinary education and the publications of interdisciplinary education research have increased significantly (Heikkinen and Räisänen 2018). However, how effective is interdisciplinary education? To what extent have students improved their interdisciplinary ability? Although some studies in the fields of medical and nursing education have responded to these issues in recent years (Bullard et al. 2019; Liu 2021), there is obviously a lack of interest in research in the fields of engineering and computer science education (Heikkinen and Räisänen 2018). Overall, due to the challenges of effectiveness evaluation on interdisciplinary education, existing research has paid little attention to the growth and evaluation of students' interdisciplinary ability (Lattuca et al. 2017a; Gao et al. 2020). It is even more difficult to find relevant literature on interdisciplinary teaching or empirical research (Lindvig and Ulriksen 2019; Van den Beemt et al. 2020). Therefore, we could hardly find studies that demonstrate the effectiveness of interdisciplinary education through teaching and learning practices in class (Gao et al. 2020). Are students involved in interdisciplinary education significantly different from students involved in a single discipline study in interdisciplinary ability? What specific teaching and learning factors affect the improvement of students' interdisciplinary

**Citation:** Xu, Cong, Chih-Fu Wu, Dan-Dan Xu, Wen-Qian Lu, and Kai-Yi Wang. 2022. Challenges to Student Interdisciplinary Learning Effectiveness: An Empirical Case Study. *Journal of Intelligence* 10: 88. https://doi.org/10.3390/ jintelligence10040088

Received: 27 July 2022 Accepted: 10 October 2022 Published: 17 October 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

ability during interdisciplinary learning? Little has been learned so far. Biggs proposed a system model for teaching activities, which consists of four parts: student attributes, learning environment, learning process, and learning outcomes. Each part of the model follows the principle of alignment (Biggs 1993). Spelt et al. believed that this theoretical model could more comprehensively explain the interrelationships among various elements in interdisciplinary teaching and pointed out that "interdisciplinary integration ability" as an interdisciplinary learning outcome would be affected by "student attributes" and "interdisciplinary learning environment" (Spelt et al. 2009). This theory was applied and confirmed in subsequent studies by Spelt and Liu et al. (Spelt et al. 2015; Liu et al. 2022). In this paper, we will combine this theory and previous research to discuss the definition and connotation of interdisciplinary integration ability, as well as the factors affecting students' learning outcomes in the area of interdisciplinary learning environment and student attributes.

#### *1.1. Interdisciplinarity Integration Ability*

Spelt et al. believed that interdisciplinary integration ability is also interdisciplinary thinking, including interdisciplinary knowledge and interdisciplinary skills (Spelt et al. 2009). The research findings of Menken et al. show that the key to interdisciplinarity being different from multidisciplinarity is the integration of related concepts, insights, theories and/or methods from different disciplines (Menken et al. 2016) (see Figure 1). Lattuca et al. argued that interdisciplinary ability enables students to integrate knowledge and methods from different domains for a comprehensive understanding of a problem (Lattuca et al. 2017a). Spelt et al. pointed out that the decisive feature of interdisciplinarity is the ability to integrate disciplinary knowledge. If there is no cultivation and training of this ability during teaching, but simply increased knowledge of different disciplines, it can still only be called multidisciplinary education (Spelt et al. 2015). Based on previous research and assertions, it is not difficult for us to come to the conclusion that interdisciplinary integration should be the core and key of interdisciplinarity. Therefore, the improvement of interdisciplinary integration ability should be the key to the evaluation of interdisciplinary teaching effectiveness and the concrete representation of students' interdisciplinary learning outcomes. However, although "interdisciplinary integration ability" is of critical importance to interdisciplinary teaching, there is not yet a unified definition in the scientific and pedagogical literature (Danilova 2018), and the expressions of its connotation vary. For example, the IPEC (Interprofessional Education Collaborative) in the US defines core interdisciplinary ability as values/ethics for interprofessional practice, roles/responsibilities, interprofessional communication, and teams and teamwork. In addition, core interdisciplinary ability defined by the University of Virginia (2016) includes: communication, professionalism, shared problem-solving, shared decision making, and conflict resolution (Chen et al. 2017). Wilhelmsson et al. pointed out that interdisciplinary ability should include: teamwork and group processes, reflection and documentation, communication, shared knowledge or general common knowledge base, and ethics (Wilhelmsson et al. 2012). Interdisciplinary ability advocated by Lattuca et al. (2013) includes: awareness of disciplinarity, appreciation of disciplinary perspectives, appreciation of non-disciplinary perspectives, recognition of disciplinary limitations, interdisciplinary evaluation, ability to find common ground, reflexivity, and integrative skill; and subsequently these eight abilities are extracted into three: interdisciplinary skill, reflective behavior, and recognizing disciplinary perspectives (Lattuca et al. 2013). The definitions of the above-mentioned core interdisciplinary integration ability are slightly different, but they all highlight similar abilities (see Table 1), including: interdisciplinary communication, facilitating the formation of shared knowledge base and problem-solving teamwork, interdisciplinary reflection and evaluation, accepting other disciplinary values or perspectives, having disciplinary awareness and perspectives, being able to recognize disciplinary limitations, integrating knowledge from different disciplines to deal with complex problems, professionalism, and other skills.

**Figure 1.** Difference between Multidisciplinarity and Interdisciplinarity. Collated, modified and drawn from Menken Steph, Keestra Machiel, Rutting Lucas, Post Ger, De Roo Mieke, Blad Sylvia, De Greef Linda. 2016. *An introduction to interdisciplinary research: Theory and practice.* Amsterdam: Amsterdam University Press, pp. 31–32.

**Table 1.** The Above-Mentioned Core Interdisciplinary Integration Abilities.


#### *1.2. Student Attributes*

Student attributes include motivation, individual traits, prior experience, etc. (Spelt et al. 2009, 2015; Liu et al. 2022). Some researchers discussed the motivations and goals of interdisciplinary learners. For example, Barnard pointed out that most students generally hold conflicting views on interdisciplinary learning (Barnard et al. 2013). In their research, Kabo et al. mentioned that some reports indicated that people with engineering educational background put up resistance to interdisciplinary learning goals (Kabo and Baillie 2009). Berasategi et al. believed that student individual conditions, including learning motivation and maturity, are closely related to their development of interdisciplinary thinking (Berasategi et al. 2020). Some scholars discussed that learners' prior experience seems to have an impact on interdisciplinary learning outcomes. Heiman pointed out that freshman students are reluctant to use learning methods different from what they have adopted in high school (Heiman 2014). Studies found that students' prior learning experience of a single discipline makes them feel overwhelmed and at a loss when faced with the teaching design and expectations of interdisciplinary courses (Strain and Potter 2012).

#### *1.3. Interdisciplinary Learning Environment*

Interdisciplinary learning environment includes elements like courses, teachers, pedagogy, assessment, etc. (Spelt et al. 2009, 2015; Liu et al. 2022). Van den Beemt et al. suggested

that any teachers and students involved in interdisciplinary education projects should be aware of the relation among the specific perspectives and visions of interdisciplinary education, and the chosen teaching methods (Van den Beemt et al. 2020). Do believed that interdisciplinary courses should have a goal that can be achieved within a semester, while corresponding tasks should be designed and learning objectives should be set related to the level of difficulty (Do 2013). Chen et al. pointed out that a course study load is critical to the effectiveness of interdisciplinary learning (Chen et al. 2009). Hansen et al. believed the motivations and goals of the teaching program as the basis for an interdisciplinary approach to pedagogy in the context of interdisciplinary curriculum development (Hansen and Dohn 2017). For the setting of teaching content, Biggs emphasized that if students want to obtain the desired learning outcomes, the basic task of teachers is to engage students in learning activities that may lead them to achieve these outcomes; during the process, deciding what students learn is far more important than what teachers do (Biggs 1993). In addition, many scholars discussed teaching activities, curriculum design, teachers, teaching methods, assessment and other topics in terms of interdisciplinary learning environment (Carreras Marín et al. 2013; Gómez Puente et al. 2013; Gouvea et al. 2013; Jones 2010; Lindvig and Ulriksen 2019).

The literature review provides a research framework for us to explore the effectiveness of interdisciplinary curriculum teaching practice that aims at the cultivation of interdisciplinary integration ability, and the influencing factors. Meanwhile, in view of the relative lack of empirical reports on interdisciplinary teaching, this paper will try to find out whether the students participating in an interdisciplinary program have a significant advantage over the students involved in a single discipline study by comparing their interdisciplinary integration ability through an empirical case study. Moreover, this paper will further analyze which elements in the areas of "student attributes" and "interdisciplinary learning environment" affect the interdisciplinary integration ability of the students participating in the interdisciplinary program based on the collected sample data and materials.

#### **2. Materials and Methods**

#### *2.1. Setting and Teaching of Interdisciplinary Courses*

In the face of rapid technological change, global climate change, and an ever-changing market, product innovation and sustainable development of manufacturing are no longer complex problems that can be completely solved by a single discipline. Some scholars pointed out that the life cycle of a product is divided into three stages: design, engineering, and sales, but these three stages are not independent, and on the contrary, they should be integrated (Buxton 2010). The researchers of design education indicated that as industrial projects are becoming increasingly complex and larger in scale, the boundaries between artifacts, structures, and processes are beginning to be more blurred. Since the requirements at each level are rising, the complexity of design problems will be significantly increased. Therefore, designers will be required to be familiar with working in interdisciplinary teams that integrate engineering and business (McDermott et al. 2014). In addition, many successful large international companies, such as GE, Sony, Philip, etc., have adopted the design perspective as a problem-solving tool for the entire company and a key element in the formation of corporate strategies. Design is increasingly recognized as a key to the success of business practices, and design thinking has become increasingly popular in the field of business (Matthews and Wrigley 2017). Gill et al. believed that interdisciplinarity integrating mechanics, electronics, information technology and design is the future of Industry 4.0, and the integration of these majors will create solutions for complex problems faced by intelligent manufacturing, and product innovation and development (Gill et al. 2021). Driven by industrial development, in fact, some educational institutions have begun to try to carry out interdisciplinary teaching activities that integrate design, business, and engineering technology, such as Jiangnan University in mainland China, Arizona State University in the United States, etc. (McDermott et al. 2014; Li et al. 2019).

#### 2.1.1. Setting of Interdisciplinary Courses

The interdisciplinary courses mentioned in this paper are developed and designed by a comprehensive university in Taiwan according to the above-mentioned industrial talent development trend. The university's mission is to cultivate applied and compound talent for industrial development. A great number of leaders of large international enterprises have graduated from the school successively. Every year, the school regularly invites people from the industry, including prominent alumni, to discuss with the school's teachers and educational administrators industrial talent needs as well as current education trends and issues. The school's interdisciplinary program was established in this context. The program is aimed at students majoring in Industrial Design, Media Design, Materials Engineering, Mechanical Engineering, Electrical Engineering, Computer Science and Engineering, Business Management, Applied Foreign Languages, etc. It integrates courses and teaching resources in the fields of business, engineering and design, with the goal of cultivating interdisciplinary integration ability, to form an interdisciplinary curriculum system consisting of interdisciplinary basic courses (i.e., introductory courses for business, engineering, and design majors) + interdisciplinary integration courses like Capstone + internship and practical courses. The interdisciplinary program is intended to improve students' skills in interdisciplinary communication, interdisciplinary teamwork, interdisciplinary reflection and evaluation, interdisciplinary values or viewpoints, disciplinary limitation cognition, and interdisciplinary knowledge integration.

This program offers 23 courses, including Business Analysis, Applied Electronic Creation, Materials Processing and Analysis, Design Fundamentals, Capstone, etc. These courses are arranged in various stages from the first year to the fourth year in this university, and there is a progressive relationship between the courses before and after (see Table 2 for details). This is a semi-closed academic program, exit only and no entry. Therefore, students must join this program in the first semester of their freshman year and those who try to join the program midway are rejected. In addition to completing the courses of their own majors, they need to complete the various courses of the interdisciplinary program. Meanwhile, students can be exempted from taking the interdisciplinary basic courses in this program within their own disciplines. In addition, after the start of the program, the courses that have been registered and selected during the semester cannot be withdrawn, but in accordance with the principle of voluntariness, the participants are allowed to stop the study of subsequent unselected courses and withdraw from the program.


**Table 2.** The Interdisciplinary Program.


#### 2.1.2. Teaching of Interdisciplinary Courses

More than 30 teachers from the College of Engineering, College of Management, and College of Design are involved in this interdisciplinary program. The interdisciplinary basic courses are taught by the teachers from these three colleges respectively. Since most of these courses belong to introductory or entry level courses, the teachers of each college mainly adopt the teaching method that combines more traditional theoretical lectures and practice training to cultivate students' cognition of other disciplines' values, viewpoints and methods. The students participating in these courses are not students of the disciplines to which the courses belong. The teaching of Capstone interdisciplinary integration courses is led and aided by an interdisciplinary teaching team composed of teachers from the three colleges in PBL teaching mode. All students participating in these interdisciplinary capstones are required to form interdisciplinary teams when taking these courses to complete corresponding subject training by solving real and complex problems.

#### *2.2. Measurements and Interviews*

Self-assessment comparisons of interdisciplinary integration ability indicators were made in the above-mentioned university between the students in higher grades participating in an interdisciplinary program and the students in higher grades involved in a single discipline study. Considering that there may be some subjective bias of student self-assessments, the research group planned to conduct sample interviews with the tested students to more objectively evaluate the difference in interdisciplinary integration ability of this group of students by collecting their coursework and referring to expert evaluations. However, due to some obstacles, such as the students' unwillingness and earlier departure of graduates caused by temporary school closures during the COVID-19 pandemic, the research group failed to collect relevant students' interview materials and interdisciplinary coursework. Therefore, members of the research group interviewed the teachers, tutors, and program administrators involved in interdisciplinary teaching in the university, to ask about their understanding of these students. It is intended to more objectively understand the performance of this group of students making self-assessments from the perspectives of the teachers. All the students participated in the self-assessments of interdisciplinary integration ability indicators voluntarily, and all the staff members invited to the interviews approved the acquisition and study of the interview recordings. The details are as follows:

#### 2.2.1. Interdisciplinary Integration Ability Measurements

Chen, Wang et al. synthesized previous research findings, summarized the connotations of interdisciplinary integration ability discussed in the first part of this paper and defined it as "common interdisciplinary integration-based core competencies", i.e., shared interdisciplinary integration ability that students of different disciplines should have. Based on this definition and combining the characteristics of the cultural background and native language habits of local students growing up and the general conditions of curriculum teaching, they developed an interdisciplinary integration ability scale (see Appendix A for details), which specifically includes three sub-ability measurements, i.e., "interdisciplinary communication, interdisciplinary reflection, and interdisciplinary practice", and a total of 16 questions (Chen et al. 2017). Among them, "interdisciplinary communication" is reflected in the three main indicators of "respecting professional opinions, understanding different professional terms, and communicating through communication tools", and includes six questions; "interdisciplinary reflection" is reflected in the three main indicators of "understanding the role differences among people with different expertise, making a reflection and generating new ideas through the process of interaction with others, and reflecting on the problems encountered in the process", and includes five questions; "interdisciplinary practice" is reflected in the three main indicators of "discovering teamwork problems and proposing practical solutions, evaluating the work efficiency of team members, and evaluating the effectiveness and making suggestions for improvement", and includes five questions. The 16 questions are answered based on a 5-point Likert scale, where 1 is strongly disagree, 2 is somewhat disagree, 3 is neutral/no opinion, 4 is somewhat agree, and 5 is strongly agree. This scale has been used in universities in Taiwan and has shown good reliability. In this case, considering the school's understanding of the cultivation of students' interdisciplinary integration ability, after discussing with some research experts, the research group adopted this scale to make measurement comparisons of "interdisciplinary communication, interdisciplinary reflection, and interdisciplinary practice" between the students participating in the interdisciplinary program and the students involved in a single discipline study in the university. In addition, in the pretests before using the scale, a total of 60 questionnaires were distributed, and 60 valid questionnaires were collected. The reliability value of Cronbach's α is .88, which indicates high reliability of the questionnaire design and thus it can be used for testing.

Samples of Students Participating in the Interdisciplinary Program

The students participating in the interdisciplinary program are undergraduate students in higher grades majoring in Industrial Design, Media Design, Materials Engineering, Mechanical Engineering, Electrical Engineering, Computer Science and Engineering, and Business Management from the above-mentioned university. When they entered the university in the first year, they attended the introduction meeting for the interdisciplinary program, and signed up to participate in the program. The research group conducted a random sampling of the students in higher grades who have completed the integrated Capstone courses, and collected 19 valid interdisciplinary student samples (excluding the student samples for the pretests), including three samples of Industrial Design majors, four samples of Media Design majors, five samples of Materials Engineering majors, three samples of Mechanical Engineering majors, one sample of Electrical Engineering major, and three samples of Computer Science and Engineering majors.

Samples of Students not Participating in the Interdisciplinary Program

The research group randomly distributed questionnaires to the students in higher grades in the university who have not participated in the interdisciplinary program. A total of 72 valid questionnaires were collected (excluding the questionnaires for the pretests), and among them, 23 are from Industrial Design majors, 29 are from Materials Engineering majors, and 20 are from Electrical Engineering majors.

For details of the independent variables, dependent variables and the number of students tested in the measurement comparisons, please refer to Table 3.


**Table 3.** Measurement Variables and Number of Students Tested.

In addition, it should be noted in this paper that, due to factors such as students' unwillingness, the research group were not able to collect any questionnaires from the students majoring in Business Management whether they participated in the interdisciplinary program or not.

#### 2.2.2. Interview

#### Interview Design

The purpose of these interviews is to verify whether the teachers' observations of the students participating in the interdisciplinary program are consistent with the students' self-assessments of their own interdisciplinary integration ability. In this way, the research group can more objectively understand the differences in learning outcomes and learning status between the interdisciplinary students and non-interdisciplinary students of various majors and discuss what factors may affect the improvement of student interdisciplinary integration ability based on these differences. In order to more comprehensively grasp the learning conditions of the students participating in the interdisciplinary program, the research group decided to conduct focus group interviews. When considering the focuses of these interviews, in order to avoid the situation that the respondents may be induced by the vocabulary with a specific connotation, the research group did not place related words mentioned above such as students' "learning methods, expectations, individual traits, prior experience, learning motivation, and maturity" in the questions when designing the discussion guide. An open-ended interview structure has been introduced to avoid any leading questions or guided answers. The clues are as follows:

Q1. What is the classroom atmosphere during class?

Q2. What is the performance of students in course learning and task completion? Are there any significant differences among them in this regard? Are the students of various majors the same in this regard?

Q3. When the courses are over, can the students achieve the ability target set when the courses were opened? If not, what is the reason? Are the students of various majors the same in this regard?

Interview Process

A total of 9 staff members (Pt-1 to Pt-9) from Industrial Design, Economics, Materials Engineering, Mechanical Engineering, and other disciplines were invited to the interviews. They are teachers of the interdisciplinary program, tutors of the interdisciplinary students, and program and teaching administrators. The respondents were asked to answer questions and have discussions according to the above clues based on their knowledge of the students in higher grades participating in the interdisciplinary program. These interviews were open-ended, and all the respondents fully expressed their opinions without any pressure or inducement. A total of 9 respondents have been interviewed, with the full process of interviews completed in 4 separated durations. The interviews were recorded only upon the agreement by the respondents. A total of 5 h, 49 min, and 13 s of audio recordings were produced.

Interview Analytical Methods

All the interview recordings were transcribed verbatim. Questions, explanations of questions, and follow-up questions in the verbatim transcript were removed, followed by a coding analysis of the verbatim transcript. The coding analysis was made within the theoretical framework of interdisciplinary integration ability, student attributes, and interdisciplinary learning environment discussed in the first part of this paper. After open coding, the textual material produced 185 units of thematic encoding distributed over 86 nodes. The same unit of thematic coding may belong to different nodes, but the same node can only be categorized into one subcategory and cannot be categorized into another subcategory. Therefore, through further summarization, mergence, and sorting, 18 subcategories were formed, and 8 categories were finally extracted. Due to the failure of sample collection of student coursework that can directly reflect students' interdisciplinary ability, we could only rely on teachers' judgments and subjective evaluations to understand students' interdisciplinary learning and to infer the improvement of their interdisciplinary integration ability. Therefore, the research group used the three dimensions of "learning condition feedback, student attributes, and learning environment" to carry out axial coding of the 8 categories, instead of "interdisciplinary integration ability, student attributes, and learning environment".

To ensure the high reliability of the initial open coding, the research assistants randomly selected 20% of the interview data for coding and compared it with the previous coding of the same part, finding no significant difference.

#### **3. Results**

#### *3.1. Self-Assessment Results of Student Core Interdisciplinary Integration Ability*

This study compares the core interdisciplinary integration ability among the students in the Departments of Industrial Design, Materials, and Electrical Engineering who have not participated in the university's interdisciplinary program and the students who have participated in the program. Questionnaires were distributed to the students in the Departments of Industrial Design, Electrical Engineering, and Materials, and interdisciplinary students, and a total of 91 valid questionnaires were collected. Among them, 23 are from Industrial Design students, 29 are from Materials students, 20 are from Electrical Engineering students, and 19 are from interdisciplinary students (including three from Industrial Design majors, four from Media Design majors, five from Materials Engineering majors, three from Mechanical Engineering majors, one from Electrical Engineering major, and three from Computer Science and Engineering majors). After sorting out the questionnaire

data of the four different groups of students, the quantitative statistical method of one-way ANOVA was used for analysis. Please refer to Table 4 for the results.



The quantitative analysis results of the students in the three sub-ability measurements of communication, reflection, and practice (see Table 4) show that a total of five questions indicate significant differences: Interdisciplinary Communication 2, Interdisciplinary Communication 5, Interdisciplinary Reflection 1, Interdisciplinary Reflection 3, and Interdisciplinary Practice 1. This means that each of the five questions can show significant differences in core interdisciplinary integration ability among the students of at least two or more disciplines. Post hoc multiple comparisons were required to explore specific situations. In addition, the results of the test of homogeneity of variances (see Table 5) show that Interdisciplinary Communication 3, Reflection 4, Practice 4 and Practice 5 all have the characteristics of heterogeneity of variance, which indicates the distribution of the samples of the above four questions, i.e., the degree of dispersion is very significantly heterogeneous. Therefore, in the case of multiple comparisons, these four questions needed to be tested by the Games–Howell method instead of the Scheffe method (Mamiseishvili et al. 2016). The results of the multiple comparisons indicate that although there are significant differences in the previous One-way ANOVA, Communication 2, Reflection 3, and Practice 1 do not show any significant differences in multiple comparisons, and finally only Interdisciplinary Communication 5, Interdisciplinary Reflection 1, and Interdisciplinary Practice 4 show significant differences (see Table 6 for details). The specific findings are as follows: the scores of Industrial Design students and Materials Engineering students in Communication 5 are significantly higher than those of Electrical Engineering students, and their means show the relation of "Industrial Design students > Materials students > interdisciplinary students > Electrical Engineering students", but the first three student groups do not show any significant difference, and there is no significant difference between interdisciplinary students and Electrical Engineering students. In Reflection 1, the means of the four student groups in core interdisciplinary integration ability show the relation of "Industrial Design students > interdisciplinary students > Materials students > Electrical Engineering students", in which only the scores of Industrial Design students are significantly higher than those of Electrical Engineering students. There is no other significant difference in Reflection 1, and in other words, interdisciplinary students and Materials Engineering students are not significantly different from Industrial Design students or Electrical Engineering students, and meanwhile, there is no significant difference between interdisciplinary students and Materials Engineering students either. Practice 4 reflects the same situation as Reflection 1: the means of the four student groups in core interdisciplinary integration ability show the relation of "Industrial Design students > interdisciplinary students > Materials students > Electrical Engineering students", and only the scores of Industrial Design students are significantly higher than those of Electrical Engineering students, and there is no significant difference among other student groups.


**Table 5.** Results of the Test of Homogeneity of Variance.

Students in the four groups show significant differences in only three out of the 16 questions of core interdisciplinary integration ability, and in these three questions, interdisciplinary students are not significantly different from any other student groups, which prevents us from drawing the conclusion that interdisciplinary students are significantly better than other students in core interdisciplinary integration ability. This result is surprising. Why the students participating in the interdisciplinary program do not have outstanding related ability deserves further discussions by the research group.

**Table 6.** Results of the Post Hoc Tests: Multiple Comparisons.


\* represents Sig. < .05, InterD = Interdisciplinary, EE = Electrical Engineering, ID = Industrial Design, ME = Mechanical Engineering.

#### *3.2. Qualitative Analysis Results of Teacher Interviews*

Table 7 shows how many times the codes covering three dimensions (D-01 to D-03) were mentioned and how many teachers mentioned them. Among them, "Learning Condition Feedback" (D-01) was mentioned 36 times by six teachers in total, and it includes two categories (D-01-01 to D-01-02) and five subcategories (D-01-01a to D-01-02c); "Interdisciplinary Learning Environment" was mentioned 41 times by seven teachers in total, and it includes three categories (D-02-01 to D-02-03) and six subcategories (D-02-01a to D-02-03b); "Student Attributes" was mentioned 108 times by eight teachers in total, and it includes three categories (D-03-01 to D-03-03) and seven subcategories (D-03-01a to D-03-03b).

In order to find shared feelings of the respondents, we only analyzed subcategories mentioned by two or more respondents. There are 15 subcategories (D-01-01a, D-01-02a to D-01-02c, D-02-01a to D-02-02b, D-03-01a to D-03-03b) and seven categories (D-01-01, D-01-02, D-02-01, D-02-02, D-03-01 to D-03-03) involved. The frequencies of the three main axis dimensions and seven categories are shown in Table 8. The specifics of each main axis dimension will be explained in order.


**Table 7.** Codes and Frequencies.


**Table 8.** Frequency Proportion and Ranking of Each Dimension and Category Mentioned.

#### 3.2.1. D-01 Learning Condition Feedback

This dimension includes the description and evaluation made by the teachers being interviewed on the behavioral performance of the interdisciplinary integration ability of the students participating in the interdisciplinary program. In the dimension, the teachers gave specific feedback on students' interdisciplinary values, interdisciplinary knowledge integration, interdisciplinary teamwork, interdisciplinary communication, and interdisciplinary team consensus building, as well as students' words, deeds, and emotions under the program. This dimension, including the two categories of positive feedback and negative feedback identified by their positive and negative connotations, was mentioned 36 times in total by six respondents successively.

In the category of "D-01-01 Positive Feedback", only the subcategory of "D-01-01a" was mentioned by two respondents. It mainly records the two teachers' recognition of the growth of some students in interdisciplinary learning, including overcoming the difficulties in interdisciplinary communication, reaching an interdisciplinary team consensus, and being willing to carry out interdisciplinary teamwork practice. They clearly stated that they have seen the students' growth in interdisciplinary ability (see Appendix B). We therefore named D-01-01a "Growth in Interdisciplinary Integration Ability". Unfortunately, positive feedback is the least among the seven categories in terms of the number of respondents who mention it and the frequency of mentions.

The category of "D-01-02 Negative Feedback" includes three subcategories: "D-01-02a, D-01-02b, and D-01-02c". Six out of the nine respondents talked about the problems and negative words, deeds and emotions of students in higher grades in interdisciplinary learning from different perspectives (see Appendix B). D-01-02a mainly reflects that the students still do not understand or agree with interdisciplinary values after participating in the interdisciplinary program; some students have interdisciplinary communication barriers and need to rely on their teachers to interpret and explain interdisciplinary knowledge. These all reflect that the students are still far from the acquisition of interdisciplinary integration ability. Thus, we named D-01-02a "Problems with Interdisciplinary Ability". D-01-02b was mentioned by two teachers, and it mainly describes students' doubts or distrusts during interdisciplinary learning. They not only distrust interdisciplinary learning, but also have

no confidence in teachers of other disciplines and in this program. We can understand that the students cannot accept the views and values of other disciplines, but we are very surprised to find that teachers of other disciplines and even the program itself cannot be trusted. As a result, we also listed this subcategory and named it "Student Distrust" instead of classifying it into the subcategory of "Problems with Interdisciplinary Ability". A total of three teachers mentioned the subcategory of D-01-02c, and they mainly talked about students' frustration, emotional ventilation, and withdrawal from the interdisciplinary program due to the intensity of the courses, the gap between expectations and perceived reality, and problems with teamwork. These are indeed negative phenomena that students experience during interdisciplinary learning, so we named this subcategory "Negative Emotions and Behaviors".

Based on the above analyses, negative feedback was mentioned 28 times by six respondents, which is significantly more than the positive feedback. Within the dimension of "Learning Condition Feedback", the frequency proportion of negative feedback accounts for 77.8%, which is more than three times that of positive feedback. From the analysis of this feedback, we believe that the overall improvement of students' interdisciplinary integration ability under this program is not satisfactory.

#### 3.2.2. D-02 Interdisciplinary Learning Environment

The dimension obtained after the axial coding was mentioned 41 times by seven respondents. Two categories of "D-02-01" and "D-02-02" were mentioned by more than two respondents. In this dimension, the respondents described students' feedback on the difficulty of interdisciplinary courses, the learning pressure imposed by the interdisciplinary teachers, the increased learning burden of interdisciplinary courses, and the influence of non-interdisciplinary teachers on the students participating in the interdisciplinary courses. Based on the previous discussion on the literature of "interdisciplinary learning environment", we coded this dimension as "Interdisciplinary Learning Environment".

The subcategories of D-02-01 include: D-02-01a and D-02-01b. D-02-01a was mentioned eight times by four respondents based on all the written materials of this study. The interviewed teachers described that the students reported that the basic design courses under this interdisciplinary program feature high-intensity learning, the engineering courses are too difficult to understand, so that they felt huge learning pressure, resulting in the negative emotions or behaviors mentioned above (see Appendix C for details). Hence, we named this sub-category "Study Pressure". D-02-01b was mentioned 19 times by six respondents. The interdisciplinary teachers found that after a period of interdisciplinary learning, some students think that they have consumed too much time or energy in interdisciplinary learning instead of in their own discipline; and they worried that the final scores of the interdisciplinary courses will lower their average score, etc., all of which make students perceive interdisciplinary learning as a burden to their own discipline study (see Appendix C). Based on this, we named the subcategory "Academic Burden", and the category of D-02-01 including the two subcategories of "D-02-01a Study Pressure and D-02-01b Academic Burden" "Pressure and Burden".

The subcategories of D-02-02 include: D-02-02a and D-02-02b. D-02-02a was mentioned 10 times by three teachers. The respondents mainly mentioned the influence of teachers of mono-disciplines on students' disciplinary thinking, which mostly features contempt or rejection of interdisciplinary values (see Appendix C). This seems to challenge students' interdisciplinary values, and indeed affects students' cognition, judgment and persistence in interdisciplinary learning to a considerable extent. Therefore, this subcategory was named "Influence of Departments". D-02-02b was mentioned five times by five respondents. In this subcategory, the respondents mainly mentioned that when the students are switching between interdisciplinary courses and disciplinary courses under this program, the teachers of different disciplines have different evaluation criteria for the output of the same student, which can bring frustration and value conflicts to the students participating in the interdisciplinary program. Some students cannot adapt to, identify

with, or accept different judging standards (see Appendix C). Therefore, D-02-02b was named "Differences among Disciplines".

Among the three main axis dimensions, interdisciplinary learning environment has the second highest number of respondents who mentioned it and the frequency of mentions. Two categories of D02-01 and D02-02 were both mentioned by six respondents. The former has a frequency of 19, and the frequency proportion within this dimension accounts for 46.3%; the latter has a frequency of 15, and the frequency proportion within this dimension accounts for 36.6%. In this dimension, the frequency difference between the two categories is about 10%. Based on this, we believe that the impact of interdisciplinary students' learning pressure and burden may bring the respondents a stronger feeling than that of differences between disciplines. However, in any case, the interviews reveal such a finding, i.e., the intensity, pressure, and burden felt by the students in interdisciplinary learning, the unsupportive teachers of their own disciplines against interdisciplinary learning, and differences among disciplines are closely related to "D01-02 Negative Feedback" mentioned in the previous axis dimension.

#### 3.2.3. D-03 Student Attributes

This dimension records the judgments made by the interviewed teachers on the students' learning motivation, and the speculation and description of the motivational causes after observing the students' interdisciplinary learning status. It also includes a description of the impact of students' experiences before being involved in the interdisciplinary program on their interdisciplinary learning, and the impact of students' individual traits, such as personal characteristics, learning habits, and learning responsibility on their interdisciplinary learning. According to the discussion of "student attributes" in the first part of this paper, we named this dimension "Student Attributes". The number of respondents who mentioned the dimension is the most and the frequency of mentions is the highest among the three dimensions. The topic of student attributes was mentioned 108 times in total by eight respondents successively. This dimension includes three categories: "D03-01, D03-02, and D03-03".

In this dimension, the respondents talked about the phenomena of students lacking motivation, dawdling their time away, and being unwilling to participate in interdisciplinary learning, and believed that students lack interdisciplinary learning motivation. Meanwhile, the teachers made some interpretations and analyses of the reasons for the lack of motivation of the students (see Appendix D). Therefore, we named D03-01 "Motivation". The number of respondents who mentioned D03-01 Motivation is the most and the frequency of mentions is the highest among the three categories. It was mentioned 56 times by eight respondents. The category of D03-01 Motivation includes three subcategories: "D03-01a, D03-01b, and D03-01c". Self-determination theory suggests that pure curiosity or a desire to master can be called intrinsic motivation; all the other behaviors are driven by extrinsic motivation, derived from the integration and internalization of social values or rules (Cook and Artino 2016). Based on this, we classified the phenomena of the students' lacking motivation, dawdling their time away, and being unwilling to participate in interdisciplinary learning, as well as other related phenomena of a lack of motivation into the subcategory of D03-01a, and named it "Intrinsic Motivation". D03-01a was mentioned 38 times by eight respondents successively. In addition, two respondents mentioned the incentives or restraints that are intended to stimulate students' extrinsic motivations and suggested that these extrinsic motivations can be transformed into students' intrinsic motivations. This was encoded into D03-01b and named "Extrinsic Motivation". The reasons for the lack of motivation mentioned by the teachers, including students' identification with the teachers, whether interdisciplinary learning can meet their short-term realistic goals, and interdisciplinary learning's relevance to their own disciplines were encoded into D03-01c and named "Source of Motivation". The subcategory of D03-01c was mentioned by six respondents.

In the category of D03-02, the respondents talked about the students' incompatibility with the teaching methods not belonging to their own discipline, and their unaccustomedness and irritation of PBL teaching when taking Capstone courses due to their lack of prior interdisciplinary learning experience. In addition, the students do not have similar experiences in social cognition and life practice before participating in interdisciplinary learning, especially before taking the interdisciplinary integration courses in the upper grades. In fact, they showed confusion about interdisciplinary cognition both before and after participating in the program (see Appendix D). The research team named this coding category "Prior Experience". Meanwhile, the text content about interdisciplinary learning incompatibility due to the lack of prior learning experience was coded as the subcategory of "D-03-02a" and named "Influence of Prior Teaching and Learning Styles"; the relevant text content about the lack of interdisciplinary cognition in previous social cognition and life practice was coded as the subcategory of "D-03-02b", and named "Prior Interdisciplinary Practice Experience and Cognition". The number of respondents who mentioned D03-02 Prior Experience is the second most and the frequency of mentions is the second highest among the three categories. It was mentioned 29 times by seven respondents. D-03-02a was mentioned 21 times by six respondents. D-03-02b was mentioned 8 times by three respondents. The students from the Department of Design and the students from the Department of Engineering were compared, and it can be seen that different prior teaching styles of the two disciplines lead to different adaption conditions of the students after participating in the interdisciplinary program. For example, design students already have problem-oriented learning experience when they study in their own disciplines. Meanwhile, due to the nature of design disciplines, design students have more opportunities to be exposed to some interdisciplinary knowledge. As a result, design students are more adaptable in interdisciplinary learning, while students in other disciplines are the opposite. The teachers believed that the students' lack of interdisciplinary experience and cognition before participating in the interdisciplinary program influences the formation of their interdisciplinary values or awareness, which may be one of the factors that cause the students to be at a loss or even withdraw from the program when facing interdisciplinary learning.

In the category of D03-03, the respondents reflected different characteristics of students in different departments in interdisciplinary learning; according to the content of the relevant text, the research group coded it as "D-03-03a" and named it "Different Characteristics of Students in Different Departments". Besides, we coded the content about the impact of student personal characteristics like sense of responsibility, learning attitudes on interdisciplinary learning as D-03-03b, and named it "Student Personal Characteristics". For the category of D03-03 that includes D-03-03a and D-03-03b, we named it "Individual Traits" (see Appendix D). Although the number of respondents who mentioned D03-03 and the frequency of mentions are the least among the three categories of the dimension of Student Attributes, it was mentioned 23 times by five respondents. D-03-03a was mentioned 10 times by three respondents. D-03-03b was mentioned 13 times by five respondents. From the encoded text, some interdisciplinary learning conditions of Engineering, Design, and Business students can be seen, and it is found that different student characteristics shaped by different discipline education also seem to have an impact on interdisciplinary learning. For example, Engineering students are relatively not good at communication, while Design students are more creative, and Business students are considered to be more inclined to take shortcuts in interdisciplinary learning. From the text of the encoded unit, it can be seen that the teachers felt that the students' own personal characteristics, learning attitudes, and sense of responsibility can also have an impact on interdisciplinary learning (see Appendix D).

As mentioned above, the number of respondents who mentioned the dimension of Student Attributes is the most and the frequency of mentions is the highest among the three main axis dimensions. The frequencies of the three categories of "D03-01 Motivation, D03-02 Prior Experience, and D03-03 Individual Traits" are respectively 51.9%, 26.9%, and 21.3%. Among all the categories, the frequency proportion of Motivation accounts for

30.3%, the frequency proportion of Prior Experience accounts for 15.7%, and the frequency proportion of Individual Traits accounts for 12.4%. According to the interview transcript, the space of the dimension of Student Attributes is the greatest. It can be said that student attributes should have a very important relation with interdisciplinary learning outcomes, and they can play a key role in student interdisciplinary integration ability.

#### **4. Discussion**

The purpose of this study is to empirically explore whether there is significant difference in interdisciplinary integration ability between the undergraduate students participating in the interdisciplinary program that integrates design, engineering, and business, and the students studying a single discipline, and to discuss the reasons for the differences. To this end, the research group invited 91 students for self-assessment analyses of core interdisciplinary integration ability and nine teachers and related staff members involved in the interdisciplinary program to interviews on the conditions of the group of interdisciplinary students. The experimental data were obtained through quantitative comparative analyses and qualitative coding analyses. The results of quantitative analyses show that the students participating in the interdisciplinary program are not significantly different from those of other disciplines in the ability level of "interdisciplinary communication, interdisciplinary reflection, and interdisciplinary practice". The results of qualitative analyses show that the teachers' negative feedback on the interdisciplinary students is significantly more than positive feedback in the number of respondents who mentioned it and frequency of mentions. Meanwhile, through qualitative analyses, it is found that the interdisciplinary students' disagreement with interdisciplinary values, distrust of interdisciplinary teachers, obstacles in interdisciplinary communication, and problems with teamwork. Therefore, the research team believes that the improvement of the students in ability after participating in interdisciplinary learning is not ideal. This clearly echo their insignificant interdisciplinary integration ability in the interdisciplinary integration ability measurements. Based on the results of the data analyses, we believe that the results of the qualitative analyses can confirm the objectivity of the students' self-assessment results to a considerable extent. Based on this finding, we are more inclined to assert that there is no necessarily significant difference in interdisciplinary integration ability between the students participating in the interdisciplinary program and the students studying a single discipline. This may not support the research conclusion that student interdisciplinary integration ability is closely related to interdisciplinary course participation, or that students involved in interdisciplinary learning have better interdisciplinary integration ability than students studying a single discipline (Li and Lin 2018; Newell 1992; Wright 1992). However, this finding is similar to the findings of the 2017 study by Lattuca et al. Their research shows that students of interdisciplinary learning are not necessarily better than students of monodisciplinary learning in interdisciplinary-related abilities, and students' acquisition of the ability may not necessarily change significantly due to the interdisciplinary characteristics (Lattuca et al. 2017b). Lattuca et al. also pointed out that their findings are consistent with evidence from Jacobs' analysis of Arum et al.'s data (Lattuca et al. 2017b).

What causes this insignificant difference? Soares believed that curriculum designers often seem to underestimate the support that students need in interdisciplinary learning (Soares et al. 2013). Borrggo et al. suggested that the design of course projects should avoid as much as possible the frustration of students due to overly difficult problem tasks (Borrego et al. 2013). In fact, the research of Soares, Borrego et al. confirmed the systematic relation between interdisciplinary teaching and learning summarized by Spelt et al. through literature review, i.e., the impact of student attributes and interdisciplinary learning environment on interdisciplinary integration ability (Spelt et al. 2009). This does echo our findings. The results of the quantitative analyses and the feedback of learning conditions in the qualitative analyses should reflect the level of student interdisciplinary integration ability. In the qualitative analyses of these interviews, student attributes, including motivation, prior experience, and individual traits, and the interdisciplinary

learning environment, including pressure and burden, and disciplinary factors, should be the influencing factors of student interdisciplinary integration ability. Meanwhile, from the analyses of the frequency of mentions in the coding analysis research, student attributes' impact on interdisciplinary learning is significantly more than the interdisciplinary learning environment; and for each category, their influence from more to less is respectively: Motivation, Prior Experience, Individual Traits, Pressure and Burden, and Disciplinary Factors. We will further explore these five categories further below.

Motivation

In our interviews, the teachers pointed out that the students are unwilling to participate in the interdisciplinary program and dawdle their time away during interdisciplinary learning due to their lack of identification with interdisciplinary learning; students may join the program for some other reasons, so they are not very active; students think that they have spent time on interdisciplinary learning, but it does not help achieve their short-term goals, so they naturally withdraw from the program; students feel that they have spent energy on interdisciplinary learning, but the results are not satisfactory, and they are worried that their average score will be lowered, so they have negative reactions. Clearly, these are manifestations of a lack of motivation in interdisciplinary learning (see Appendix D for details). Motivation is defined as the process of initiating and maintaining goal-directed activities, while the goal-directed theory states that learners tend to engage in tasks related to mastering content or to do better than others or to avoid failure (Cook and Artino 2016). In addition, as an important part of motivational structure, self-efficacy (Lishinski et al. 2016) determines how much effort people are willing to put in, as well as people's ability to cope and persevere in the face of challenges and difficulties (Bandura 1977). The discussion of motivation by Cook, Lishinski, and Bandura et al. should be sufficient to explain the impact of students' lack of motivation for interdisciplinary learning. Therefore, whether from the frequency results of the qualitative analyses or from previous research on learning motivation, perhaps the primary task of interdisciplinary education should be the cultivation, shaping, and enhancement of learning motivation.

How can we shape or enhance student motivation for interdisciplinary learning? In the interviews, some teachers mentioned that students may need to know what kind of ability the interdisciplinary program is designed to cultivate, or what they may get after completing the program, which may be important motivation to support them to continue their studies. In this regard, some researchers pointed out that understanding the utility and importance of interdisciplinary learning is very important for student interdisciplinary learning outcomes (Chen et al. 2009; Matthews et al. 2010). In addition, Keller pointed out that establishing students' motivation to learn requires successfully establishing the relevance of teaching to students as an individual (Keller 1987). In fact, the respondents reported that they have conveyed under the interdisciplinary program to the students the idea that interdisciplinary learning is more conducive to acquiring the ability and vision of innovation and entrepreneurship, but this does not seem to be related to students' more realistic short-term goals of furthering their study, going abroad, finding a job, or improving their average score of their own discipline, so they fail to convince these students to realize the importance of interdisciplinarity. Obviously, this relatively superficial interdisciplinary concept transfer has not successfully established the relevance of teaching to students. This may be one of the reasons for not effectively stimulating students' motivation for interdisciplinary learning. This is similar to the findings of Self et al.'s 2019 study, i.e., compared with British students, Korean students cannot be identified with the interdisciplinary nature of a particular occupation, and they are particularly concerned about the appropriateness of interdisciplinary education in terms of employment, its negative impact on employment, and are worried about whether interdisciplinary education will be valued by discipline-oriented industries. Self et al. believed that different regional cultures may influence students' driving force of interdisciplinary learning (Self et al. 2019). Based on this, we infer that the students in East Asia may be more concerned about the relevance of interdisciplinary learning and the realization of short-term goals. In interdisciplinary

education, the shaping or enhancement of student learning motivation should focus on this. In addition, judging from the introduction to the course teaching mentioned in the second part of this paper, for the students from different disciplines, the interdisciplinary basic courses under this program still use the original traditional teaching methods of each college. We speculate that this is bound to make it difficult for the students to establish the relevance of their own disciplines and interdisciplinary course teaching. Meanwhile, the original teaching methods of various disciplines retained in the teaching of interdisciplinary basic introductory courses have turned the teaching of interdisciplinary basic courses into multi-disciplinary teaching of disciplines plus disciplines, and fail to promote the integration of knowledge, methods, and viewpoints of various disciplines, which may make it difficult for the students from different disciplines to have effective interdisciplinary communication and interdisciplinary teamwork under this program. As Keller once pointed out, students' effective learning and expectations of success are hindered, which will also lead to a decrease in learning motivation (Keller 1987). Therefore, although this program has interdisciplinary integration courses in the later stage, it still uses the traditional teaching methods in the interdisciplinary basic introductory courses in the early stage, which should also be the reasons that lead to the lack of students' motivation and the hinderance of the improvement of students' interdisciplinary integration ability.

Prior Experience

The courses students have taken can significantly influence their learning experience (Chen et al. 2009). The experience may affect student interdisciplinary learning. Spelt et al. pointed out that past social and educational experience, such as students' previous thinking styles, the teaching styles they have been exposed to, and beliefs about the nature of knowledge and learning, may impact their interdisciplinary integration ability and thinking (Spelt et al. 2009). In our interviews, some teachers mentioned that students are not used to the teaching methods of the interdisciplinary teachers; if the students do not start to get used to the teaching methods in their freshman year, it will be hard for them to be adapted to them in their junior year; the engineering students cannot adapt to problem-oriented learning in basic design courses, and cannot understand teaching methods that do not have the best solution to problems in integrated courses; the students have no successful experience in innovation and entrepreneurship, so it will be difficult for them to understand and identify with the teachers' perspectives on interdisciplinary learning(see Appendix D for details). In contrast, design students, as mentioned above, have more opportunities to be exposed to interdisciplinary knowledge, have earlier problem-oriented learning experiences, and are more adaptable to interdisciplinary learning. Meanwhile, judging from the quantitative results of students' ability, non-interdisciplinary design students have significant performance in Communication 5, Reflection 1, and Practice 4 in the questions can also explain this. The information gathered supports Spelt et al.'s perspective. Based on this, the authors infer that students may experience discomfort or confusion in new learning due to differences in their previous study habits or teachers' teaching styles. Meanwhile, the lack of specific interdisciplinary experience will lead to students' failure in interdisciplinary value formation, which is not conducive to the construction of interdisciplinary learning motivation and the improvement of interdisciplinary integration ability level. This is consistent with Ramalingam et al.'s point that student self-efficacy and academic performance are positively related to their prior experience (Ramalingam et al. 2004), i.e., the amount of prior experience affects the amount of student's interdisciplinary learning motivation and how much the learning effectiveness will be improved. On the other hand, as far as learning is concerned, researchers in the field of cognitive theory believed that how new information is organized and interrelated with previous knowledge has an important impact on learning, and interdisciplinary teachers should help students create a clear link between what they are going to learn and their prior experience, including what they have learned in the past (Lattuca et al. 2004). We obtained similar confirmation from the discussion of motivation in the previous part of this paper. In practice, however, it is not an easy task for interdisciplinary teachers to correlate students' experience before and after interdisciplinary

learning. It can be seen from the interviews that it is especially difficult for interdisciplinary teaching practitioners to understand and organize students' prior non-educational experience. Therefore, interdisciplinary education should be regarded as a long-term process, and students should be exposed to interdisciplinary learning earlier to have interdisciplinary experience, which may gradually build students' interdisciplinary cognition, establish their interdisciplinary values, facilitate the growth of interdisciplinary learning motivation, and promote the improvement of interdisciplinary integration ability. In this regard, Wilhelmsson et al. have the same understanding: the acquisition of interdisciplinary integration ability is a process that must start early in education (Wilhelmsson et al. 2009).

Individual Traits

The results of the interview analyses of this category show that the students majoring in Engineering, Management, and Design have different focuses and ways of dealing with problems in interdisciplinary learning, and the impact of their learning attitudes on learning outcomes. Several teachers pointed out that in interdisciplinary learning, Engineering students seem to be more conservative, so they think in a less creative way; Design students are original and have many ideas, but their consideration of practical application may be incomprehensive; Business students intend to save effort in their learning, and they often avoid wasting energy, time, and other learning risks. In this regard, some teachers pointed out that it is easier for the students to believe the value that is easier for them to understand or is more similar to their own major. On the other hand, the teachers believed that a serious attitude, a sense of responsibility and self-discipline reflected in students' individual traits are still important factors for positive outcomes in interdisciplinary learning. Especially students who are willing to use what they have learned to analyze and organize can be a high achiever in the end (see Appendix D for details). It may be inferred that the values or learning styles of different disciplines affect students' perspectives on problems and the learning strategies and actions they take. Meanwhile, students' individual traits also seem to affect their own learning strategies, and thus affect the final interdisciplinary learning outcomes. Some researchers believed that disciplines affect the learning methods students adopt over time (Tarabashkina and Lietz 2011). The study by Bruce et al. found that for successful interdisciplinary learning, personalities and attitudes should be at least as important as disciplinary foundations and specialization. They believed that an excellent interdisciplinary person should have a high tolerance for ambiguity, and they should not prematurely narrow a problem to a limited set of dimensions, but instead, they should spend time exploring a range of dimensions and testing several potential boundaries; therefore, they also believed that an ideal interdisciplinary person should have curiosity about and willingness to learn other disciplines, and be open to the ideas and experience from other disciplines, etc. (Bruce et al. 2004). In this regard, Woods also believed that curiosity and openness represent a willingness to suspend doubts about other disciplinary cultures and suspend a hold on beliefs in their own disciplinary culture (Woods 2007). Tik believed that openness refers to the characteristics of students who are curious and intelligent. They are open to new experiences and willing to adopt other learning strategies; while responsibility refers to the characteristics of achievement, organization and perseverance, and students with these traits tend to be more inclined to use higher-order cognitive skills, such as critical thinking and metacognition (Tik 2020). Therefore, both students' own individual traits and their characteristics caused by discipline attributes should have an impact on their interdisciplinary learning outcomes. Meanwhile, as shown in Table 7, the influence of students' individual traits is greater than students' characteristics caused by discipline attributes based on the number of respondents who mentioned them and the frequencies of mentions. Together with the previous research on student learning attitudes, this may show that the influence of student individual traits is slightly more important than the influence of student discipline characteristics on interdisciplinary learning outcomes.

Pressure and Burden

The interview materials of this category reflect that the intensity of learning and the increased strictness of teachers' demands for task completion appear to lead to negative effects on student interdisciplinary learning. For example, the teachers pointed out that the students reported that the courses in the department of Design bring a heavy course load, and the courses given by the teachers of the department of Engineering are too indepth, so that the students feel a heavy burden; in the later integrated courses, the students cannot accept the course output requirements and strictness of the teachers. In addition, students' inadaptation of interdisciplinary learning also causes them to worry that their academic GPA will be lowered, so they think interdisciplinary learning is a burden for them, and eventually many students withdraw from the program (see Appendix C for details). This finding may be supported by Matthews et al. They embed programming teaching content in the study of first-year Biology undergraduates and required students to apply their programming skills in a quantitative real-world setting. However, it was too complex for the students to respond effectively, so the students' feedback on this were negative to a large extent (Matthews et al. 2010). Moreover, Chen et al. have similar findings. They pointed out that a heavy study load increases the difficulty of students participating in various courses outside their own discipline and reduces their attention paid to interdisciplinary learning. Meanwhile, this may be a reason for the declining trend of students' interest and value in interdisciplinary learning (Chen et al. 2009). Indeed, judging from the total credits of undergraduate majors in the three colleges of the school, each major has 150 credits, and participating in this interdisciplinary program will add 35 credits, which is equivalent to adding more than four credits per semester and 70 credit hours of lessons. In fact, the number of all the courses is not evenly distributed in each semester. If the interdisciplinary teachers have higher requirements on coursework and put more pressure on their students, especially in certain semesters with more class hours, students will definitely feel the weight of a heavier study load, and as a result, they will naturally choose to give up interdisciplinary learning to ensure their own disciplinary learning. Therefore, when designing interdisciplinary curriculum content and student output requirements, teachers should comprehensively consider the pressure and burden brought to students by the learning load of both interdisciplinary courses and the courses of their own discipline. This requires more adequate and effective communication and coordination between interdisciplinary teachers and teachers of different disciplines, in order to bring positive effects on student learning outcomes.

Disciplinary Factors

Teachers' disciplinary views and biases can influence how students learn and experience in interdisciplinary learning (Self et al. 2019). Self et al. found that some teachers' own disciplinary biases can be transformed into their expectations for students, which leads students to change their learning methods and learning outcomes in their studies to meet the expectations of disciplinary teachers. Our research findings support this view. The interview participants indicated that disciplinary teachers are accustomed to using their values to influence students. They lack support for the students participating in interdisciplinary courses. For example, non-interdisciplinary teachers show their inhibition or contempt for interdisciplinarity or the interdisciplinary program before the students being involved in interdisciplinary leaning when teaching their own disciplinary courses, so that the students have distrust of interdisciplinary courses and teachers. In fact, this attitude of disciplinary teachers towards interdisciplinary learning should be relatively common. First, teachers who lack interdisciplinary experience may also lack enthusiasm or willingness to develop interdisciplinary projects (Gardner et al. 2014; Van den Beemt et al. 2020). Second, the academic community and higher education community generally regard disciplines as cornerstones, so they tend to marginalize more comprehensive areas of knowledge or educational programs (Palaiologou 2010). Brew suggested in his research that many scholars tend to overemphasize the importance of disciplinary affiliation (Brew 2008). In this regard, Lindvig et al. believed that interdisciplinary teaching, which is different from the accustomed way of disciplinary teaching, may be regarded as a

threat to hinder the construction of the disciplinary identity, so this should be one of the difficulties that interdisciplinary teaching is facing (Lindvig and Ulriksen 2019). Obviously, the influence of teachers' words and deeds based on disciplinary thinking and values brings challenges to students in interdisciplinary learning. In the operation and management of interdisciplinary programs, schools need to establish common interdisciplinary educational values among teachers of various disciplines to avoid negative impact on interdisciplinary teaching by disciplinary teachers who are not involved in interdisciplinary teaching. In addition, some respondents pointed out that Design teachers and Engineering teachers have different evaluation criteria for student outputs, which has led to students' frustration in interdisciplinary learning. For example, Engineering students' award-winning works in disciplinary competitions cannot be recognized by Design teachers (see Appendix C for details). This is consistent with the findings of Self et al., and in their study, professors of Industrial Design rarely collaborate with professors of Ergonomics, and the differences between these two disciplines have an impact on course learning outcomes. What is considered important by everyone is not considered important in Ergonomics (Self et al. 2019). If such disparities between disciplines are not balanced and integrated to form judging criteria based on a shared value, challenges will be created for interdisciplinary learning and teaching.

#### **5. Conclusions**

This study uses the Core Interdisciplinary Integration Ability Scale developed by Chen et al. to measure the interdisciplinary integration ability of the students participating in the interdisciplinary program that integrates design, engineering, business and other disciplines. Under the theoretical framework of Biggs, Spelt, and Liu et al. on interdisciplinary learning outcomes, student attributes, and interdisciplinary learning environment, a qualitative analysis of interviews with interdisciplinary teachers and related personnel is conducted. The research group found that there is no significant difference in interdisciplinary integration ability between the students participating in the interdisciplinary program and the students involved in a single discipline study, including the Industrial Design, Electrical Engineering, and Materials Engineering students. Based on the qualitative analysis results of the interview data, the authors believe that the reasons why there is no significant difference may be problems with student attributes, including the lack of motivation, lack of prior interdisciplinary experience, influence of individual traits, and problems with interdisciplinary learning environment, including the increased learning pressure and burden, and interference of disciplinary factors. Our findings can provide some references for the future development and design of interdisciplinary programs and interdisciplinary teaching. Especially for the establishment and shaping of interdisciplinary learning motivation, for students in East Asia, attention should be paid to the substantial connection between students' short-term goals and interdisciplinary learning, as well as to the construction of the correlation between students' own disciplines and interdisciplinary learning content; meanwhile, for the interdisciplinary basic course teaching in the early stage of interdisciplinary programs, we should take into account the fact that the students under these programs are from different disciplines, and carry out teaching from the perspective of knowledge integration, so as to avoid using original discipline teaching methods to simply make interdisciplinary teaching into multi-disciplinary teaching. Besides, it may be beneficial to start students' experience in interdisciplinary learning or research at an earlier stage to gradually form students' interdisciplinary cognitions and values. In addition, when establishing a teaching design for students, attention should be paid to their individual traits and there should be sufficient communication and coordination with students' disciplinary teachers to achieve a balance between interdisciplinary and disciplinary learning, form a commonly recognized evaluation standard, and try to avoid negative effects on the learning outcomes of interdisciplinary students due to the increase of students' learning pressure and academic burden or the interference of disciplinary factors.

Due to the different systems and structures of interdisciplinary programs among universities, this study did not collect data from other universities for comparison. Besides, because of some students' unwillingness and the impact of the COVID-19 pandemic, the research group did not collect any samples of students majoring in Business Management and all tested students' opinions on the directness of the interdisciplinary courses in this university. With the graduation of this group of students, the collection of relevant samples has become unlikely. The lack of such sample data makes it difficult for us to truly and directly understand the psychological state and opinions of the students participating in this interdisciplinary program. Only relying on the teachers' observation, description and evaluation of the students may miss the details of some students' conditions, resulting in some problems not being discovered in time. This indeed brings about some limitations to this research. Fortunately, the measurement results of students' interdisciplinary ability and the analysis results of teacher interviews can confirm each other, so this research group believes that our experimental data are convincing. Our findings further confirm the previous view that students participating in interdisciplinary learning may not necessarily improve their interdisciplinary ability. Meanwhile, on this basis, according to the empirical results, this study points out the specific factors that bring interdisciplinary learning challenges to students. This provides inspiration for subsequent related research. In addition, this study only discusses the impact of student attributes and interdisciplinary learning environment on learning outcomes, but from the theoretical model of Biggs et al., learning outcomes can also affect student attributes and learning environment. This will open the way for our future research considering, e.g., how the improvement of students' interdisciplinary ability will stimulate students' interdisciplinary motivation.

**Author Contributions:** Data curation, D.-D.X.; formal analysis, K.-Y.W.; investigation, C.X., D.-D.X., W.-Q.L. and K.-Y.W.; methodology, C.X.; project administration, C.-F.W.; supervision, C.-F.W.; writing—original draft, C.X.; writing—review & editing, C.X. All authors have read and agreed to the published version of the manuscript.

**Funding:** The study was financially supported by MOST 110-2410-H-036-005-.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Research Ethics Committee of National Taiwan University (protocol code 202105ES161).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The data are not publicly available due to The relevant data involves the internal information of the institution and personal information, and there is a need for confidentiality.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

**Table A1.** Core Interdisciplinary Integration Ability Scale Questions.



Compiled and translated from (Chen et al. 2017).

#### **Appendix B**

**Table A2.** Coding of D-01 Learning Condition Feedback.



**Table A2.** *Cont*.

Under the principle of not changing the meaning of the respondents' conversations, in order to show the content of the conversations more accurately, the authors annotate what has been omitted or referred to in the conversations through ( ) according to the context of the interviews.

#### **Appendix C**


**Table A3.** Coding of D-02 interdisciplinary Learning Environment.

Under the principle of not changing the meaning of the respondents' conversations, in order to show the content of the conversations more accurately, the authors annotate what has been omitted or referred to in the conversations through ( ) according to the context of the interviews.

#### **Appendix D**


**Table A4.** Coding of D-03 Student Attributes.


**Table A4.** *Cont*.


**Table A4.** *Cont*.


Under the principle of not changing the meaning of the respondents' conversations, in order to show the content of the conversations more accurately, the authors annotate what has been omitted or referred to in the conversations through ( ) according to the context of the interviews.

#### **References**


## *Article* **Creativity and Artificial Intelligence—A Student Perspective**

**Rebecca Marrone 1,\*, Victoria Taddeo <sup>1</sup> and Gillian Hill <sup>2</sup>**


**Abstract:** Creativity is a core 21st-century skill taught globally in education systems. As Artificial Intelligence (AI) is being implemented in classrooms worldwide, a key question is proposed: how do students perceive AI and creativity? Twelve focus groups and eight one-on-one interviews were conducted with secondary school-aged students after they received training in both creativity and AI over eight weeks. An analysis of the interviews highlights that the students view the relationship between AI and creativity as four key concepts: social, affective, technological and learning factors. The students with a higher self-reported understanding of AI reported more positive thoughts about integrating AI into their classrooms. The students with a low understanding of AI tended to be fearful of AI. Most of the students indicated a thorough understanding of creativity and reported that AI could never match human creativity. The implications of the results are presented, along with recommendations for the future, to ensure AI can be effectively integrated into classrooms.

**Keywords:** creativity; artificial intelligence; student attitudes

#### **1. Introduction**

There is a strong consensus that creativity is a crucial 21st-century competency. Education systems report the importance of creativity (Patston et al. 2021). Similarly, Artificial Intelligence (AI) is significantly impacting a growing number of fields, including education (Gabriel et al. 2022). Globally, education systems are developing strategic plans to embed AI in classrooms adequately (see Singapore, Estonia, Australia, New Zealand, and Scotland, to name a few) (Gabriel et al. 2022). Whilst the importance of both creativity and AI are well established, less is known about how students perceive and value the relationship between AI and creativity. This paper will explore how students perceive AI and creativity, and endeavour to ensure that education systems support the development of both competencies.

#### *1.1. Artificial Intelligence in Education*

Artificial Intelligence (AI) is a branch of computer science that uses algorithms and machine learning techniques to replicate or simulate human intelligence (Helm et al. 2020). There are three types of AI: narrow AI, general AI, and Artificial Superintelligence. Narrow AI is the most common and realized form of AI to date. It is very goal-orientated and uses machine learning techniques to achieve one specific goal or task (e.g., image and facial recognition, Siri/Alexa). General (or deep) AI is AI that is deemed on par with human capabilities (e.g., AI that can discern the needs and emotions of other intelligent beings). Thirdly, Artificial Superintelligence is AI that is more capable than humans (similar to a sci-fi movie portrayal of AI that supersedes humans in every regard) (Hassani et al. 2020).

Within the education context, artificial intelligence development will likely remain in the form of narrow AI. Current educational technologies include speech semantic recognition, image recognition, Augmented Reality/Virtual Reality, machine learning, brain

**Citation:** Marrone, Rebecca, Victoria Taddeo, and Gillian Hill. 2022. Creativity and Artificial Intelligence—A Student Perspective. *Journal of Intelligence* 10: 65. https://doi.org/10.3390/ jintelligence10030065

Received: 5 August 2022 Accepted: 3 September 2022 Published: 6 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

neuroscience, quantum computing, blockchain, et cetera. These technologies are rapidly being integrated within classrooms. An ever-increasing number of artificial intelligence education products are being applied to K-12 education (Yufeia et al. 2020). Literature studies show that artificial intelligence technology in education has been used in at least 10 aspects: "the (i) automatic grading system, (ii) interval reminder, (iii) teacher's feedback, (iv) virtual teachers, (v) personalized learning, (vi) adaptive learning, (vii) augmented reality/virtual reality, (viii) accurate reading, (ix) intelligent campus, and (x) distance learning" (Yufeia et al. 2020, p. 550).

The Artificial Intelligence in Education (AIED) community emphasises the creation of systems that are as effective as one-on-one human tutoring (VanLehn 2011). Over the last 25 years, there have been significant advances toward achieving that goal. However, by enforcing the human tutor/teacher as the gold standard, a typical example of AIED practices has often included a student working with a computer to solve step-based problems focused on domain-level knowledge in subjects such as science and mathematics (Trilling and Fadel 2009). However, this example does not consider the recent developments in education practices and theories, including introducing 21st-century competencies. The 21st-century competency approach to education emphasises the value of general skills and competencies such as creativity. Today's classrooms strive to incorporate authentic practices using real-world problems in collaborative learning settings. To maintain its relevance and increase its impact, the field of AIED has to adapt to these changes.

#### *1.2. What Does Creativity in an AI Classroom Look Like?*

Boden (1998), in her paper, suggests that AI techniques can be used to enhance creativity in three ways: 'by producing novel combinations of familiar ideas; by exploring the potential of conceptual spaces; and by making transformations that enable the generation of previously impossible ideas' (p. 1). While there have been attempts to combine the fields of AI and creativity, and to define them through the emerging field of computational creativity, it has often ended in confusion. Computational creativity (CC) (also known as artificial creativity or creative computation) places AI/computers at the centre of creativity (Colton and Wiggins 2012). Computational creativity is underpinned by Rhodes' 4P's of creativity theory, which emphasises that creativity is an interaction between four factors: process, person, product, and press (environment) (Rhodes 1961). While all four factors are crucial for human creativity, Cropley et al. (2021) have suggested that only two factors are important for human and artificial creativity: process (i.e., cognition), and product (i.e., outcome). Creative products are measured by novelty and effectiveness (Cropley and Cropley 2012; Cropley and Kaufman 2012), where novelty refers to a new or original idea or concept, and effectiveness is the ability of the product or solution to achieve its desired result. Process is defined as the cognitive mechanisms of creativity and is key to understanding what artificial intelligence can offer to develop novel and effective solutions to problems. Therefore, to encourage the use of creativity and AI, educators should consider the process by which creativity has unfolded and/or the product of the creative endeavour.

There is emerging research on assessing the creative product using AI-based methodologies. Cropley and Marrone (2021) demonstrate how AI can successfully assess figural creativity using convolutional neural networks. Beaty and Johnson (2021), and Olson et al. (2021) also demonstrate the use of latent semantic analysis to assess the creativity of student responses to a traditional alternate uses task. While this is a growing field, this research focuses more on the outcome or product of creativity and less on the process.

#### *1.3. The Process of Creativity and AI*

Students should be aware of how AI can support their creativity and learning. Modern education favours problem-solving-based pedagogies, which emphasise the importance of fostering children's ability to think creatively. However, considerable research supports the existence of a creativity slump in younger children across subjects (Torrance 1968; Tubb et al. 2020). One proposal for this slump is an overly structured school curriculum and a lack of play-based learning activities in educational practices (Alves-Oliveira et al. 2017). Emerging research shows how AI can support skills often associated with creativity, such as curiosity (Gordon et al. 2015), grit, persistence, and attentiveness (Belpaeme et al. 2018). The ability of AI to support creativity is also being explored. Kafai and Burke (2014), in their study, report that the purpose of AI in education is to encourage and support skills such as problem-solving and creativity through collaboration with AI, rather than simply acquiring knowledge in the specific domain. The paper suggests that AI can help creativity unfold and is therefore related to the process through which creativity occurs. Furthermore, Ryu and Han (2018) studied Korean school teachers' perceptions of AI in education and report that teachers with experience in leadership recognized that AI would help to improve creativity. Therefore, it is proposed that AI in education may address some of the main concerns associated with the creativity slump, particularly an emphasis on the creative process. This may help improve creative thinking in students and comfortability using AI, and to adequately prepare students to enter the modern workforce.

To successfully combine and integrate AI and creativity, we must better understand how students perceive the relationship between the two concepts. To understand this perception, we should also situate AI with other predominant creativity theories, including the 4C model of creativity.

#### *1.4. A 4C Approach to AI*

Creativity and AI in an educational context can be viewed through a 4C model (Kaufman and Beghetto 2009). Mini-c or 'personal creativity' embodies the personal (Runco 1996; Vygotsky 2004) and developmental (Cohen 1989) aspects of creativity. Mini-c relates to subjective self-discoveries that are creative to the individual involved and not necessarily others. An example may be an individual making a slight variation on a well-known recipe. Little-c is also called 'everyday creativity' and refers to something other people recognise as creative, such as generating a new recipe. Pro-c or 'professional creativity' is defined as becoming an expert in any field or discipline. An example may be the chef, Gordon Ramsey. Big-C or 'legendary creativity' is defined as eminent creativity and will be remembered for centuries. An example may be August Escoffier, who is credited as the founder of modern cuisine and has dramatically altered the field of cooking (Beghetto et al. 2016).

Most obviously, AI can support creativity at the pro-c and potentially Big-C levels, as it can help extend expert knowledge in specific domains. Less obvious is how AI can support mini-c and little-c contributions. At the mini-c and little-c levels, the creative output is not as crucial as the self-discovery that occurs through the creative process. It is therefore essential to develop both an appreciation and understanding of when and where AI is most valuable, that is, in what narrow domains does AI best suit education, and how can AI be used to encourage mini-c and little-c contributions?

This research will investigate how students perceive AI and creativity, and the relationship between the two. We expect insights to highlight how AI can be designed to support creativity in the classroom.

#### **2. Materials and Methods**

#### *2.1. Participants*

Eighty secondary school students from four South Australian schools (mean age 15) participated in an eight-week programme. Students were tasked with the challenge of: 'How do we sustain life on Mars?' Sixty students completed this task as part of their regular science class. Twenty students completed this task as an extracurricular after-school programme. The programme's content was identical, irrespective of whether the student participated in their regular science class or as an extracurricular activity. The same staff conducted both versions.

#### *2.2. Method*

Grounded theory (GT) is a structured yet flexible methodology that is appropriate when little is known about a phenomenon (Chun Tie et al. 2019). Grounded theory investigates the experience of people and their responses and reactions and generates a theory. A defining characteristic of GT is that it aims to generate a theory that is grounded in the data. Considering there is minimal research on student perceptions of AI and creativity, this methodology was chosen.

#### *2.3. Context*

The students explored a variety of sub-problems related to their task; however, one task was around designing and building a Mars Rover. Those who engaged as part of their science class worked in groups of 4–5 students, and each team spent one week (four × 50-min lessons) engaging solely with artificial intelligence and building their Rover. For the other seven weeks, students engaged with the AI system, once a lesson for approximately 10 min each time (40 min per week over seven weeks). The students who engaged in the extracurricular version of this programme also were in groups of 4–5 and engaged with the AI system for six hours over a one-day, in-person event. The other lessons were hosted on Zoom and did not involve AI. The students physically built a Mars Rover using Fischer Technik kits and then engaged with an AI-based vision analytics tool to receive feedback on their build. Whilst the technology behind the vision analytics tool has been created by individuals at the pro-c level, its application in the classroom was created to elicit mini-c or little-c creativity in students. This is because the students use the system to get specific and targeted feedback on every step of their build. The students can then use this information to decide if the AI is helping them achieve their goals of creating the Rover. Once students had built their Rover, the vision analytics system could scan it and upload it into a 3D virtual environment, where students could drive their Rover on Mars. Here they learnt about planetary factors, such as gravity, and terrain.

This was an open-designed task with no instructions, and students were instructed to be creative with their choices and designs. They received creativity training, specifically: "What is creativity and what is it not?".

#### *2.4. Data Analysis*

Twelve focus groups were conducted with the students engaged with this project in their regular science lessons. Eight one-on-one interviews were conducted with those students who participated in this programme as an extracurricular programme. The questions asked to all the students were the same, regardless of whether they engaged in their class or as an extracurricular activity. The interviews were framed around how students perceive both AI and creativity. See Appendix A for the interview questions. A content analysis methodology was used to analyse the meaning of the participants' narratives. Fraenkel et al. (2006) define content analysis as 'a technique that enables researchers to study human behaviour in an indirect way, through an analysis of their communications (p. 483). The purpose of content analysis is to explore participants' verbal communication and social behaviour without influencing it. Content analysis allows a researcher to interpret what is being communicated, why it is being communicated, and with what effects (Wagenaar and Babbie 2004). An objective codification process characterises content analysis and involves placing coded data into key categories and more abstract concepts.

One conceptualisation of creativity and AI that emerged from the students' remarks was labelled 'Social Factors'. Typical categories defining the concept were 'conversation and lack of awareness', 'student interest' and 'social intelligence/social skills'. Another different conceptualisation identified in the content units was 'affective'. Typical categories defining this concept were 'comfortable with AI' and 'not comfortable with AI'. A different kind of conceptualisation was observed in the cognitive view expressed by some of the students interviewed. This led to the concept 'Technological Factors'. The typical categories

here were 'access and use of AI', 'technology focused', 'robotics', and 'computers'. The final concept was labelled as 'Learning Factors'. The typical categories related to the student's current school environment were 'AI provides a learning aid', and 'creativity takes time'. These concepts are shown in Appendix B, along with the content units from which they were derived, and the categories defined by these content units.

#### **3. Results and Discussion**

This study aimed to understand how the students view the relationship between AI and creativity. This topic was addressed through a content analysis interpretation of the students' responses to key questions. The results highlight that the students in the study understood the relationship between AI and creativity as four fundamental concepts: social, affective, technological and learning factors.

#### *3.1. Social Factors*

The results from the interviews suggest that secondary school students in Australia hold opinions that AI can negatively impact their social skills. The AI facilitators/barriers category tended to include negative views and perceptions of AI. Previous research notes that AI will drive us into roles that require more social skills and typically encourage these social-based roles (Deming 2017; Makridakis 2017). However, the students believed that AI would negatively impact their social skills. Comments such as 'AI can make people lack 'social-wise'. AI can make social intelligence weaken a little bit, which can affect them (students), and another comment: 'Well, if we're talking about robots and such for computers and phones and digital media social media, that kind of stuff ... it's taking away from people's social lives, and they're just more concerned about having a digital platform to present themselves on, rather than focusing on presenting themselves in the physical world.'. One student reported that getting AI to become 'a mainstream thing so everyone can speak to everyone on it, so we can ask whole communities and get out with a lot of people' was essential to changing the conversations about AI. These somewhat negative perceptions may hinder students' willingness to adopt AI technologies in their classrooms. Chai et al. (2021) demonstrate that the intention to learn AI in primary school students is influenced by the students' perception of the use of AI for social good. Furthermore, Chai et al. (2020) highlight that students perceive the purpose of learning of AI for social good as the most powerful predictor for their behavioural intention to continue learning AI. The students also reported that AI will never work in fields where human skills are required for problem-solving. When asked whether AI can match human skills, one focus group reported that the father of a participant in this group was a pilot. They mentioned that it was crucial AI never entered the cockpit as humans should be tasked with solving a complex problem like flying a plane. Interestingly, every member of this group agreed and seemed apparently unaware of the level of technology that is associated when flying. This represents a gap in student understanding of how AI can be used to assist humans. The students in this group failed to see the value of AI as a teammate and solely viewed this role as a human skill. Further emphasis should be placed on educating students on the role of human–AI teaming, and that AI can support humans, even in seemingly social or complex situations. The belief that AI can negatively hinder their social skills also represents an opportunity to demonstrate how AI can benefit social skills and enhance connections across communities.

#### *3.2. Affective Factors*

Students reported various affective responses to AI. Those students who verbally reported feeling more familiar with AI also reported feeling more comfortable using AI technologies. However, the students who said they were not sure what AI was, also said they felt less comfortable defining AI, as well as integrating it into their classrooms. This finding is supported by both Chiu (2017), and Teo and Tan (2012). These authors highlight that a positive attitude towards technology can explain one's intention of using

the technology. One student reported feeling comfortable because he had 'all the safety programmes on it (his computer)', so he reported having trust in his AI systems. Another student responded, 'depends on the type of AI, so, I guess computers and programming and telling a computer instructions'. When prompted, they reported they wouldn't feel as comfortable using 'robots and machines'. This transparency in the AI system relates to an increase in trust in the AI. This is in line with previous research that transparency and the avoidance of 'black box' suggestions can foster AI adoption. This is referred to as explainable AI (Lundberg et al. 2020).

#### *3.3. Technological Factors*

Interestingly, the majority of the students' perceptions of AI were related to technological factors. Categories such as advanced technology, automation, coding/programming, futuristic, not human and robots, all had a lot in common. Students typically thought of AI as robots or computer-based, as this is how they interact with AI in their daily lives. These comments can be interpreted as the students possessing quite a limited view of AI applications, and they all struggled to move beyond the idea that AI is more than robots and computers. Several students felt that AI was a 'futuristic' phenomenon and was not as impactful in their current lives. All students reported that AI, to them, included some form of robotics. Chiu et al. (2021) and Chiu and Chai (2020) suggest that students should learn about AI by referring to real-life applications that they are likely to encounter in their daily experiences.

When asked if AI can ever match human creativity, students reported that, despite AI being technically superior to humans, human creativity will always be a uniquely human trait that should be fostered. One student commented, 'Basically, most things in artificial intelligence are made by humans so, unless we actually create a robot which can be a human, it probably won't be able to match the creativity of humans.'. The students who did believe that AI could match human creativity suggested that 'maybe over time, when technology gets a lot more advanced, I think that it would be eventually possible to be as creative as humans'. Thus, they didn't think AI could currently match human creativity but may do so in the future. When asked 'do you think AI could ever match human creativity?' One student made a very interesting comment. She said, 'Yes, kind of. It's a very interesting question. I think it can spark creativity. I don't know if AI itself (can be creative). I don't know if a robot can be creative because, in order for a robot to be creative, someone has had to create the robot and give it its creativity as such, so I don't know if they can be creative themselves, but I think they can spark creativity.'. Therefore, they view AI as a way to facilitate or 'spark' creativity. Based on these comments, it is suggested that AI should be used to enhance creativity. Markauskaite et al. (2022), in their recent paper, demonstrate how AI can be used to support creativity across different age groups. The authors polylogue provides concrete suggestions based on a 4C theory of creativity approach on how and where AI can be used to enhance creativity, particularly for students.

#### *3.4. Learning Factors*

The most frequent and mentioned categories are related to the concept of learning factors. The students reported a positive view of AI and that it can support them to access information more efficiently; it can promote global connections, support their ideas, and aid learning. The students also reported that the benefits of creativity include time management and increasing their novel ideas. However, students also reported that their current school environments sometimes negatively impact their ability to exhibit creativity. Unsurprisingly, students mentioned not having enough time to be creative and that assignments were not designed to allow creativity to develop, indicated by comments such as 'sometimes you can't (be creative); sometimes you do have a set structure of things that you have to follow, and you can't always be creative, which can sometimes be a bit sad because you want to do something interesting but sometimes you know you have to follow a set structure for an assignment or something'. The students provided suggestions on how

their learning environments could support creativity. The students felt that AI could help develop their creativity by encouraging independent thinking and creating opportunities to be creative, such as encouraging 'new ways to approach different situations'. Another student mentioned, 'Also, if you're trying to make a robot move down a path or something, sometimes it's going to bump into things and it's going to, you know, go a bit wonky, so you've got to think out of the box and you, hang on a second, what's going wrong here and then backtrack kind of thing, thinking in a different mindset, I guess, to how you usually think.'.

The students think AI can assist creativity when asked to deepen their thoughts in their learning. It is suggested that schools adopt opportunities for students to engage with creativity and AI as the students desire to engage in these activities.

#### *3.5. Theoretical and Practical Contribution (From 4C to 4AI)*

The students' perceptions of AI varied; those more comfortable with AI had a more comprehensive understanding of the concept. This is in line with the research on trust with AI research (Ashoori and Weisz 2019). Similarly, those who accurately defined creativity and valued the competency tended to think AI could never match human creativity. However, what was notable was that, when students were asked to define AI, they had a very limited understanding of the concept and tended to view AI as general AI or Artificial Superintelligence. The students had experienced an intensive programme using narrow AI, so it was surprising that they did not acknowledge this. Adopting a 4C approach to these results, we propose that the students do not value what we have termed 'everyday-AI' (a combination of mini-c and little-c).

It is proposed that the effective integration of AI into classrooms must address the misconceptions students may have about AI. By extending the 4C theory of creativity, we propose a '4AI model of Artificial Intelligence'. Following the same principles of the 4C model, we suggest mini-AI, little-AI, Big-AI and legendary-AI. Students described an evident appreciation of Big and legendary AI but did not appear to appreciate the mini or little AI (despite the AI tool being created to support mini-c and little-c). Drawing analogies with the 4C theory of creativity, we propose that thinking about four aspects of AI, perhaps as a '4AI Model of Artificial Intelligence in Education' may be useful. Therefore, educators should focus on this aspect as it is unlikely that Big- or legendary-AI will be as frequently experienced by students in the same way that children are more likely to experience mini-c and little-c. This could include explaining the myths and misconceptions of AI and encouraging students to look for and appreciate examples of mini- or little-AI in their everyday lives. There is also the suggestion that, as with creativity, where there is teaching with creativity, for creativity, and about creativity, there should be teaching for AI, with AI, and about AI. Within these three domains, mini- and little-AI can be explored. It is proposed that students would increase their realistic understandings of AI over time, and some of the issues raised by the students who participated in this programme could be minimised.

#### *3.6. Future Research*

This study investigated student perceptions of AI and creativity and has proposed a 4AI model of creativity and AI. Future research could establish this model through both qualitative and quantitative methods. Quantitively, AI-based tasks could be employed in classrooms, delineating mini-AI (perhaps around personalized feedback in learning) versus little-AI. Furthermore, this model could be compared against pre- to post-measures of creativity. Further qualitative work could explore broader perceptions of everyday AI in children and adolescents. Finally, future research should focus on increasing students' limited views of AI to incorporate more of what AI entails and how widely it permeates society and their learning environments (Yufeia et al. 2020).

#### *3.7. Limitations*

This study has several limitations. First, this study was limited to secondary school students in South Australia, Australia. Further research should examine and compare K-12 students' perceptions from other countries and demographics. Secondly, the students reported issues with the AI system effectively working every time they used it. These issues may have contributed to some poorer attitudes for students, if this was their first experience working with AI. Thirdly, whilst the interviews provided rich and in-depth insights into student perceptions, more empirical attitude measures could have been used, which would have provided further insights.

#### **4. Conclusions**

The interviews highlighted that the students view the relationship between AI and creativity from four key concepts: social, affective, technological and learning factors. Most of the students reported that, although AI could never match human creativity, AI could certainly help them develop their creativity. A 4AI model of Artificial Intelligence has been proposed to help educators support mini-AI and little-AI experiences, which the findings show was overlooked by the students, despite these being the core of the programme they had experienced. Future research could focus on using AI to address the concerns students mentioned and be used to enhance their creativity.

**Author Contributions:** Conceptualization, R.M., V.T. and G.H.; methodology, V.T. and R.M.; formal analysis, V.T. and R.M.; writing—original draft preparation, R.M., V.T. and G.H.; writing—review and editing, R.M., V.T. and G.H.; project administration, R.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of The University of South Australia (protocol code 203661 and date of approval 13 January 2021).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Restrictions apply to the availability of these data. Data was obtained from students and are available from the authors with the permission of the students.

**Acknowledgments:** The authors would like to acknowledge the participants and their teachers.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

**Creativity and Artificial Intelligence—a student perspective** Interview Questions for one-on-one interviews **Creativity:**


I will now move into some questions on artificial intelligence.


#### **Artificial Intelligence:**


#### **Creativity and AI:**


Due to nature of the focus groups, we condensed the above 18 questions into 11 questions **Interview Questions for Focus Groups Artificial Intelligence:**


#### **Appendix B**

**Table A1.** Content units, categories and concepts derived from the qualitative data.


**Content Units Category Concept** AI provides easy access to information Impact of school environment Learning AI provides global connection AI provides idea support AI provides learning aid AI can increase perspectives Creativity helps manage time Creativity can increase novel ideas Structure of assignments limit opportunities for creativity Creative block Lack of foundational knowledge to be creative Creativity takes time It is a risk being creative (Lack of) creative experience Independent thinker Encourage creativity Creates opportunities Creative Problem Solving

**Table A1.** *Cont.*

Table A1 illustrates that the students in the study understood the relationship between creativity and AI in terms of four fundamental dimensions (referred to as 'concepts' in the table): social, affective, technological and learning factors.

#### **References**


Fraenkel, Jack R., Norman E. Wallen, and Heo Hyun. 2006. *How to Design and Evaluate Research in Education*. New York: Mac Graw Hill.


Torrance, E. Paul. 1968. A longitudinal examination of the fourth grade slump in creativity. *Gifted Child Quarterly* 12: 195–99. [CrossRef] Trilling, Bernie, and Charles Fadel. 2009. *21st Century Skills: Learning for Life in Our Times*. San Francisco: John Wiley & Sons.

Tubb, Adeline L., David H. Cropley, Rebecca L. Marrone, Timothy Patston, and James C. Kaufman. 2020. The development of mathematical creativity across high school: Increasing, decreasing, or both? *Thinking Skills and Creativity* 35: 100634. [CrossRef]

VanLehn, Kurt. 2011. The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. *Educational Psychologist* 46: 197–221. [CrossRef]

Vygotsky, Len S. 2004. Imagination and creativity in childhood. *Journal of Russian & East European Psychology* 42: 7–97.


## *Article* **Teacher Creativity: When Professional Coherence Supports Beautiful Risks**

**Marie-Hélène Massie 1,\*, Isabelle Capron Puozzo <sup>2</sup> and Marc Boutet <sup>1</sup>**


**\*** Correspondence: marie-helene.massie@usherbrooke.ca

**Abstract:** Environmental, cultural, and social issues are becoming increasingly complex, and the educational context is no exception to this trend. The relevance of teachers' creativity in examining situations from different angles, in imagining new approaches, in adapting to the varied needs of students, and in training them so that they too can grasp the teeming complexity seems obvious. However, creativity sometimes seems to be taken for granted among teachers and educational programs leave a gap around this theme. Since the scientific literature tends to show that teachers' creativity is still little explored in educational contexts, this doctoral research studies its manifestations within a group of teachers enrolled in a professional master's program in preschool and elementary education (Université de Sherbrooke, Canada). Within the framework of this program, each one elaborates a professional development project over a three-year period. Using a variety of authentic data sources (observations in natural occurring situations, reflective writing by participants, and semi-structured interviews), their creative process is documented and analyzed. This multiple-case study (n = 9) that draws on the concept of creativity as related to that of professional coherence reveals that the pursuit of greater professional coherence not only enables the implementation of creative skills to foster teachers' professional development, but also elicits beautiful risk-taking.

**Keywords:** creativity; beautiful risks; professional coherence; onion model; professional development; continuing education

#### **1. Introduction**

Several researchers agree that the development and maintenance of creative potential generally contribute to improving the ability to solve problems of various kinds, to adapt or to face complexity more serenely (Capron Puozzo 2016a; Craft 2005; Csikszentmihalyi 1996; Lubart et al. [2003] 2015; Plucker et al. 2004; Robinson [2001] 2017). For teachers who perceive that their profession is becoming ever more complex (Mukamurera et al. 2014), the implementation of their creativity remains an interesting avenue, both for reaping the benefits and providing suitable conditions in the classroom to foster its expression in their students (Beghetto 2005; Cayirdag 2017; Craft 2005).

Officially, *the Quebec Education Program* (Quebec, Canada) mentions that the professional character of teaching is based, among other things, on pedagogical practices that rely on the creativity of teachers (Government of Quebec 2006), without specifying how. The *Référentiel de compétences professionnelles pour la formation du personnel enseignant (Professional Competencies Framework for Teacher Education)* (Government of Quebec 2020) also states that teaching requires "a certain level of creativity" (p. 28). In addition to noting that this formulation is rather nebulous, we observe, by examining the preschool and elementary teacher initial education plans of Quebec universities, that the intentions to teach creativity competencies are scarcely included. In general, the programs do not focus on "the acquisition of skills specific to leadership, ambiguity tolerance, risk-taking or creative problem-solving" (Ouellet 2012, p. 158). As a result, teacher creativity seems somewhat

**Citation:** Massie, Marie-Hélène, Isabelle Capron Puozzo, and Marc Boutet. 2022. Teacher Creativity: When Professional Coherence Supports Beautiful Risks. *Journal of Intelligence* 10: 62. https://doi.org/ 10.3390/jintelligence10030062

Received: 31 July 2022 Accepted: 29 August 2022 Published: 2 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

taken for granted since little procedural knowledge of creativity is provided in pre-service development. Inevitably, the barriers to implementing it when it arrives in the school setting persist (Beghetto 2010; Capron Puozzo 2016b).

The scientific literature tends to show that the development of student creativity has been studied from a variety of perspectives, but that there is still an insufficient amount of research on creativity in teacher education (Terzidis 2019). To better understand how creativity translates once initial education is completed and how it can support their professional development, in a master's program in preschool and elementary education (Université de Sherbrooke)1. Over a three-year period, students, in-service teachers, design a professional development project on a topic of their choice, according to their interests and concrete pedagogical or didactic needs. The main intention of the process is to achieve greater professional coherence (Korthagen 2004). This program appears has a favourable context to study teachers' creativity, since certain factors conducive to its development come together. In particular, the accompanying posture favoured by the trainers' (a professor and a lecturer) aims to emancipate the students (Boutet et al. 2021; Paul 2012). Believing in reflective practice as a driving force for professional development and changes in practice (Schön 1983) as well as in the value of experience (Dewey 1967; Kolb 1984; Lewin 1951), they encourage risk-taking, openness to new experiences and ambiguity tolerance. Those pedagogical components foster creativity, according to Lubart et al. (Lubart et al. [2003] 2015).

Aware that creativity is attractive in educational settings (Cropley 2010) and that it seems good to be defined as a creative person (Karwowski 2009), we were keen to study the phenomenon in a natural context (Guba and Lincoln 1982), i.e., where students are not pressured externally to be creative, not being evaluated in this regard. Broadly, we are interested in the contribution of creativity to professional development, but for this article we focus on one of our research sub-questions which concern the risk-taking that students took in developing their project. What is the nature of the risks (Beghetto 2019) that were taken? What place do they take in view of their professional coherence (Korthagen 2004)? What level of creativity (Beghetto 2019) did the students express in the development of their project? This article brings forward some of the answers.

#### **2. Conceptual Framework**

To introduce the conceptual framework, it is important to first clarify the relevance of professional coherence, specified by Korthagen's (2004) onion model, as it is foundational to the program's approach. The concept of creativity and one of its influential conative factors (Lubart et al. [2003] 2015), taking beautiful risks (Beghetto 2019) is then brought into light to provide a complementary scientific insight for this professional development space.

#### *2.1. Professional Coherence*

For the teaching community, which daily modulates its interventions and practices according to singular situations by assuming the great complexity of the school environment (Wanlin and Crahay 2012), reflective practice seems to be a precious, even essential asset (Boutet et al. 2016; Guillemette and Gauthier 2008; Lison 2013; Perrenoud 2001, 2004; Tardif et al. 2012; Vacher 2015; Wentzel 2010). Since the work of Schön (1983), it has become increasingly important in research and devices on professionalization. He points out that competent practitioners generally know much more about their practice than they can say about it, hence the interest in getting them to seek out all the knowledge buried in the depths of their professional action.

Engaging in the program, students are prepared for the development of their project, by being led to explicit the elements that compose their professional universe based on Korthagen's onion model (Korthagen 2004) (Figure 1). Resting on positive psychology, this model of reflection allows to examine each of the layers that are intimately related to go beyond a purely cognitive reflection (Korthagen 2017).

**Figure 1.** Onion Model—Level of Change (Korthagen 2004).

Thus, with the help of a framework, each person puts into words specific personal and professional components that influence each other: their fundamental qualities, their deep mission (why they chose this profession, what inspires them), their identity (who they are as a teacher), their beliefs (what matters to them and underlies their choices), their skills (what they feel capable of), their behaviours (the concrete actions they take) and the environment in which they navigate (and which influences them daily). The students can then become aware of what is likely to hinder the coherence between their internal and external factors. In this regard, Dewey (1947) reminds us that during moments of rupture, questions come up and consciousness, "whose role consists precisely in readjusting experience, a critical and creative role, an essential role" (p. 6), emerges. The feeling of uneasiness felt in the rupture then creates an imbalance. Returning to one's core mission during difficult times would support and propel the teacher's commitment to change (Day 2004) as it gives meaning to the profession. This step back also allows practitioners to realize that they hold the power to prevent certain constraining factors in determining their behaviours; this awareness of choice contributes positively to autonomy (Korthagen and Vasalos 2005).

#### *2.2. Creativity*

The need to adapt to the environment is the basis of the creative act, since without an imbalance with the world around, individuals will not feel the need to exercise their creativity (Csikszentmihalyi 1996; Vygotsky 2004). The work of Terzidis and Darbellay (2017), establishes the relevance of supporting creativity at the service of professional development, as an avenue to ensure its sustainability. Returning to core quality, identity, and professional beliefs (Korthagen 2004) beforehand serves as an anchor for the creative process. Several definitions of creativity exist, and our research is based on one proposed by Lubart et al. (Lubart et al. [2003] 2015), which is "the ability to produce work that is both novel and appropriate to the context in which it occurs" (p. 23). Originating from differential psychology, it is agreed upon by several researchers such as Amabile (1983), Beghetto (2021), Capron Puozzo (2016a), Robinson (Robinson [2001] 2017), Runco and Jaeger (2012), or Sternberg and Kaufman (2010) who also recognize the novelty and relevance necessary as criteria for creativity. Supported by several works, including those of Corazza (2016), Botella and Lubart (2019) bring forward the dynamism perspective for the creative process. This approach incorporates, among other things, interactions with

the environment and the perpetual motion of the process, even after production has been completed.

To avoid reinforcing the myth that creativity is only about a handful of privileged individuals revolutionizing history (Plucker et al. 2004; Robinson [2001] 2017), Kaufman and Beghetto (2009) propose the Four C model distinguishing four levels of creativity:


This model provides a framework for including creativity in school curricula and helping students develop their creativity at higher levels (Beghetto and Kaufman 2013). To make it even more appropriate in the school setting, Beghetto (2019) refines it by removing the *big-C* and adds *surprising little-c* (when the student meets the criteria in a surprising, original, or significantly different way) and *no-c* (when there is no creative attempt). By incorporating them into the continuum, they can help students better situate and challenge themselves. It is this modified model that we adopted for our study.

To understand the leverages and obstacles that can influence or inhibit the creative process, the multivariate approach (Lubart et al. [2003] 2015) provides a systemic view that binds four types of influencing factors: cognitive, conative, emotional, and environmental factors. Cognitive factors refer to the knowledge and intellectual abilities that underlie creative thinking. Conative factors, which refer to the ability to induce intellectual energy in the performance of a task to achieve a solution or completion (Reitan and Wolfson 2000), specifically relate to perseverance, ambiguity tolerance, risk taking, openness to new experiences, and individualism (Besançon and Lubart 2015). Emotional factors are based on the affective state inducing positive sensations such as well-being, appeasement, joy or negative such as stress, frustration, or anger (Capron Puozzo and Botella 2018). The environmental factors refer to the contextual elements in which the creative process takes place.

Through all these variables, creativity is, therefore, complex, and Beghetto (2005) denotes that the most valuable form of creative expression often occurs within the boundaries of real-life structures, rules, and norms. In the school setting, teachers who take more risks and initiatives in difficult situations are more likely to demonstrate creative teaching behaviours, as they accept constraints rather than challenge or attempt to eliminate them (Cayirdag 2017). They then manage to turn adversity into an advantage (Kaufman and Gregoire 2016).

#### *2.3. Taking Beautiful Risks*

Risk-taking has been identified as influential to the creative process, as creative ideas differ from the usual ideas of a community (Csikszentmihalyi 2006; Lubart et al. [2003] 2015; Sternberg and Lubart 1995). Tyagi et al. (2017) further demonstrates that it is social risktaking that is linked to creativity. People who are considered creative are more likely to present their ideas and creative outputs to a group for evaluation. Therefore, they show a high level of risk, as it is possible that some or all members of the group may reject the idea or production in question. Research has shown that school environment is still often seen as rigid and not very open to risk-taking (Besançon and Lubart 2015; Csikszentmihalyi 2006). Ouellet (2008) adds that the dominant conventional views of society, the education system and professional environments impose significant challenges to the development of a creative societal culture. For fear of displeasing colleagues, managers or parents, many teachers prefer to keep their ideas, however attractive, to conform, to blend in, rather than risk being disturbed or rejected (Huret 2014). The pressure of the environment leads many to look for quick, short-term solutions to patch things up rather than to address the real causes of the problem and solve it (Korthagen and Vasalos 2005). However, creative

risk-taking could contribute finding innovative ways to solve problems, for the benefit of students or the school.

At the root of all historical advances, creativity can have very positive repercussions, but also negative ones, if ill-intentioned. Amabile (2018) reminds us that "it is only by combining creative capacities, strong passions, and conducive environments with equally strong moral values that we will be able to harness the power of creativity for the good of humanity and not its destruction" (p. 13). Thus, the values underlying creativity matter. Beghetto and Anderson (2022) abound in this sense by suggesting an approach to creativity that aims to a beneficial contribution to others. Beghetto (2019) calls for taking "beautiful risks" (p. 2) which he distinguishes from good and bad risks. The good risk occurs when the potential benefits outweigh the potential costs and the bad risk, conversely, presents potential costs higher than the possible benefits. Beautiful risk occurs when the potential to make a positive contribution to others outweighs the potential costs. Beghetto gives examples of a teacher who shares a pedagogical practice she or he has developed with colleagues, who accepts the uncertainty caused by a new and complex challenge, or who continues to believe in a student's potential even though other colleagues have stopped doing so. To plan such a contribution, one must first ensure that the planned action can benefit others, that possible hazards have been addressed, and finally, that the plan is carried out.

In the context that we studied, the participants are asked to design a professional development project that builds on core qualities "considered as the driving force of productive teacher learning, and also as fundamental to the development of competencies" (Korthagen 2017, p. 396) as well as a positive mission and beliefs of teaching (Korthagen 2004, 2017). This process is part of a creative one, according to Lubart et al.'s ([2003] 2015) definition of creativity, as students are challenged to update their practice (novelty) while respecting the elements of their context such as the program, their pupils, and the constraints of their environment (relevance). It is therefore a question of seeing whether the students took creative risks to harmonize their coherence.

More specifically, our research sub-objectives regarding this conative factor aim to determine the nature of the risks taken in the development of the project (based on Beghetto's three criteria), to identify the components of the onion model (Korthagen 2004) that are consistent with taking beautiful risks and to specify the level of creativity expressed by the students, according to Beghetto's scale.

#### **3. Method**

To describe and understand in depth the issues of creativity, particularly concerning the taking of beautiful risks in a real situation of professional development, that is, in a research space where the phenomenon and the context merge (Arbarello 2011; Merriam 1988; Miles and Huberman [1994] 2003; Yin [1984] 2018), the multiple-case study was privileged. This methodological choice was appropriate for our exploratory research, in accordance with Stake (2006) and Yin ([1984] 2018) who state that case studies can explain, describe, or explore events or phenomena in their singular context. The choice to cumulate several cases was intended to take advantage of the possibility of comparison and add to the scientific rigour (Creswell [2006] 2013; Gagnon [2005] 2011; Yin [1984] 2018). Without testing or corroborating a hypothesis, the analytical approach of this multiple-case study is deductive, as it draws on existing theoretical frameworks (factors of creativity and creative risk-taking), allowing for the delineation of cases (Alexandre 2013).

The sample (n = 9) is drawn from a cohort of students, in-service teachers, enrolled in a three-year master's program in preschool and elementary education (Université de Sherbrooke, Canada). Initially, 17 people in the group expressed interest in participating in the research project at the time of the first contact in April 2021, that is, once the master's degree was completed. Fourteen individuals completed and returned the ethical consent form, but only nine followed up to participate in a semi-structured interview (Savoie-Zajc [1984] 2016). This purposeful sampling (Creswell [2006] 2013) allows nonetheless for an optimal number of cases to conduct a multiple-case study, conforming to Stake (2006) and Yin (Yin [1984] 2018) who suggest between 4 and 10 as an appropriate sample. This sampling strategy allows for consistency in case studies, as the research object is specific to this continuing education context (Hamel 1997; Merriam 1988; Pires 1997; Savoie-Zajc 2007). Moreover, the sample's representativeness in terms of the gender distribution in teaching can be observed: the sample includes eight women and one man, which corresponds well to the current portrait of the school environment, where in 2017–2018 men represented approximately 12% of the teaching staff (Government of Quebec 2019).

In order to achieve the richest and most rigorous multiple-case study possible, three qualitative data collection methods were used (Creswell [2006] 2013; Merriam 1988; Stake 2006; Yin [1984] 2018) (Figure 2): (1) written assignments from all three years of the educational program were selected; (2) individual meetings with the professor and course activities that were filmed during the third year; (3) semi-structured interviews were conducted six months after the end of the master's program.

**Figure 2.** Data Collection Methods.

Data collection methods covering a three-year period allows for some "temporal thickness" (Leplat 2002, n.p.), a strength of the research design. Through various studies, Cropley (2010) identifies some inconsistencies in the teaching community's discourse about creativity. Runco and Johnson (2002) also point out that it looks good for teachers to say they value creativity in their classrooms even if, in fact, they do not necessarily do so. Given that this type of gap often found between espoused theory (what individual claims to follow) and theory-in-use (deducted from the action) (Argyris and Schön 1999), observations to capture live data seemed necessary to corroborate the reported practices and intentions mentioned in the written work. COVID-19 pandemic made it even easier to access the natural context without interfering with it, since to provide quality support, the trainers recorded individual meetings and class sessions. The semi-structured interviews allowed for a privileged return on the experience (Savoie-Zajc [1984] 2016) and mainly identified the students' perceptions of creativity through the development of their project. To personalize each of the interviews, a chronological pre-analysis of the training course of each participant was conducted (Bardin [1977] 2013), with the goal of designing a specific timeline. This tool supported for the targeting of moments, recalling details already far away, and again decreasing the desirability bias, by relating the reflection to observable facts that happened during the program. In addition to ensuring the triangulation of data, using this variety of data collection methods helped to reduce the inherent biases and limitations of each (Mucchielli 2002) and increase the level of objectivity.

To process the collected data, a qualitative content analysis according to Bardin ([1977] 2013) was completed in three steps: (1) pre-analysis, (2) exploitation of the material and (3) processing of the results including inference and interpretation. In addition to preparing for the semi-structured interviews, the pre-analysis granted for the constitution

of the corpus, following the rules of completeness and non-selectivity (considering all documents that justify its rigour), representativeness (analyzing a sample), homogeneity (comparability of documents), and relevance (ensuring that the documents are adequate as a source of information).

Subsequently, a categorical analysis, consisting of cutting the text into units of meaning and classifying them into categories with a common character (Alexandre 2013; Bardin [1977] 2013), was carried out based on Lubart et al.'s ([2003] 2015) creativity factors (cognitive, conative, emotional, and environmental). Chronological analyses, based on in-class situations, written works, and interviews verbatim, allowed us to respond to our research objectives. To ensure a thorough analysis of each data source, Gaudet and Robert's (2018) vertical analysis proposal was chosen. To do so, the first step is to accomplish a contextualizing condensation by identifying the material and documenting its production context. In a second step, a semantic condensation is performed with the aim of reducing the material in terms of meanings derived from the research questions. Several readings of the material or viewings of the recordings were necessary to identify relevant themes and make links to the scientific literature on creativity before reducing, to avoid losing important content. Moreover, following Miles and Huberman ([1994] 2003), a within-case analysis was carried out to provide a full description for each of the nine cases. Finally, the cases were linked through cross-analysis to extend the generalization and deepen the understanding and explanation of the phenomenon of creative risk during the design and implementation of the professional development project.

#### **4. Results**

In line with the sub-objectives of this doctoral research, this section summarizes what the participants mentioned, orally or in writing, about the nature of the risks taken in the development of the project (according to Beghetto's three criteria), the components of the onion model that are compatible with taking beautiful risks and the level of creativity they expressed.

#### *4.1. Taking Beautiful Risks for the Professional Development Project*

The first finding that emerges from the analysis concerns the taking of beautiful risks, the main element of interest in this article. To be identified as such, this type of risk must meet three criteria (Beghetto 2019): Is there a potential benefit to others? Have potential hazards been addressed? Were actions taken? (Table 1) Regarding the first question, participants specified the goals of their project in a written assignment during the 3rd year (December 2021), which made it clear how their production could benefit others (their students, their school, the school community in general). The analysis of this artifact was supported by what was said during the semi-structured interview. Concerning the potential hazards, qualitative data collected during a team activity conducted in October 2021, reported on their occurrence. They were also corroborated by excerpts of verbatim from the semi-structured interview. Finally, the meetings with the professor at the end of the course and the public presentation of the project in an online symposium were clear evidence of whether the project was put into action or not.

In conformity with Beghetto's criteria, the Table 1 shows that 7 of the 9 cases took a beautiful risk to design and implement his project. For example, as early as Year 1, Case 9 expressed wanting to support students' motivation in his music class.

Extract of verbatim

*For me, motivation is the focus point. I was a student who didn't like school, I found it boring as hell, but I loved learning, so it was clear from the start that my project revolved around motivation.*

After having thought of several ideas (putting on a show, collaborating with a high school teacher, making a recording of traditional songs, having a musical tournament between two groups from different schools. . . ) and having dealt with several constraints (coping with the departure of a sick colleague, dealing with the pandemic, having very limited audiovisual equipment. . . ), an entrepreneurial project was launched so that the students could make the choices for themselves. They chose to produce a video clip that denounced sexism. Both devices (entrepreneurial approach and producing a video clip) were novelty to him, and he was curious to explore them. With the support of an expert in the entrepreneurial approach and a colleague (the lead teacher of the target group), he was able to foresee certain pitfalls (choosing the group of students, finding the missing material, organizing time outside of class. . . ) and complete the project.


**Table 1.** Professional Development Project: Beautiful Risks?

Two of the nine cases (Cases 2–6) did not meet the three criteria for a beautiful risk. They were certainly having a project that benefited their pupils and for which they had addressed the potential hazards. However, being absent on maternity leave towards the end of the course, a leave that was still in effect at the time of their participation in the semi-structured interview, they were unable to put their planning into action at the time of data collection. A clarification is necessary for Case 6. At the beginning of the master's training, the participant was teaching in an international program where the academic curriculum includes heavily built teaching modules. She had expressed in the 2nd year that she felt uncomfortable in this work environment, despite a pleasant and stimulating climate. Following the development of her project, which consisted of building an outdoor education plan, she made the choice to change school in preparation for her return to work. This new environment would permit her to teach where fewer academic constraints existed and, moreover, was geographically located just a stone's throw from a wooded area. So, in this case, it can be considered that one step for action was taken.

#### *4.2. Components of the Onion Model Related to the Professional Development Project*

Still concerning Case 6, it is important to note that this change of school is congruent with some of her professional beliefs, i.e., the importance of cultivating the well-being of her students and taking time to avoid creating stress (Table 2—The excerpts regarding the

onion model are from the very first assignment in the program (December 2018) when participants were asked to paint a picture of their professional reality). This precision leads us to a second result: in general, the beautiful risks were taken in coherence with some components of their onion model. Case 5, for example, expressed her mission as offering its students stimulating and varied activities to support the pleasure of learning, while being concerned with encouraging the development of their critical thinking. She chose to plan a website around three ethical topics (overconsumption, environment, and differences) in a multidisciplinary approach. By creating stimulating interrelated activities that integrated art projects, children's literature, math problems, or science experiments, she was able to explore these themes in a more articulate and in-depth way. For Case 4, the importance of collaboration and sharing among colleagues was a common thread in her onion model and the primary intent of her project was to support in service teachers by providing a well-organized resource bank.

**Table 2.** Professional Development Project and Onion Model Components Related.


Emotional factors seem to have played a role in beautiful risks taking, especially for the students who carried out their project in real time, within their school (Cases 3-7-9). In keeping with their professional beliefs, they were confronted with some colleagues who did not have the same. For Case 3, who wanted to use different evaluation methods than her colleagues, the pressure from the environment was heavy. In a meeting with the professor (December 2021), she cried and mentioned that her sense of personal effectiveness was no longer as strong. The professor suggested that this painful emotion be seen as rich material for reflection and use it to further strengthen the foundations of her project. During the semi-structured interview, she said that this emotional moment was a turning point to find a solution and take action. She illustrated this with a metaphor of rafting:

#### Extract of verbatim

#### *In the beginning, you must paddle and it's very hard, but when you catch the rapids, it's so fun! And even today, I would say that I'm still in the fun part, because it's not over, it's still going on!*

This beautiful risk taken for the benefit of her pupils has helped to anchor her beliefs and she said she felt much stronger, grounded, and she is now able to defend her choices in front of her colleagues. This excerpt also unveils the dynamic side of creativity, as her project is still in movement, corroborated by other statements. This dynamic trend was found in five other cases (1-5-6-7-9), who mentioned that their project was still evolving.

#### *4.3. Level of Creativity of the Professional Development Project*

The third finding concerns the level of creativity. Recall that to be creative, the idea or production must be both new and appropriate (Lubart et al. [2003] 2015). Therefore, from a perspective of coordinating some components of their onion and the objectives of their project, students demonstrated creativity by introducing something new and appropriate (respecting the Quebec School Training Program (Government of Quebec 2006) and the constraints of their school environment) to their practice or school. In light of some exchanges with the professor and the semi-structured interview, we noted the presence of mini-c (self-recognized creativity) for all cases except one (Case 8) for whom the element of novelty was not present.

Extract of verbatim

*Was the project creative for me? No, because there weren't many new things that enhanced my practice; the project allowed me to better organize what I was doing and be able to share it with others.*

Nonetheless, she says that the knowledge she has gained about neuroscience has confirmed the pedagogical choices she was instinctively making. In addition, the creativity of her planning was recognized by colleagues in her school and members of the teaching community who attended the conference (little-c). In this regard, according to the nine individuals, they all perceived their project to be creative for their school. Analysis of the data revealed that by living their project in their community (Cases 3-7-9) or by sharing it with their school colleagues and at the conference (Cases 1-2-4-5-6-8), the feedback received by peers seemed to confirm their perception. For example, the website produced by Case 4 was shared by the educational advisor in professional insertion to new teachers in her service center. The outdoor planning of Case 6 was praised by conference participants, and she subsequently incorporated into an outdoor education committee at her school service center. While designing their project, they conducted a variety of research to support their beliefs and pedagogical choices, to form their ideas, or to see what already existed. Knowing they would have to present their project at a conference and would be invited to disseminate their production, a general concern not to create something that already existed was present in all cases. As a result, they produced something that was creative for their school environment.

Finally, students noted that interactions with peers and accompaniment from the trainers were beneficial in increasing their level of creativity (environmental factors). In six cases, teams' activities during the program to discuss their project was one of the main leverages of their process. Input from school colleagues or even positive comments from parents helped four cases to push their creativity further. Case 7 felt that having a committee of colleagues was positive in overcoming her limitations, despite some constraints related to her school context that caused her project to deviate.

#### Extract of verbatim

*It was a "collective" creativity [* ... *] if I had been on my own, I would have just had my own ideas, but the fact that I was working with other people, the ideas germinated and went even further.*

For eight of the nine cases, the program approach also became one of the most significant leverages for their creativity.

Extract of verbatim (Case 8)

*We always felt the safety net. "Go ahead and explore, you're the leader of your project, but if you have a need, were always here." I could really feel the confidence in us.*

Analysis of the data revealed a recurrence of this sense of security across different artifacts, particularly in the team meetings and the semi-structured interview.

#### **5. Discussion**

The results of this exploratory doctoral research report that most of the students dared to take beautiful risks to design and implement a professional project, which, it should be remembered, was aimed at harmonizing their professional coherence. Given their personal context (absent on maternity leave), two cases were unable to take action. This was not by fear, but a circumstantial cause. For Case 6, who changed schools before returning to work, perhaps it is a good risk in preparation for a beautiful risk? It would be interesting to see if, they will put their project into action when they return to the classroom.

Three of the cases, the more experienced ones (Cases 3-7-9) took greater risks by developing their project in real time and involving colleagues in their school (Tyagi et al. 2017), rather than opting to share planning or resources after the project was completed. Following the potential benefits of continuing professional development (Gaudreau et al. 2012; Mukamurera et al. 2019; Paquay et al. 2012; Uwamariya and Mukamurera 2005), it is possible that this willingness to take such risks was facilitated by their greater professional stability, effectiveness, and teaching knowledge. In addition, data analysis shows that these individuals have a positive view of their creative potential and were disposed to take this risk (Beghetto et al. 2021).

In terms of levels of creativity expressed, students went beyond the mini-c to express a little-c. The results of the analyses do not allow us to determine whether there were any surprising little-c, as it would have been necessary to interview the course instructors to know whether some students met the project process criteria in a surprising way (Beghetto 2019). The same applies to the pro-c, as a much broader investigation would be needed to find out whether some projects were recognized by experts in the educational field. Also, in wanting to improve their professional coherence, the beautiful risks involved seem compatible with the dynamic process of creativity (Botella and Lubart 2019, the project being always in motion for most of them. Therefore, it is possible that it will eventually evolve to a higher level of creativity. It should be remembered that teachers play an important role in establishing the necessary conditions in the classroom to allow for creative learning, while also serving as role models (Aschenbrener et al. 2010; Besançon and Lubart 2015; Capron Puozzo 2016a; Jeffrey and Craft 2004; Zhou 2003). Furthermore, the *Référentiel de compétences professionnelles pour la formation du personnel enseignant (Professional Competencies Framework for Teacher Education)* (Government of Quebec 2020) state that since the turn of the century, social expectations of the school and the teaching profession have multiplied, while the ecological, social, cultural, and economic environment in which new generations are born, grow up and develop has become more complex. Hence, cultivating creativity seems to be a valuable path.

Certain conditions, having contributed to the taking of beautiful risks and the overcoming of a mini-c, lead us to the words of Lucas (2001), who proposes four keys to foster creative learning:


Even though this professional master's program in preschool and elementary education is not focused on creativity, students naturally expressed their creativity to harmonize certain components of their professional coherence given the educational context that provided favourable conditions. It should be noted that Lubart et al.'s multivariate approach to creativity (Lubart et al. [2003] 2015) is intended to be systemic, entangled, we can see the contribution that emotional factors (e.g., welcoming emotions and using them for creativity), cognitive factors (e.g., setting goals, deepening one's knowledge), environmental factors (e.g., accompaniment, safe space, feedback), or other conative factors (e.g., uncertainty tolerance) can make beautiful risk taking. Since taking beautiful risks is about daring/caring for others, is this a purely conative factor or does the necessary empathy make it also an emotional factor?

It is worth recalling that reflective practice is at the heart of this master's program. This quote from Schön (1983) may lead to a parallel between creative and reflective postures:

the practitioner gives an artistic performance. He responds to the complexity, which confuses the student, in what seems like a simple, spontaneous way. His artistry is evident in his selective management of large amounts of information, his ability to spin out long lines of invention and inference, and his capacity to hold several ways of looking at things at once without disrupting the flow of inquiry. (p. 129)

Is there not an interesting parallel here with creativity process? The relevance to further this research project by pursuing the work of Capron Puozzo and Wentzel (2016) on reflexivity and creativity would certainly be of interest. Moreover, by linking them with the aim of improving teachers' professional coherence, perhaps they would find innovative ways to feel more well-being at work (Kenny et al. 2021).

One of the fields that constitutes Lubart (2017) 7 C's model is the importance of the collaboration in the creativity process. The main leverages for project development through the accompaniment and exchanges with the other students were significantly noted. The other C's of the model (Creator, Creating, Contexts, Creations, Consumption, and Curricula) can allow researchers who study some of these invariants to link them together for further research. This is another interesting way to consider for our future research.

Based on core qualities, the internal spheres of their onion model referred to beliefs and educational practices at the service of the child and their development (interest in neuroscience, concern for considering children in their entirety, adopting caring practices, forging an attachment bond with each child, considering their motivation, helping them to develop emotional, social, and interdisciplinary skills. . . ). All these positive elements and the risks associated with the implementation of the project seem to be pointing towards the proposal for positive creativity (Beghetto and Anderson 2022; Sternberg and Chowkase 2021). In addition to proposing creative devices and providing a framework conducive to creative learning, it would be a matter of teaching creativity as a skill, making explicit what beautiful risk-taking (Beghetto 2019) and positive creativity (Beghetto and Anderson 2022) are, to go even deeper and further.

As this is a purposeful sample, it is possible that these volunteers may have personality traits that are favourable to the research topic (e.g., openness to experience and risk-taking) which may have led to some bias. Since the study was conducted in a very specific training context, reproducibility remains low. Since the study was conducted in a very specific training context, reproducibility remains low. This limitation could be overcome by studying a whole group, or by opting for purposeful maximal sampling (Creswell [2006] 2013) to select more typical cases that might allow larger variations to emerge. Nevertheless, this study sheds new scientific light, as we did not find any research that combined beautiful risk-taking and professional coherence. If taking beautiful risks is

compatible with certain components of the teacher's professional world, then educational benefits may be possible for both the students/school environment (recipients of the beautiful risks) and the teacher. Other initial and continuing education contexts could also be considered for supplementary and broader research on professional coherence supported by a creative process.

**Author Contributions:** Conceptualization, M.-H.M.; methodology, M.-H.M.; validation, M.B. and I.C.P.; formal analysis, M.-H.M.; investigation, M.-H.M.; data curation, M.-H.M.; writing—original draft preparation, M.-H.M.; writing—review and editing, I.C.P., M.B. and M.-H.M.; visualization, M.-H.M.; supervision, M.B. and I.C.P.; references, M.-H.M., I.C.P. and M.B. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study has obtained prior approval of the Social Sciences and Education Ethic Committee of the Université de Sherbrooke (2021-2883).

**Informed Consent Statement:** Informed consent was obtained from all participants involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the first corresponding author and are not publicly available to ensure the privacy of the participants and to comply with the instructions of the ethic committee that approved this study.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Note**

<sup>1</sup> In Quebec (Canada), a Master of Education program is a continuing education program at the graduate level. People who register for it do so on a voluntary basis.

#### **References**


Beghetto, Ronald A. 2005. Does Assessment Kill Student Creativity? *The Educational Forum* 69: 254–63. [CrossRef]

Beghetto, Ronald A. 2010. Creativity in the Classroom. In *The Cambridge Handbook of Creativity*. Edited by Robert J. Sternberg and James C. Kaufman. Cambridge and New York: Cambridge University Press, pp. 447–66.


Beghetto, Ronald A., and Ross C. Anderson. 2022. Positive Creativity Is Principled Creativity. *Education Sciences* 12: 184. [CrossRef] Beghetto, Ronald A., and James C. Kaufman. 2013. Fundamentals of creativity. *Educational Leadership* 70: 10–15.


Botella, Marion, and Todd Lubart. 2019. From Dynamic Processes to a Dynamic Creative Process. In *Dynamic Perspectives on Creativity*. Edited by R. A. Beghetto and G. E. Corazza. Cham: Springer Nature, pp. 261–78.

Boutet, Marc, Manon Arsenault, Lise Ferland, Lise Francoeur, and Line Gagné. 2016. Le pari de l'accompagnement pour la formation de praticiens réflexifs de l'enseignement: Possibilités et limites d'un système de stage. *Approches Inductives* 3: 189–218. [CrossRef]

Boutet, Marc, Andrée Dasharnais, and Séréna Royer. 2021. Le temps de l'émancipation: Accompagner au fil de la carrière. In *L'archipel des temps de la formation*. Directed by Philippe Maubant. Nîmes: Champ social, pp. 51–80.

Capron Puozzo, Isabelle. 2016a. Du concept de créativité à une pédagogie de la créativité: Un défi pour le XXI siècle. In *La créativité en éducation et formation*. Directed by Isabelle Capron Puozzo. Louvain-la-Neuve: De Boeck Supérieur, pp. 13–29.

Capron Puozzo, Isabelle. 2016b. Créativité et apprentissage: Dilemme et harmonie. *Revue française de pédagogie* 197: 5–11. [CrossRef] Capron Puozzo, Isabelle, and Marion Botella. 2018. Emo-tissage et créativité. In *Emotissage. Les émotions dans l'apprentissage des*

*langues*. Directed by Françoise Berdal-Masuy. Louvain-la-Neuve: Presses Universitaire de Louvain, pp. 143–61. Available online: http://hdl.handle.net/20.500.12162/1981 (accessed on 30 July 2022).

Capron Puozzo, Isabelle, and Bernard Wentzel. 2016. Créativité et réflexivité: Vers une démarche innovante de formations des enseignants. *Revue française de pédagogie. Recherches en éducation* 197: 35–50. [CrossRef]

Cayirdag, Nur. 2017. Creativity Fostering Teaching: Impact of Creative Self-efficacy and Teacher Efficacy. *Educational Sciences: Theory & Practice* 17: 1959–75. [CrossRef]

Corazza, Giovanni Emanuele. 2016. Potential Originality and Effectiveness: The Dynamic Definition of Creativity. *Creativity Research Journal* 28: 258–67. [CrossRef]

Craft, Anna. 2005. *Creativity in School. Tension and Dilemmas*. New York: Routledge.

Creswell, John W. 2013. *Qualitative Inquiry and Research Design. Choosing among Five Approaches*, 3rd ed. Thousand Oaks: Sage, Original published in 2006.

Cropley, Arthur J. 2010. Creativity in the Classroom: The Dark Side. In *The Dark Side of Creativity*. Edited by David H. Cropley, Arthur J. Cropley, James C. Kaufman and Mark A. Runco. Cambridge and New York: Cambridge University Press, pp. 297–315.

Csikszentmihalyi, Mihaly. 1996. *Creativity. Flow and the Psychology of Discovery and Invention*. New York: Harper Collins.

Csikszentmihalyi, Mihaly. 2006. *La créativité. Psychologie de la découverte et de l'invention*. Paris: Robert Laffont.

Day, Christopher. 2004. *A Passion for Teaching*. New York: Routledge.

Dewey, John. 1947. *Expérience et éducation*. Paris: Bourrelier et cie.

Dewey, John. 1967. *L'école et l'enfant*. Neuchâtel: Delachaux et Niestlé.

Gagnon, Yves-C. 2011. *L'étude de cas comme méthode de recherche*, 2nd ed. Québec: Presses de l'Université du Québec, Ouvrage original paru en 2005.

Gaudet, Stéphanie, and Dominique Robert. 2018. *L'aventure de la recherche qualitative*. Ottawa: Les Presses de l'Université d'Ottawa.

Gaudreau, Nancy, Égide Royer, Éric Frenette, Claire Beaumont, and Tara Flanagan. 2012. Le sentiment d'efficacité personnelle des enseignants et leurs pratiques de gestion de classe et des comportements difficiles des élèves. *Revue canadienne de l'éducation* 35: 82–101. Available online: https://www.jstor.org/stable/canajeducrevucan.35.1.82 (accessed on 30 July 2022).


Guba, Egon G., and Yvonna S. Lincoln. 1982. Epistemological and Methodological Bases of Naturalistic Inquiry. *ECTJ* 30: 233–52. Available online: https://eric.ed.gov/?id=EJ276746 (accessed on 30 July 2022). [CrossRef]

Guillemette, François, and C. Gauthier. 2008. La pratique réflexive. Condition essentielle au développement des compétences en stage. In *L'accompagnement concerté des stagiaires en enseignement*. Directed by Marc Boutet, and Joanne Pharand. Québec: Presses de l'Université du Québec, pp. 127–52.

Hamel, Jacques. 1997. *Études de cas en sciences sociales*. Montréal and Paris: L'Harmattan.

Huret, Sandra. 2014. *Des étoiles dans la tête. Approche psychologique de la créativité chez les artistes et les scientifiques*. Saint-Denis: Connaissances et savoirs.

Jeffrey, Bob, and Anna Craft. 2004. Teaching creatively and teaching for creativity: Distinctions and relationships. *Educational Studies* 30: 77–87. [CrossRef]

Karwowski, Maciej. 2009. I'm creative, but Am I Creative? Similarities and Differences between Self-evaluated Small and Big-C Creativity in Poland. *The International Journal of Creativity & Problem Solving* 19: 7–26.

Kaufman, James C., and Ronald A. Beghetto. 2009. Beyond Big and Little: The Four C Model of Creativity. *Review of General Psychology* 13: 1–12. [CrossRef]

Kaufman, Scott Barry, and Carolyn Gregoire. 2016. *Wired to Create*. New York: Perigee books.

Kenny, Annie, Marie-Élaine Desmarais, and Geneviève Bergeron. 2021. L'engagement de l'apprenant et autres facteurs qui influencent le développement professionnel en éducation. In *Le développement professionnel en milieu éducatif: Des pratiques favorisant la réussite et le bien-être*. Directed by N. Gaudreau, N. S. Trépanier, and S. Daigle. Québec: Presses de l'Université du Québec, pp. 56–75.

Kolb, David A. 1984. *Experiential Learning: Experience as the Source of Learning and Development*. Upper Saddle River: Prentice Hall.

Korthagen, Fred A. J. 2004. In search of the essence of a good teacher: Towards a more holistic approach in teacher education. *Teaching and Teacher Education* 20: 77–97. [CrossRef]


Leplat, J. 2002. L'étude de cas à l'analyse de l'activité. *Perspectives interdisciplinaires sur le travail et la santé* 4: 1–31. [CrossRef]

Lewin, Kurt. 1951. *Field Theory in Social Sciences: Selected Theorical Paper*. New York: Harper and Row.


De Boeck Supérieur.


MDPI St. Alban-Anlage 66 4052 Basel Switzerland www.mdpi.com

*Journal of Intelligence* Editorial Office E-mail: jintelligence@mdpi.com www.mdpi.com/journal/jintelligence

Disclaimer/Publisher's Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Academic Open Access Publishing

mdpi.com ISBN 978-3-0365-9472-9