Characterizing Team Orientations and Academic Performance in Cooperative Project-Based Learning Environments

Abstract

Information technology professionals are required to possess both technical and professional skills while functioning in teams. Higher education institutions are promoting teamwork by engaging students in cooperative and project-based learning environments. We characterized teams based on their collective orientations and evaluated their team performance in a cooperative project-based learning environment situated in a sophomore-level systems analysis and design course. We explored the orientation patterns in terms of goals, roles, processes, and interpersonal relations (GRPI). Specifically, we analyzed team retrospectives of 23 teams using a mixed-method approach. Findings characterized teams into balanced and unbalanced orientations. Teams with balanced orientations demonstrated a higher level of team performance in terms of academic achievement than the unbalanced category.

1. Introduction

The interdisciplinary nature of information technology (IT) work requires graduates to possess a blend of technical and professional skills. Such skills include communication, collaboration, leadership, and problem-solving abilities [1]. Lecture-based modes of instruction often focus on the development of technical skills among the students. In recent years, however, traditional methods of instruction have undergone a continuous change, where the focus of learning has become more student-centric [2,3]. The transition of instructional methods from ‘instructivitist to constructivist’ approaches [4] (p. 640) provides learners with the opportunity to collaborate, interact, and construct new knowledge. Learning in groups or in teams is considered an important approach to helping students develop communication, collaboration, and teamwork skills [5]. However, a study by [6,7] revealed that working in teams or groups does not necessarily help students to develop teamwork skills. Prior studies (e.g., [7]) have demonstrated that students prefer to work individually rather than in teams even though they know that working in teams can improve their teamwork skills. Other studies have revealed that students have found team-based learning a positive experience when the instructor takes the initiative to form groups, follow up on the progress, and train the team on teamwork skills [8,9]. Therefore, it is essential for the instructors to design their course on known pedagogical approaches such as cooperative learning, project-based learning, or other similar approaches that encourage students to actively participate while they work in groups [9].

Specifically, research has identified that pedagogies such as cooperative learning in a project-based course help students develop teamwork skills such as goal setting, collaboration, communication [10]. Lau and colleagues’ study also revealed that working in groups helped the students overcome their competitive attitude and follow a cooperative approach. Further, Lau et al. emphasized the role of the instructor in providing teamwork training as crucial to help students master teamwork skills. While many studies have been conducted on the topic of teamwork, there is scarce research that characterizes specific teamwork skills students develop when they work in groups or teams [11]. Although cooperative and project-based learning pedagogies are popular in classroom settings, we still do not have an exhaustive list of skills that students develop when engaged in teamwork [11].

In order to understand the specific teamwork skills that students develop while they work in teams as they develop an IT software solution, we conducted a study where we followed 23 teams throughout an entire semester as part of a systems analysis and design class. The class implemented cooperative learning in a project-based learning environment, and we investigated how the team members oriented themselves in terms of their teamwork skills and identified the team’s academic performance. We posed three research questions to understand the specific teamwork skills that students reported they applied and characterized the collective orientation based on such reported teamwork skills. The study focused on four teamwork skills: setting goals, assigning roles, implementing coordination processes, and developing interpersonal relationships. Therefore, in the context of cooperative and project-based learning, the guiding research questions for this study are:

(1) What are student-reported implemented teamwork skills, such as setting goals, assigning roles, implementing coordination processes, and developing interpersonal relationships?

(2) What are collective orientations in terms of patterns associated with students’ reported teamwork skills?

(3) What is the relationship between collective orientations and team performance as evidenced by teams’ academic performance in each of the project milestones?

2. Literature Review

Teamwork has become an important 21st-century skill and, therefore, often implemented as a teaching pedagogy at the undergraduate level [12,13,14]. Working in teams allows the students to collaborate and cooperate with other members when engaged in problem-solving. Students need to develop teamwork skills, especially when they work in multi-disciplinary groups. As students work in groups, they learn collaboration that improves their motivation, persistence, and professional skills [13,15]. Teamwork skills refer to all qualities and abilities that enable individuals to work effectively with their peers when engaged in collaborative activities such as meetings or projects [16]. Teamwork skills are integral to success in today’s professional world [17], as stated by national associations. For instance, the Association of American Colleges and Universities (AAC&U) conducted a poll in 2009 and found that 71% of employers [18] reported “teamwork skills and the ability to collaborate with others in diverse group settings’’ [18], (p. 2), [19] (p. 53) as an important learning outcome. Due to the increased focus on teamwork skills, higher education institutions are constantly assessing how undergraduate students develop teamwork skills and how it helps to improve team effectiveness and team performance [20].

Salas et al. [21] determined the five core components of teamwork and referred to them as the “Big Five” of teamwork. These components are team leadership, mutual performance monitoring, backup behavior, adaptability, and team orientation. They suggested that the “Big Five” serves as a robust framework capturing the core components of teamwork and its effect on team performance and team effectiveness. Teamwork skills and team performance are closely related to collective orientation. Collective orientation in teams is defined as “the propensity to work in a collective manner in team settings” [22] (p. 317). They further suggested that collective orientation can predict team performance on a variety of team tasks. In addition, Salas et al. [23] highlighted the importance of team members working in a collective orientation, facilitating coordination and communication, therefore improving team performance. Collectivistic orientation in teams was found to be related to cooperative team behaviors [24]. These cooperative team behaviors are the keys to the relationship between team collectivistic orientation and team performance. Hagermann and Kluge [25] also demonstrated the importance of team processes such as transition, action, interpersonal, and learning processes in complex situations.

The increasing importance of teamwork has urged higher education institutions to help students develop teamwork skills that improve collective orientation and team performance. Working in teams or groups is recognized as one of the most important pedagogical interventions to improve student success [26,27]. Pedagogical approaches such as cooperative learning and project-based learning allow students to work in groups.

Prior to the 1950s, teamwork in an educational setting was not a popular concept, as competition rather than cooperation was common [28]. Johnson and Johnson introduced teamwork through cooperative learning in the later 1960s [29]. Cooperative learning has its grounding in the social interdependence theory by Koffka and Lewin [30]. Social interdependence theory posits that group members’ dependence on one another is needed to achieve a common goal. Vygotsky’s sociocultural theory also highlights the importance of social interaction among peers, which can result in the acquisition of new knowledge and academic success [30]. As teamwork is recognized as a 21st-century skill, practicing cooperative learning in educational settings can allow students to develop the ability to work with others. Working in groups in a cooperative learning environment promotes understanding and positive relationships with peers as students develop social skills such as active listening, sharing thoughts and resources, providing constructive feedback, taking responsibility, and working harmoniously in a group [31]. For instance, a study conducted by Phino-Lopes [32] in an undergraduate engineering class demonstrated that implementation of cooperative learning principles in the course (a) helped the students to understand the importance of teamwork, (b) allowed students to learn efficient time and work management, and (c) helped students to become active learners and involved in the construction of new knowledge along with their peers. Giraud [33] compared the impact of cooperative learning with the traditional lecture method of teaching, and the results of the study demonstrated that students in the section that used the cooperative learning method of instructions obtained significantly higher grades than students taught by the traditional lecture method.

As working on projects or enrolling in capstone courses are recognized as a high-impact educational practice across the United States [34], project-based learning is a common pedagogy used in undergraduate classrooms. Project-based learning (PBL) is a collaborative and interactive instructional approach that allows students to engage actively in the classroom [35,36]. Prior studies [37,38,39] have identified that PBL has proved to be an effective strategy for promoting teamwork skills in the classroom [40]. PBL motivates students to think critically and improves knowledge construction [41]. PBL allows the instructor and students to immerse themselves in a simulated environment to collaborate and solve problems creatively [38,42]. The PBL approach improves student engagement when they collaborate as a group. Student engagement helps them develop important employability skills such as; communication, teamwork, and time management [43]. Besides developing professional skills, project work improves technical skills such as designing, creating, and producing a system for the students [44]. Working on a project also delivers a real-world experience by exposing the students to the challenges that professionals face on the job [45,46]. The project’s socio-constructivist approach helps the students engage and work as a team to attain the desired learning objectives [47]. As current teaching pedagogies rely heavily on students collaborating as teams to work on projects, it is highly important to understand how teams develop and function efficiently [48,49,50].

Subsequently, prior studies have demonstrated that following cooperative and project-based learning approaches help students develop teamwork skills. Our study will take steps toward filling the gap by characterizing specific teamwork skills that students developed in a cooperative project-based learning course. Specifically, the study will identify teamwork skills such as setting goals, assigning roles, implementing coordination processes, and developing interpersonal relationships as they work together over a semester on a semi-capstone project.

3. Conceptual Framework

The study characterizes teamwork skills in terms of setting goals, assigning roles, implementing coordination processes, and developing interpersonal relationships. For this, we will use the lens of the GRPI model, also known as the goals, roles, process, and interpersonal relationships model [49,51,52,53,54]. The model proposes that setting goals, assigning roles, implementing coordination processes, and developing interpersonal relationships are essential teamwork skills that are integral to team effectiveness [53]. The team should have clearly defined goals by explicitly stating the main purpose of the team, major tasks to be performed, agreement on the desired results, boundaries, standards, and expectations. The next step for the team is to move forward and define the roles and responsibilities, including the acceptance of a team leader (if available), defining each team members’ roles, and sharing responsibilities. After defining goals and roles, the team then needs to implement coordination processes by delineating the project workflow, decision making, and mitigating conflicts. Lastly, teams must work towards helping the team members to develop interpersonal skills. Strong interpersonal skills lay the foundation to fortify interpersonal relationships among the team members. Interpersonal relationships include relating to the other team members, building trust, being sensitive and flexible to each other, having good communication skills, and effective methods for dealing with conflict [53]. The development of teamwork skills is crucial in determining collective orientation and team performance. Collective orientation allows the members of the team to work collectively towards a common goal, and when the team members work together towards a common objective, they deliver high performance [22]. For the purpose of this study, we defined collective team orientation as the team’s ability to efficiently use all four teamwork skills to attain the desired objective. Effective collective orientation was then related to team performance. The team performance consisted of the students’ academic performance in terms of grades obtained in the project milestones.

4. Research Design

The study utilized a concurrent triangulation mixed-method design [55] to characterize team orientations and their relationship with team academic performance in a project-based learning environment. Specifically, this study identified (a) teamwork skills as described by how students set goals, assigned roles, implemented coordination processes, and developed interpersonal relationships; (b) collective orientations in terms of patterns associated with the implemented teamwork skills; and (c) the relationship between collective orientations and team performance as evidenced by their performance in each of the milestones. As described by Kroll and Neri [56], in concurrent designs, qualitative and quantitative data are collected at the same time. We implemented a triangulation design as data from both methods were used to validate the findings.

4.1. Participants

This study focused on a sophomore-level semi-capstone system analysis and design course offered in the Fall 2019 semester with a total population of 113 students. Most students were in their second year of college education and were either pursuing a computer and information technology major or minor. The demographic information of the class, acquired from institutional databases, is presented in

Table 1. Gender and Academic Level.

Male	Female	First-Year	Second-Year	Third-Year	Fourth-Year
18	95	0	40	41	32

and

Table 2. Ethnicity and Race.

Two or More Races	American Indian or Alaska Native	Asian	Black or African American	Hispanic or Latino	International	White	Unknown
2	0	13	5	7	25	59	2

.

Table 1 depicts gender and academic level demographic information. The class had 16% female students and 84% male students enrolled for Fall 2019. This also reveals that student teams comprised majorly male students. Table 2 depicts ethnicity and race demographic information. Even though 52% of the students were white, 48% percent represented other ethnicities and races.

All students enrolled in this course were required to complete an introductory systems development course as a prerequisite. They would have had experience with programming through coursework or practical experience. These same students were required to take a design thinking in technology course in their first year, where they developed some experience working in teams. Students were organized into teams consisting of four or five members, with a total of twenty-three teams.

4.2. Learning Design and Context

The learning environment of the system analysis and design course was based on the five principles of cooperative learning: (i) positive interdependence; (ii) individual and group accountability; (iii) interpersonal and small group skills; (iv) face-to-face promotive interaction; (v) group processing; to implement active learning and project-based learning [57]. Implementing these principles aims to help students develop teamwork skills such as setting goals, role allocation, interpersonal communication, and process definition.

The course implemented a scrum approach towards project-based learning. Scrum is a framework that enables developers to share knowledge to address complex problems, all the while delivering high-value products [58,59]. The goal was to develop the final solution across multiple iterations, with each iteration building incrementally upon previous iterations. The project guided students in applying their modeling knowledge after determining a system’s requirements and developing a prototype. The project, consisting of the system’s documentation and the corresponding prototype, was organized into four milestones. The milestones and prototype were evaluated using rubrics that assessed the accuracy and how complete the solutions were. Students utilized the class hours to work on the project and sought feedback from the instructional team. Teams consisting of four or five students were provided with the autonomy to decide meeting times and working styles outside of class. The interpersonal and group skills were promoted by encouraging students to work on project deliverables during class time.

Positive interdependence was accomplished by dividing the project into milestones and clearly delineating the deliverables for each milestone. The teams were encouraged to set goals and a clear plan for the deliverables that would be included in each project milestone. Accountability at individual and group levels was established by setting expectations that the students were to work together through all milestones while still being responsible for individual deliverables. Student teams were encouraged to assign responsibilities based on the strengths and preferences of team members. Approximately a third of the project was graded individually. Furthermore, self and peer evaluation surveys were issued at the end of the semester for students to rate themselves and their peers. This score also contributed to their individual grades. Face-to-face promotive interaction was facilitated by the structure of the term project, where student team members would become proficient in a specific aspect of the prototype that they implemented. However, to complete the prototype, student team members must have been aware of the system’s overall functionality and how their individual modules must interact to achieve said functionality. Group processing was promoted by incorporating team retrospectives as part of the milestone submission. The retrospective required students to reflect on the positive aspects of each milestone as well as the challenges they faced and ways to overcome the same for the next milestone.

4.3. Pedagogy

Each class session was structured such that students can work in teams to apply newly acquired conceptual knowledge while getting feedback from the instructional team. The course instructor would provide walkthroughs of various approaches that can be taken to design and develop the different milestone deliverables through the use of mini case studies which describe a specific functionality of an information system [57].

Student teams were required to rotate the role of team leader, also referred to as Scrum master, for each milestone of the project. Scrum teams are self-managed, and team members work cooperatively to achieve the project goals [59]. As such, the instructional team—consisting of the lead instructor and teaching assistants—did not influence or direct student team decisions as to who served as the leader for each milestone. The team leader was responsible for facilitating communication over the course of each milestone in terms of defining the collective goals that the team wants to achieve and the exact tasks to be performed by each team member. The team leaders could take more active or passive approaches to leadership depending on specific team dynamics and individual tendencies [51]. The instructional team would actively monitor group performance and provide feedback on deliverables. The instructional team would also facilitate conflict resolution sessions for teams as required.

4.4. Procedures

For this study, we investigated all twenty-three teams from the course. Each team was assigned a pseudo-name in order to ensure the privacy of the students. Approval from the Institutional Review Board was obtained prior to conducting the study. Team retrospectives that were submitted along with the four times throughout the semester were analyzed to determine the collective orientations. Additionally, the study explored the relationship between academic performance given by their milestone grades.

4.5. Team Retrospectives

The semester-long project was divided into four milestones and one final prototype. Each team was required to submit a team retrospective post-completion of each milestone. The team retrospectives consisted of written reflections that revealed each team’s goals, roles, processes, and interpersonal relations for every milestone.

Table 3. Alignment between GRPI themes and team retrospective questions (adopted from [51]).

Theme	Definition	Team Retrospective Questions
Goal	Explanation of the planning or overall vision for the current milestone.	How did you plan the organization of work for the milestone?
Roles	All team members must know what part they play, what is expected, and how they are held accountable and responsible.	What were the team members’ roles? How were activities assigned to each team member, and what was the justification for that?
Processes	Explanation about procedures that the team has to follow, in terms of workflow or review, for current milestones or improvements to be made for future milestones	What are areas or sections of the milestone that you just completed you think could be improved? What are the aspects you think can be done better for the next milestone in terms of team performance? What are the possible concerns? What do you think as a team was particularly good about the milestone you just completed?
Interpersonal Relationships	Explanation about the quality of communication and collaboration among team members; any reference to communication platform; team participation; conflict management and resolution.	How was the communication handled among team members? What aspects of the team coordination/collaboration went well in this milestone? What aspects of the team coordination/collaboration went wrong in this milestone?

illustrates the questions related to all the four sub-constructs of team effectiveness, the corresponding definition for each of the themes, and the alignment between the GRPI themes and the team retrospective questions [51].

4.6. Data Analysis Methods

To address our first research question, what are teamwork skills as described by how students set goals, assigned roles, implemented coordination processes, and developed interpersonal relationships? We used a qualitative research method known as apriori thematic analysis [60]. The apriori thematic analysis intends to obtain an in-depth knowledge of how teams made the strategic decision in terms of setting goals, assigned roles, implemented coordination processes, and developed interpersonal interactions when they worked on each milestone. Rubrics were developed to score each code per team (see

Table 4. Rubric representing the definition and scores for each level (adapted from [51]).

Criterion	0	1	2
Goal	Did not address the overall plan for the current milestone in terms of goals and/or organization	Addressed goals and organization of the team in an insufficient manner	Comprehensively addressed the goals and organization of the team
Roles	Did not delineate the roles and responsibilities of team members	Vaguely defined the roles and responsibilities of some team members or did so for all team members but was lacking clarity	Explicitly delineated roles and responsibilities of every team member
Processes	No detailed explanation for procedures the team has to follow	Vaguely defined procedures for the team to follow	Explicitly defined procedures for the team to follow
Interpersonal Relationships	Exhibited poor quality of communication and collaboration	Exhibited moderate quality of communication and collaboration	Exhibited excellent quality of communication and collaboration

, [51]). As the data were qualitative in nature, two researchers manually coded it. They followed these steps to perform the analysis: (1) in the first step, a researcher independently coded the data available in the form of team retrospectives using the rubrics described in Table 4; (2) in the next step, a second researcher coded twenty percent of the retrospectives; (3) subsequently, the two researchers met and discussed their findings and resolved any discrepancies; (4) finally, based on the discussion in step 3, the researchers re-coded the data and results were reported. Cohen’s Kappa coefficient was calculated to ensure interrater reliability. Cohen’s Kappa coefficient was calculated to be 0.673. The Cohen Kappa value suggests that the raters agreed on the coding [61]. The maximum mean score for team collaboration effectiveness that a team can obtain was 8 (provided the team obtained a perfect score of 2 for each construct in all the four milestones), and the minimum mean score a team could obtain was 0.

Further, to answer the first research question, we categorized the average scores obtained by each team for each teamwork skill (goals, roles, process, and interpersonal relations) into three levels of teamwork skill formation: developing (0–2.67), emerging (2.68–5.37), proficient (5.38–8.00). Further, we created a contingency Table 3, representing the percentage of teams in each level. To answer the second research question, what are collective orientations in terms of patterns associated with the implemented teamwork skills? The data were further analyzed using clustering to determine the team orientation types. Clustering is an unsupervised machine learning technique used for grouping data. We used Ward hierarchical clustering to group the teams based on the orientation types. Clustering algorithms are commonly used techniques for educational data mining [62,63,64,65,66,67]. Ward hierarchical clustering is also appropriate when the sample size is small [62,64]. Prior to performing clustering, it is crucial to identify the optimum number of clusters; the study used the elbow method to determine the optimum number of clusters [68]; for our study, the elbow method determined the optimum number of clusters as two, see the graph in Appendix A Figure A1; therefore, based on the results of the elbow curve all the 23 teams were grouped into two clusters.

Further, the Mann–Whitney U test was used to determine if there was a significant difference between the cluster orientations. The rationale for using the Mann–Whitney U test is the small sample size (n < 30) and the ordinal nature of the data [69,70].

To answer the third research question, what is the relationship between teamwork orientations and team performance as evidenced by the team’s academic performance in each of the milestones? Logistic regression was performed. Logistic regression is a statistical technique applied when the dependent variables are categorical [71]. Logistic regression provides the predicted increase or decrease in the probability of possessing a characteristic based on the unit change in an independent variable while other independent variables are held constant. A higher predicted value indicates that it is “…more likely that any individual with particular scores on the independent variables will have a characteristic…” [68], (p. 1). For this study, student teams were categorized as high-performing or moderate-performing based on their total milestone grades. Student teams that scored above the 50th percentile were classified as high performing, and those that scored below the 50th percentile were classified as moderate performing. Student teams had an average total score of 93.22 with a standard deviation of 6.03. Logistic regression was computed to determine the relationship between student team performance and the quality of their reflections of goals, roles, interpersonal relations, and processes. The logistic regression output provided the probability of a student team being high performing in response to unit increases in the factors of goals, roles, interpersonal communication, and processes.

5. Results

The results are organized and presented in terms of the three research questions. The first section, teamwork skills, presents the descriptive analysis results that characterized the reported teamwork skills. The second section, collective orientations, presents the clustering analysis that grouped teams based on their reported teamwork skills. The third section describes the relationship between team orientations and team academic performance.

5.1. Reported Teamwork Skills

This section presents the results that focused on characterizing the teamwork skills as described by how students set goals, assigned roles, implemented coordination processes, and developed interpersonal relationships.

Table 5. Distribution of the four teamwork skills and levels of teamwork skill formation.

Teamwork Skills	Developing	Emerging	Proficient
Setting Goals	26%	44%	30%
Assigning Roles	13%	35%	52%
Implementing coordination Process	4%	13%	83%
Establishing Interpersonal relations	4%	22%	74%

presents the distribution of the four teamwork skills: goals, roles, interpersonal relations, and processes under the three levels of teamwork skill formation, referred to as developing, emergent, and proficient. The results of Table 5 demonstrate that around 26% of teams lacked the skills to set goals, whereas only 30% of teams demonstrated proficiency in establishing goals. With respect to assigning roles, it was observed that around 52% of teams demonstrated proficiency in assigning roles, but around 48% of teams were in developing or emerging categories. For establishing interpersonal relations and implementation of coordination processes, it was observed that the majority of teams were in the proficient category. The overall interpretation of the results from Table 5 reveals that the majority of teams lacked proficiency in establishing goals and assigning roles, whereas teams were proficient in establishing interpersonal relations and implementing group processes.

5.2. Collective Orientations

This section presents the results focused on identifying the collective orientations in terms of patterns associated with the implemented teamwork skills. The rubric scores [51] were obtained for each construct, and each team was used to perform the clustering. As the optimum number of clusters that could be generated from the data was two, we divided the results of the clusters into two orientation types.

Cluster 1—Balanced Orientation Team: Focused on at least three goals, roles, interpersonal relations, and processes in their reflections and must have obtained a mean score of 5 or above in each construct.

Cluster 2—Unbalanced Orientation Team: Focused at most on two of goals, roles, interpersonal relations, and processes in their reflections and must have obtained a mean score of 5 or above in each construct.

Table 6. Represents the team orientations for the two clusters.

Cluster 1 (Balanced Orientation)		Cluster 2 (Unbalanced Orientation)
Teams	Orientation	Teams	Orientation
V	GRPI	K	I
B	GRPI	O	I
R	GRPI	U	I
H	RPI	C	I, P
Q	RPI	N	I, P
E	RPI	J	I, P
S	RPI	M	I, P
		G	P
		L	I
		P	P
		W	P
		A	G
		F	G, R
		D	R
		I	NO ORIENTATION
		T	NO ORIENTATION

represents the teams in the balanced and unbalanced orientation clusters and their basis of orientation. Table 6 demonstrates that teams in the balanced orientation category were oriented for all four constructs or at least three constructs, whereas unbalanced orientation demonstrates that teams were more oriented towards at most two constructs. It is also important to note that unbalanced teams were more focused on interpersonal relations and processes while being less focused on the goals and roles. Two teams demonstrated no orientation whatsoever as they scored less than five for all four constructs.

Table 7. Mean and standard deviation for balanced and unbalanced team orientations.

Balanced Team Orientation (n = 7)		Un-Balanced Team Orientation (n = 16)
Mean	SD	Mean	SD
7.60	0.40	5.03	1.32

represents the mean and spread for balanced and unbalanced team orientations. It is important to note that the balanced teams demonstrated a balance of the application of teamwork skills, whereas the unbalanced teams demonstrated an unbalance of the application of teamwork skills.

The quote from team H, from the balanced orientation category, demonstrates team task planning and task execution.

“Each of us was assigned to finish our own part. For [the] package diagram, we need to modify the class diagram that we made before. The package diagram is a little bit complicated, we assigned 1 person to finish it. As for the other material for the milestone, we split the tasks up by one person. Some of the more difficult tasks had 2 people completing them. At the end of the project for milestone 4, we finally worked well, we knew the strengths and weaknesses of our groups and implemented tactics to mitigate those weaknesses. We will separate the mission much more carefully and let each team member do their skilled part. Working as a group, we can learn a lot”.

The quote above exemplifies how the team members actively participated in completing the milestone. The quote also illustrates elements of planning, efficient role allocation, collaboration, interaction, and group processes. For example, Team U demonstrated very shallow planning for goals and roles: in contrast, the teams in the unbalanced category demonstrated a moderate level of use of teamwork skills. It was also observed that the teams in the unbalanced orientation mostly focused on one or two teamwork skills. That is, most teams in the unbalanced category focused on interpersonal relations or processes and less on deciding the roles and goals.

“In our planning stage, we divided the work among our members. Two members worked on the package, one worked on Gantt and update backlog, one work on the summary.”

However, the team members in Team U focused more on the aspects of collaboration and workflow through interpersonal relations and group processes.

“We should keep in contact with each other after the work assignment constantly so we can make sure everything goes on well as we expect and there is nothing left for the whole project. Assign some work to every team member after each meeting can improve efficiency. Plan ahead (a week before) the deadline.”

This quote demonstrates that team members in Team U were ready to collaborate and work on the project, but lack of goal setting and role assignment resulted in ineffective collective orientation and low team performance.

After performing the Mann–Whitney U test, as shown in Table 4, it was found that Cluster 1 (balanced orientation) (M = 7.61, SD = 0.40) demonstrated a significantly higher level of application of teamwork skills (U = 0, z = −3.708, p = 0.0002) than Cluster 2 (unbalanced team orientation) (M = 5.03, SD = 1.32). U equations can be understood as the number of times observations in one sample preceded or followed observations in the other sample when all the scores from one group are placed in ascending order [70]. Therefore, the U statistic value of 0 indicates that all the teams in the balanced orientation demonstrated a higher level of use of teamwork skills than unbalanced orientation teams [70].

5.3. Relationship between Team Orientations and Team Performance

This section presents the results that describe the relationship between collective orientations and team performance as evidenced by teams’ academic performance in each of the milestones. The average teamwork skills scores in terms of goals, roles, interpersonal communication, and processes were the independent variables. The Hosmer and Lemeshow test was used to test the goodness-of-fit of the proposed regression model [72]. The null hypothesis for the Hosmer and Lemeshow test is that the regression model is correctly specified and fits the data well. The test did not reveal any statistically significant results (χ² = 8.238, p = 0.411), which suggests that the data fit the model. The results of the logistic regression are shown in

Table 8. Logistic regression results.

	B	SE.	df	Exp(B)
Goals	1.811	1.677	1	6.117
Roles	0.591	0.984	1	1.806
Interpersonal Communication	−1.09	1.673	1	0.336
Process	−1.271	2.041	1	0.281

, and it suggests the following:

• A unit increase in goals, with all other factors, held constant, increases the odds of team-high performance by a factor of 6.117.
• A unit increase in roles, with all other factors, held constant, increases the odds of team-high performance by a factor of 1.806.
• A unit increase in interpersonal communication, with all other factors, held constant, decreases the odds of the team-high performance by a factor of 0.336.
• A unit increase in process, with all other factors, held constant, decreases the odds of the team-high performance by a factor of 0.281.

The logistic regression results suggest that the quality of reflections in terms of goals and roles was most important in determining team high performance. The results also indicate that teams focusing exclusively on the quality of reflection of interpersonal communication and goals are likely to negatively affect their level of performance.

6. Discussion and Implications

The study investigated the application of teamwork skills by student teams in a cooperative project-based learning environment. Further, the study also investigated the relationship between team collective orientation and team performance in terms of academic achievement. Specifically, we characterized teamwork skills based on students’ reflections on setting goals, assigning roles, implementing coordination processes, and developing interpersonal relationships. The study’s initial results demonstrate that the majority of teams lacked teamwork skills such as setting goals and assigning roles appropriately. However, the teams were able to establish interpersonal relationships and implement coordinated processes. These results can be rationalized based on the Tuckman model. The Tuckman model [73] emphasizes that the longer the team members work together, they develop strong interpersonal relationships and processes. Therefore, all the teams demonstrated a high proficiency for these specific teamwork skills, as they were engaged in a semester-long project. To take a deeper dive into how the teams function, we explored the collective orientation of the teams. The study used a clustering algorithm to categorize the teams into two types of collective orientations: balanced-orientation and unbalanced-orientation. The results of the study indicate that teams in the balanced orientation demonstrated a significantly high level of teamwork skills. The justification for the high level of teamwork skills can be attributed to the team members’ efforts to establish the right set of goals and roles and focus on group processes and interpersonal relationships. The results of the balanced-orientation team can be supported by the underlying principles of the GRPI model, where teams with well-defined goals, roles, processes, and interpersonal relationships are successful and effective [52,53]. Further, the results align well with the existing studies, such as in [49,51], i.e., that efficient use of teamwork skills such as goals, roles, interpersonal relations, and processes in a sequential order can lead to higher team effectiveness. Both the studies used the GRPI framework to investigate team effectiveness and team development.

The teams in the balanced-orientation category also illustrated their adherence to the principles of cooperative learning. Specifically, the literature on cooperative learning mentions that students develop teamwork skills as they work in groups, allowing the students to collaborate, communicate, provide feedback and learn from one another [29,31,32,57].

Table 9. Cooperative learning principle, definition, and sample of student quotes.

Principle	Definition [28,57]	Student Quotes
Positive interdependence	The group has a clearly defined plan for accomplishing tasks or goals	“The team planned for this project by communicating over what needed to be done during class time and using GroupMe for anything they did not mention in class. They made clear who was responsible for completing which parts of the milestone so that everyone was clear on exactly what their job was. People volunteered for their respective parts, and no one had to be forced to do a part of the milestone that no one wanted to do.”
Individual and group accountability	The group holds members accountable for an individual share of work as well as overall group goals.	“No one had a greater share of the work on this milestone than the other group members. The roles of each group member were as follows: Student 1—4 Sequence Diagrams 2 Activity Diagrams and the team retrospective. Student 2—4 Sequence Diagrams 2 Activity Diagrams and the class diagram. Student 3–4 Sequence Diagrams 2 Activity Diagrams. Student 4—4 Sequence Diagrams 2 Activity Diagrams and the Gantt chart. Student 5—4 Sequence Diagrams 2 Activity Diagrams”
Interpersonal and small group skills	Effectiveness in teamwork skills such as communication, conflict management, leadership, decision making, and trust-building.	“The aspects of team collaboration that went right during this project were our work ethic and communication. Everyone was done with their parts 24 h before we had to submit which gave us plenty of time to make needed corrections to the diagrams. The team divided work evenly and completed it efficiently. The only aspect of the milestone that needed work was communicating about how the diagrams were being made, but due to the excess time that was a non-issue as all errors were quickly corrected.”
Face-to-face promotive interaction	The students work together as a group, help each other to achieve success by sharing resources and providing feedback.	“This team did very well on this milestone. The team communicated with one another and had a solid outline for who would do which parts of the project well before the deadline, as everyone knew exactly what it was that they needed to get done. The project was completed several hours in advance of the deadline, which allowed us to review our work before submission.”
Group processing	Teams collectively decide as to which behaviors are acceptable and which behaviors need to change.	“While communication has continually improved over the course of this project, the team could still do better as several corrections were needed on diagrams due to there not being much communication on what each member was doing on the ones they had been assigned. Communication during work needs to be improved.”

shows the alignment of balanced team orientation and cooperative principles, as evidenced by quotes from Team R.

The quotes from Table 9 demonstrate the team members actively participated in completing the milestone. They also illustrated confidence in planning, efficient role allocation, collaboration, interaction, and group processes. In contrast, the unbalanced-orientation teams could not adhere to all the five principles of cooperative learning as they lacked the ability to develop the required teamwork skills to function as a team [22,25]. Moreover, the teams in the unbalanced category demonstrated a moderate level of team collaboration effectiveness, as they focused on one or two aspects of team collaboration effectiveness. Most of the teams in the unbalanced categories focused on defining interpersonal relationships or processes and less on determining the roles and goals.

The logistic regression was used to determine the relationship between the quality of reflections and overall team performance across the project milestones. The logistic regression model confirmed that effectively defining the goals for each milestone and appropriately allocating roles were crucial to a team achieving high performance. In contrast, the teams that focused more on the quality of interpersonal communication and processes were less likely to achieve high performance. The result of the logistic regression aligns with the GRPI model of team collaboration effectiveness, as the model prioritizes the determination of goals and roles over processes and interpersonal relationships [53]. The logistic regression model also corroborates our findings in Table 3, where only 30% of the teams were proficient at setting goals, and just over half the teams were proficient in allocating roles. Based on these results, teams not having effective interpersonal communication or processes can still achieve high performance if they effectively define goals for each milestone and appropriately allocate roles to team members.

Our results align with prior studies that have demonstrated the effectiveness of cooperative and project-based learning approaches in helping students develop teamwork skills. The results from our study add to the existing body of literature by characterizing specific skills that students developed. Specifically, setting goals, assigning roles, implementing coordination processes, and developing interpersonal relationships. Furthermore, the study revealed how pedagogy plays an important role in the development of these skills and subsequent team orientations.

The results of the study have implications for teaching and learning regarding practices instructors can adopt when they engage students in work in groups or teams. First, it can be inferred from the results that the instructors play a critical role in orchestrating teamwork. Teamwork orchestration needs to be considered at the instructional design level. The learning environment as a whole must be conducive to the development of teamwork skills. For this, cooperative learning and project-based learning have been identified as effective pedagogies for coordinating teamwork at the design level [9]. Such pedagogies provide guidelines in terms of: (1) phases to implement as students solve a challenge or problem and (2) guidelines to coordinate teams [38,42]. A holistic learning environment that implements the tenets of cooperative learning can result in students developing teamwork skills and teams exhibiting a balanced orientation. Second, the results also indicate that in addition to orchestrating teamwork, instructors need to facilitate and monitor it throughout the implementation of the project. For instance, instructors can facilitate teamwork by taking the initiative to form groups and by following up on the progress [8,9]. Instructors can also facilitate teamwork by using team contracts at the start of the project. Team contracts can also help students define their goals and establish roles effectively, which simultaneously improves team commitment [74,75]. Finally, equally critical is for instructors to provide teamwork training. For instance, instructors can provide training to help students to prioritize setting goals and assigning roles to promote team performance and team effectiveness [76,77]. Exercises portraying effective goal definition and role allocation based on team-member skill sets could also help student teams attain high performance in terms of academic achievement. Instructors can also provide explicit training that consists of providing students with strategies to manage and solve conflict [78,79]. This, in turn, can improve interpersonal communication and processes employed by the teams.

7. Conclusions, Limitations, and Future Work

The study characterized teamwork skills in terms of teams setting goals, assigning roles, implementing coordination processes, and developing interpersonal relationships. The instructor was broadly successful in implementing the tenets of cooperative learning, as evidenced by the broadly proficient scores of reflections in terms of interpersonal relationships and implementing coordination processes. However, the student teams were only moderately proficient at assigning roles and defining goals. This led to teams being categorized into balanced and unbalanced collective orientations according to the ways they described their implementation of teamwork skills. The teams in the balanced-orientation category demonstrated higher team collaboration effectiveness and high quality of reflections. In contrast, teams in the unbalanced-orientation category demonstrated a moderate level of collaboration effectiveness and moderate quality of reflections. To achieve a balanced orientation, a team must focus on being proficient with at least three out of the four described teamwork skills. The results also revealed that teams whose reflections focused on setting goals proficiently and allocating roles appropriately were more likely to achieve high performance compared to teams that focused on interpersonal communication and processes. It is important to note that simply assigning students to teams without providing any support or guidelines from the instructor will not necessarily result in effective teamwork [9].

The primary limitation of our study is that only team retrospectives were used to analyze the teamwork skills that teams develop over a semester working in a cooperative project-based learning environment. Conducting interviews or team observations with the team members could provide deeper insights into how teams determined their goals, roles, processes, and interpersonal relationships. Additionally, the sample size for this study was not particularly large, and team demographics were not available for the study; therefore, we mentioned the class demographics. Future work can involve working with a larger sample size to understand the team members’ strategic decision-making process in a classroom environment using the GRPI approach. Additionally, we plan to investigate the changes in the team dynamics and decision-making after the course went online due to the COVID-19 pandemic. Further investigations could be conducted regarding the effect of HyFlex learning environments on team orientation.