Collaboration Skills and Puzzles: Development of a Performance-Based Assessment—Results from 12 Primary Schools in Greece

Abstract

Collaboration skills are essential for the effectiveness and efficiency of collective efforts in both professional and personal contexts. However, their complex and intertwined nature poses challenges in both defining and assessing them. To develop educational methodologies aimed at enhancing the cultivation of collaboration skills, there is a need for developing pre-post experimental assessment tools that target the construct in real-life school settings. Research indicates a lack of performance-based assessment tools designed to assess collaboration skills. This research proposes a performance-based assessment developed through collecting evidence from individuals participating in both solo and group puzzle activities. The assessment is created by viewing collaboration skills through the lens of the ATC21S framework. A total of 148 students aged from 11 to 12 years old from 12 different public schools in Greece participated in this study, carried out over a period of 7 months between December 2023 and June 2024 in both the test and retest phases. The results show that, regardless of the group compositions, individual student collaborative performance was stable. The validity of using Spearman’s correlation coefficient was measured at 0.623.

1. Introduction

Among the many educational challenges at the dawn of the 21st century is the cultivation and development of collaboration skills in students. Although collaboration is often characterized as a vague and abstract term, the lack of this skill frequently results in poor performance when individuals face complex problems that require collective efforts, such as financial crises, global pandemics, and natural disasters. The absence of effective collaboration can even cost lives. Therefore, the need to develop collaboration skills from an early age is evident due to its interdisciplinary impact on many aspects of personal and professional life.

Adapting puzzles to evaluate collaboration skills presents significant challenges. Several key factors influenced the decision to utilize puzzles. Firstly, there was a need for an instrument that captures relatively stable individual problem-solving abilities, devoid of the confounding effects of hierarchical knowledge from standard curricula. Secondly, the instrument had to be collaboratively integrated, maintaining consistent difficulty while allowing individual components to be traceable. Thirdly, the procedure required standardization and replication across numerous schools under stringent regulatory constraints. Fourthly, the assessment aimed to gamify a typically unenjoyable process for students, with its covert nature providing “stealth” assessment characteristics [1], making students perceive it as gameplay rather than evaluation. Lastly, if deemed suitable, the instrument should be user-friendly, cost-effective and simple in its implementation for regular primary school educators. The instrument’s usability could serve both research purposes as a pre-post assessment tool and as a regular evaluation mechanism for teaching programs targeting the development of collaboration skills at students aged 11–12.

The proposed approach (CSB Test 1) utilizes all the elements that can potentially create a solid ground for its implementation in typical school settings. The use of puzzles in both the individual and collaborative phase paired with regulations that allow for individual grading to be possible within a collaborative process sheds more light on the challenging task of assessing collaboration skills in individual students.

The primary research question of our proposed approach is as follows:

RQ: Is it feasible to measure the individual collaboration skills performance of 11–12-year-old students using the “Collaboration Skills Benchmarking Test 1” (CSB Test 1) assessment tool?

The proposed methodology (CSB Test 1) was tested and retested from December 2023 to June 2024 in 12 public schools in Greece. This study presents the outcomes of experiments conducted within typical school environments. The instrument’s validity, measured using Spearman correlation coefficient, is 0.62. Consequently, the application of CSB Test 1 provides enhanced insights into the assessment of collaboration skills.

This paper is structured as follows:

Section Relevant Literature outlines the relevant literature, while Section 2 elucidates the theoretical foundations of our proposed approach, accompanied by detailed explanations of the computational methodologies employed. Section 3 and Section 4 outline the experimental procedures and results. Section 5 summarizes the contributions of this study, delineates future research directions, and discusses the implications of this work for assessing collaboration skills through gamified puzzle activities.

Relevant Literature

The global research community is developing various methodologies [2,3,4,5] to enhance collaboration skills in students of various ages. However, there is significant difficulty [6,7,8,9,10,11,12,13] in assessing the effectiveness of such interventions due to the construct’s complexity. Consequently, the first major issue to address is how the operationalization of collaboration is defined.

According to Evans [9], there are seven frameworks (ATC21S, Cambridge Assessment, Pearson P21, Essential Skills & Dispositions, PISA, P21 EdLeader21, and ETS) from which researchers can derive guidance in constructing assessments for evaluating students’ collaboration skills. The most widely used frameworks [12] are the Assessment and Teaching of 21st Century Skills [13] and the PISA [14] assessment. Although most frameworks view collaboration skills similarly, the ATC21S framework has a more strictly defined approach to analyzing the collaborative aspect of collaborative problem-solving. Therefore, this research adopts the ATC21S approach for constructing the assessment instrument, while the guidelines of Child and Shaw [15] provide vital insights for grading based on reflective, meaningful actions during the collaborative process.

The choice of assessment type significantly impacts capturing the desired elements for research. Self, peer, and teacher reports are extensively used in assessing collaboration skills and related skills such as perseverance and self-regulation. Out of 16 validated and publicly available instruments in the U.S. identified by Cox, Foster, and Bamat [16], 14 were student self-reports, 1 was a teacher report, and 1 was task-based. From the self-report assessments, one was used in students aged between 9 and 15 [17] in a summer sports-based program and three [18,19,20] were compatible to 11- to 12-year-old students, within typical school settings.

While self-reports provide important information, they are often considered deficient for assessing collaboration skills [21]. Other types of assessments include global rating scales [22,23], standardized assessments [24,25], observational measures [26,27], and performance assessments. These methods offer significant benefits, but focusing on what students express through their actions is crucial. This does not minimize the teacher’s role, which remains valuable, but the focus should remain on students’ actions.

In this context, there is a research trend toward designing assessments based on evidence [28]. These Evidence-Centered Design (ECD) assessments track students’ actions within collaborative activities. Simulated environments often require students to perform tasks like pressing mouse buttons or writing scripts, and these logs are used to evaluate their collaboration skills [12]. A promising assessment tool created by Rojas et al. [10] following the PISA framework aimed to assess collaborative problem-solving skills. However, the conclusions state that the social component of CPS was only partially measured, while its reliance on text-based communication resulted in the loss of rich information that would otherwise be expressed through non-verbal means. Additionally, implementing such technologically demanding assessments in regular public school settings can be challenging.

The collection of evidence should primarily focus on capturing individual problem-solving attributes that are directly comparable to collaborative problem-solving, thereby elucidating their differences. Furthermore, the learning effect is inevitable; hence, it should be minimized or at least quantifiable. Puzzles have demonstrated their utility in various scientific research domains, ranging from medical diagnostics and assessments [29,30,31] to algorithm teaching [32]. Recently, there has been a growing body of research integrating puzzles into educational settings as evaluation support tools [33,34] and as instruments for assessing computational thinking skills [35].

2. The Proposed Methodology

The main focus of CSB Test 1 (Collaboration Skills Benchmarking Test 1) was to create an assessment tool that measures the student’s actual collaborative performance, before and after a didactical intervention. In simple words, the test aimed to understand whether the student actually expresses (if any) transferable or generic collaboration skills [9] in a context/task that is not related to the subjects taught from the didactical intervention.

The element of a task that is not related to the subjects taught was derived from the notion that the students should enter the assessment without the established classroom grading or subject knowledge hierarchies [3] in their mind. Therefore, it is less likely that students will assume knowledge and force their will on their group mates, which directly increases the likelihood of negotiation [15]. In this way, all the students would face the assessment with a sense of having a clean slate to start with.

Additionally, the selection of puzzles is very amusing among all the ages of students within primary schools. A gamified assessment was seen as a fun activity that broke the school program routine in all 12 schools that was implemented and all the students that participated with high levels of motivation.

However, the most significant element that the puzzle brought into the development of this assessment tool was the element of tangibility. The fact that every single assembly could be tracked by color coding and every assembly was the result of deep meaningful interactions gave appropriate insights into individual contributions. Consequently, this unique form of tangibility was the cornerstone for measuring both the physical and mental collaborative process of an individual student.

But is it enough to hand a puzzle to four students and let them play? In order to transform this useful and tangible game into an assessment, it had to be infused with elements that define the construct of collaboration. The first concern was how to divide the 120 puzzle pieces between 4 students. This was addressed following the notion that sharing of resources [15,36] is of vital importance in the process. Consequently the “task should not be able to be solved by individual effort” [37]. For this reason, the pieces were divided equally among the group members (30 pieces each).

Additionally, the element of social interdependence [38] had to become evident in the design. Even though the pieces would be divided equally among the peers, how would that create the element of social interdependence? The solution in our understanding could come from coding the pieces and dividing them in such a manner that none of the individual bags contained pieces that could be connected with each other (Figure 1). There were numerous benefits in the design that were expected as a result of this element. By increasing the level of complexity of the task [15,39], the students are drawn even deeper into the search for possible solutions, further igniting the possibility for constructive argumentation.

Apart from the fact that it becomes explicit to the students that they have to depend on each other’s efforts (eliminating student loafing), it ignites in, an indirect but evident way, the element of interaction. The element of interaction is the “…minimum requirement for successful coordination” [40,41]. Enriching the interactive properties of the assessment lays the ground for unfolding the elements or verbal and non-verbal communication [42,43] attributes of the students.

By unfolding the communication channels of students, conflicts and disagreements become evident, and students are expected to resolve them in order to proceed with the assembly process. Conflict resolution [44] and negotiation [40] are important elements to include in the assessment design. Underscoring this rationale, ground rules were added that prohibited the students from touching or grabbing their peer’s pieces and only permitted them to activate the existent verbal and/or non-verbal (showing, pointing, and gestures) collaborative communication skills.

This effort also supports, to an adequate level (provided the educator/researcher supervises the process), the prerequisite condition of the symmetry of action that Dillenbourg [45] pinpointed. It also protects the assessment from some students overtaking the range of actions from other students, an issue that would clearly compromise the ability to measure the physical and mental contributions of the individual within the group.

In parallel, the fact that the students are of the same age and the groups basically consist of classmates also supports the elements of symmetry of knowledge and status [45]. The fact that all group members aim to assemble the puzzle also supports, in a direct and explicit manner, the last prerequisite guideline for a collaborative activity, which is based on the symmetry of goals [45].

The element of handing roles to the students was thought of as very restrictive to the range or actions and ideas. An ill-structured task approach [46] was seen as supportive for students to constantly re-evaluate the possible solutions and constantly redefine and optimize their individual and collaborative efforts.

The next and final stage of this assessment was to create the numerical representation of the individual’s collaboration skills.

In order to create the marks and grading of collaboration skills, the marks should reflect the individual’s contribution to the group effort [15]. However, this was by no means restrictive towards placing a mark for the individual based on the outcome of the group.

It is important to pinpoint the way that CSB Test 1 (Figure 2) views collaboration skills before moving forward. According to Hesse et al. (ATC21S) [40], individuals need a number of social skills to help them coordinate actions in synchrony with other participants. The social skills associated with this are participation skills, perspective taking skills and social regulation skills. The element of participation refers to a subset of skills that reflect the level of action (activity within environment), interaction (interacting and responding to the contribution of other) and perseverance (undertaking and completing a task or a part of a task individually). The element of perspective taking skills refers to a subset of skills that reflect adaptive responsiveness (ignoring, accepting or adapting contributions of others) and audience awareness (awareness of how to adapt behavior to increase suitability for others). The element of social regulation skills refers to negotiation (achieving resolution), metamemory (recognizing own strengths and weaknesses), transactive memory (recognizing strengths and weaknesses of others) and responsibility initiative (assuming responsibility for ensuring parts of task are completed by the group).

In order to develop CSB Test 1, it was important to mold and adapt the group puzzle task, settings and numerical measurements into a form that would allow the collaboration skills describe by the ATC21S described to be first expressed and then measurable for the individual participant. Additionally, it had to reflect the young age of the participants, while complying with public school regulation (no video recorders and a limited ability to photograph) in order to bring the assessment into a larger scale of implementation and derive useful conclusions through measuring the evidence from the reflective actions of the students.

The following section describes in detail how the proposed approach transforms, in numerical representation, the collaboration skills expressed on the puzzle tasks processes.

• Point 1 = TPAG% (Total Pieces Assembled by Group%)

The student is assigned a mark based on the group’s performance. If the group as a whole performs well or poorly, it directly reflects on the individual’s mark. Including a group mark for the individual serves the purpose of reflecting the individual’s input, which cannot be directly numerically measured. For example, if a student’s contribution to the strategic approach is verbal or non-verbal but is not followed by a measurable action, and if many of these ideas, when materialized, are successful, and then this input has to be recognized in some manner. The following calculations specify the details of the TPAG marking:

• If, for example, 45% of pieces are assembled by the group, then five marks are awarded to each individual.

This, obviously, can be criticized according to Child and Shaw [15], on the basis of fairness. However, the weighting still remains an open question, as the relevant research reveals. This research proposes that the weighting of the group mark to the individual should be 20% of the total mark of the individual, and the remaining 80% should be awarded based on the individual’s measurable contributions. The main reason behind this is the focus on the collaborative process rather than on the collaborative outcome.

• Point 2 = IPPAG% − IPPAA%
  (Individual Puzzle Pieces%Assembled within Group − Individual Puzzle Pieces%Assembled Alone)

The student is assigned a mark for the increased or decreased number of puzzle pieces assembled within the group. After the 45-min individual puzzle test, the number of puzzle pieces assembled is counted and a percentage is derived. When the group puzzle test is finished, the puzzle pieces of all the individuals are counted and, again, a percentage of the individual’s assigned pieces is derived. These two numbers are directly compared by subtracting the two percentages. Depending on the result, the individual student has an increased, decreased, or the same quantity of assembled pieces. The marking is assigned following the scales in

Table 1. Numerical measurements.

	Point 1		Point 2		Point 3		Point 4		Point 5
0–10%	1	+41% Decline	1	0–10%	1	0–10%	1	0–0.5 pieces/min	1
11–20%	2	31–40% Decline	2	11–20%	2	11–20%	2	0.51–1 pieces/min	2
21–30%	3	21–30% Decline	3	21–30%	3	21–30%	3	1–1.5 pieces/min	3
31–40%	4	11–20% Decline	4	31–40%	4	31–40%	4	1.5–2 pieces/min	4
41–50%	5	1–10% Decline	5	41–50%	5	41–50%	5	2–2.5 pieces/min	5
51–60%	6	0–10% Improvement	6	51–60%	6	51–60%	6	2.5–3 pieces/min	6
61–70%	7	11–20% Improvement	7	61–70%	7	61–70%	7	3–3.5 pieces/min	7
71–80%	8	21–30% Improvement	8	71–80%	8	71–80%	8	3.5–4 pieces/min	8
81–90%	9	31–40% Improvement	9	81–90%	9	81–90%	9	4–4.5 pieces/min	9
91–100%	10	41%+ Improvement	10	91–100%	10	91–100%	10	4.5+ pieces/min	10

. The weight of this mark is 20%. The following calculations specify the details of this particular marking:

• If, for example, IPPAG = 45% and IPPAA = 30% then we have a 15% increase, consequently the mark that is awarded = 7.

Slicing the individual’s problem-solving abilities to the collaborative context is very important in determining the level of collaboration skills. The notion behind this marking is to understand if the individual’s collaboration skills, described by Hesse et al. [40], are expressed through their impact on the quantity of interactive actions. If the individual expresses high levels of perspective-taking skills that permit them to constantly adjust their actions and finally increase their individual contribution within the group beyond their individual problem-solving abilities, then the collaboration skills are evident in an explicit and tangible manner.

• Point 3 = IPPAG%
  Individual Puzzle Pieces%Assembled within Group

The student is assigned a mark for their quantity of pieces assembled within the group. Similarly, this mark is high when the contribution is high and low when the contribution is low. The weight of this mark is again 20%. The following specify the details of IPPAG marking:

• If, for example, IPPAG = 55%, then the awarded mark = 6.

The basis for this particular marking follows the guideline of individual contribution [43,47] within a group effort, which is the cornerstone in determining the individual’s ability to effectively function in a highly demanding, collaborative activity. Additionally, the increased level of complexity of the collaborative puzzle test contributes to the decrease in the level of learning between the test and retest phase. Therefore, the mark represents a closer reflection of the individual’s collaboration skills.

• Point 4 = ICPG
  Individual Convergence with Peers in the Group

The student is assigned a mark for their ability to converge efforts with the rest of the group. If the assemblies of the individual fall within the collaborative efforts of all the group members, then the percentage of the assemblies is awarded with a complete mark. If the efforts fall within the efforts of three or two students, then the points are lower. The following calculations specify the details of the ICPG marking:

• If 45% of pieces are within efforts of 2 then—5 marks × 0.50 = 2.5
• If 45% of pieces are within efforts of 3 then—5 marks × 0.75 = 3.75
• If 45% of pieces are within efforts of 4 then—5 marks × 1 = 5

In a sense, every student carries a 0.25-point convergence factor within the group. The weight of this mark is 20%.

The element of convergence [48] is again of incremental value in order to calculate the utterances of collaboration skills. Many students (at the age of 11), when participating in a group activity, have preferences for whom they will be interacting with. This results in neglecting the ideas, knowledge, and contributions of other group members. Even though the task is designed in a manner that social interdependence is very high among the participants, it was decided to fine-tune it even more to further reveal the individual’s level of adaptability. The rationale, again, was simple: can the student converge their individual efforts with the efforts of others? If they do, with how many (of the three others) do they achieve it with? If the student contributes a puzzle piece in an effort that all four group members worked together to assemble, then the student is graded with the highest mark of convergence. Similarly, if a student assembles pieces only with one of their group members (which is the absolute minimum collaborative requirement), then the student is marked with the lowest convergence mark. In this way, with the use of simple, replicable conditions and tools, there are higher possibilities of reaching useful conclusions.

• Point 5 =Individual Performance Efficiency in Group
  Group Completion Rate − Individual Completion Rate

The student is assigned a mark for their ability to effectively and efficiently increase their individual performance in assembling puzzle pieces within the group. Practically, we derive the group’s completion rate by dividing the puzzle pieces by the time elapsed to assemble them. Then, we derive the individual completion rate from the individual puzzle test and directly subtract these two numbers. The following specifies in detail the calculations:

• If, for example, GCR = 2.6 pieces/min and ICR = 1.4 pieces/min, then we can understand that the student within the group increased their performance in assembling the pieces to +1.2 pieces/min, which is awarded 3 marks (Table 1).

If the GCR is much higher than the ICR, then the student is awarded a higher mark as it is measurable that the collaboration skills allow this increase in performance to take place. The weight of this mark is 20%.

This is one of the most important factors in deriving the expression of collaboration skills. This particular point gathers the sum of all existing collaboration skills within an individual in a reflective and indirect manner. It again slices the individual performance from the collaborative performance, and from the remaining measurable result, we can derive the level of efficient coordination with peers. Participation, perspective-taking, and social regulation skills [40] are expressed when the individual efficiently coordinates their efforts with peers. Even if the students decide to reach a temporary consensus, it is highly difficult to perform at high levels if all do not coordinate efficiently. Even if the individual decides to follow or lead, due to the interconnectivity of the puzzle pieces, audience awareness, perseverance, and participation skills have to reach the highest levels in order to achieve the group’s increased outcome. If these skills are not activated by the individual, then the group rate will drop, and their individual rates (in the solo puzzle test) will subsequently either be higher (resulting in a lower mark) or have a smaller difference with the group rate (again resulting in a lower mark). In a sense, we can understand if the expertise or knowledge is spread and not localized [7] within the group, while measuring the individual’s collaborative ability to increase or decrease their own performance.

By adding all the individual marks (Figure 3) that are awarded to the student, we derive the total mark that reflects the sum of all the utterances and individual contributions directly or indirectly to collaboration skills performance. The basic focus of the experimental procedure is to investigate the association of total points between the individual and the rest of the group.

Total Points = P1 + P2 + P3 + P4 + P5.

(1)

3. Methods and Procedures

3.1. Context and Participants

The experiments aim to identify if the students as individuals express collaboration skills through their performance, in a stable manner in a standardized collaborative activity. A scale was created (Section 2) in order to measure the student’s collaborative performance, based on the evidence that was gathered from each individual. In order to understand whether the proposed approach produces results that are characterized by stability for each individual, it was decided to run a test and retest validation process. The following section portrays all the typical and regulatory settings of the experiments paired by data analysis of the total results derived.

3.1.1. Ethical Approvals

In order for the experiments to take place, it was necessary to obtain permission from the ethical committee of the University of the Peloponnese. Additionally, the Administration of Primary School Education of Argolis reviewed the experimental procedures and granted permission for their implementation. Written parental consent was collected from all participants. Ethical guidelines and standards were followed throughout the study to ensure the well-being and rights of the participants.

3.1.2. Study Location and Duration

The research was conducted in 12 public primary schools in Argolis Prefecture, Eastern Peloponnese, from 11 December 2023 to 5 June 2024. The average interval between the test and retest phases was 23 days (minimum 21 days).

3.1.3. Participant Selection

Students were selected using convenience sampling. Whole classes were chosen to ensure a diverse representation of the student population. The only restriction was the exclusion of students with mental disabilities who required the presence of a special education teacher. This ensured the sample represented the general student population of the specific age.

3.2. Procedures

3.2.1. Pilot Phase

Initially, a pilot phase was conducted in 2 schools, yielding positive results in terms of feasibility and initial data trends. These results included manageable time frames, clear student engagement, and preliminary data supporting the assessment tool’s validity. This led to testing the assessment tool in a larger population for a more comprehensive quantitative analysis.

3.2.2. Main Study

The main study took place in 12 primary schools of the public sector. The guidelines that were used were the same as the one in the pilot phase.

Table 2. Experimental setup.

Settings/Instruments	Test Phase	Retest Phase
Number of Participants	148	148
Gender Breakdown	76 Boys/72 Girls	76 Boys/72 Girls
Age	11–12	11–12
Groupings	Mixed gender of 4	Mixed gender of 4
Total Number of Groups	37	37
Educational Level	6th Grade Elementary	6th Grade Elementary
Individual Puzzle Time Limit	45 min	45 min
Group Puzzle Time Limit	45 min	45 min
Individual Puzzle	120 Pieces Puzzle (Age 6+)	120 Pieces Puzzle (Age 6+)
Group Puzzle	120 Pieces Puzzle (Age 6+)	120 Pieces Puzzle (Age 6+)

presents the experimental setup in detail.

3.2.3. Experimental Conditions

Experiments were conducted during the first two hours of the school day to avoid mental and physical fatigue. More specifically, the individual puzzle process took place during the 1st didactical hour, followed by a 10 min break, and then the group puzzle process took place during the 2nd didactical house. Classrooms were closed to external interactions, with the classroom teacher present but not interacting and uninvolved.

3.2.4. Pre-Experiment Preparations

Classrooms were inspected for adequate space, desks/chairs, and lighting. Adjustments were made to standardize conditions. On the experiment day, each student and each group participating received a six-digit code sticker for identification purposes (Figure 4).

3.2.5. Individual Puzzle Process

• Setup: Each student was assigned a desk with their code and puzzle
• Instructions:
  • Students start upon the researcher’s signal;
  • Students raise their hand upon completion;
  • Assembly stops at the 45-min mark.
• Data Collection:
  • Time of completion and photo of the completed puzzle;
  • Photo of the puzzle at the 45-min mark.

3.2.6. Group Puzzle Process

4.

Setup: Each group table was assigned a six-digit code sticker. Students received colored bags with 30 puzzle pieces each (red, blue, yellow and green).

5.

Instructions:

• Every student within the group is assigned a colored bag (red, blue, yellow, green) of 30 puzzle pieces;
• The researcher will visit every group to assign the colored bag to the student;
• In order for a piece of a puzzle to enter on the assembly paper, there has to be another one connected to it. If the attempt fails, then the piece returns back to the individual’s side;
• The students cannot attempt connecting the puzzle pieces outside the white assembly paper;
• The students cannot give their pieces to another group member;
• The students can communicate with verbal or non-verbal patterns, but touching or grabbing the pieces that are assigned to other group members is not allowed;
• If there are groups of pieces that need to be connected with other groups of pieces, then all the group members have to coordinate and move them collectively to be connected;
• When a group finishes the assembly, the students should raise their hand.

6.

Data Collection:

• Group puzzle completion time;
• Puzzle completion percentage;
• Individual pieces used;
• Photos of puzzle connections.

3.2.7. Data Recording and Retest Phase

Data were recorded in a detailed table, tracking student/group codes and performance. Additionally, a cell phone camera and a timer were used to time and capture the students’ efforts. During the retest phase, the procedure was repeated with different group compositions to assess individual collaboration skills across varying groups.

The notion behind having different groups of students, as previously mentioned, was to understand whether the students as individuals express their collaboration skills, through performance in the same manner, regardless of the group composition. For this reason, the main focus of the measurements was to identify the individual’s contributions, efficiency and convergence, as described more thoroughly in Section 2.

The next stage of the research process was to transfer all the relevant information gathered and start the calculations for every student individually.

4. Results

At this stage, it was important to investigate whether the students’ performance had any similarity with the performance of the group, since it is important to primarily measure the performance of the individual’s collaboration skills. After running the two tests, the students’ performance was found to be stable. Specifically, the mean and median were very close between the first (M = 87.40, Mdn = 89.00) and second test (M = 87.80, Mdn = 88.00). The standard deviation decreased from 9.28 to 8.11 (

Table 3. Results of test and re-test phases.

	First Test	Second Test
N	148	148
Missing	0	0
Mean	87.40	87.80
Median	89.00	88.00
Standard deviation	9.28	8.11
Minimum	52.50	74.00
Maximum	100.00	100.00

) in the second test, mainly due to the improvement in performance at the lower end of the distribution (Min1st = 52.20, Min2nd = 74.00).

Upon examining the normality of the data, a violation of the assumption of normality was found for the first (W = 0.933, p < 0.0001) and second (W = 0.944, p < 0.0001) tests. The same result can be seen in histograms below (Figure 3). Based on this result, nonparametric statistics were used, and the median was also used to summarize the variables.

Additionally, no differences were found in performance between schools, reinforcing the view that the results reflect a general trend. For the first test, where there was no homogeneity in variances, the Kruskal–Wallis test was used (F_(137,10) = 3.675. p < 0.001, KW = 14.566, p = 0.149); for the second test, ANOVA was used (F_(137,10) = 0.919, p = 0.5350; F_(137,10) = 10.402. p < 0.412). A visual representation of the distributions of the scores in the two tests is shown in the frequency histograms below (Figure 5). The first histogram also shows some outliers.

The basic criterion for drawing any conclusions based on the theoretical framework presented above is the relationship between the correlations among students’ grades in the first and second test with correlations (Figure 6) between student’s grades and the rest of the group’s performance. Given the lack of normality in the grade distributions, the Spearman correlation coefficient was used. A strong positive linear correlation was found between the grades received by students in the two tests (r_s = 0.623, p < 0.0001). It was also found that there was no correlation between the student’s grade and the average grade of the remaining three students (Figure 7, Figure 8 and Figure 9) in the group for the first (r_s = −0.017, p = 0.8387) and second (r_s = −0.119, p = 0.1505) tests. It is also apparent in the scatterplots below that there was a positive relationship between performances on the two tests (Figure 6). The lack of correlation between student performance and group averages is also evident.

5. Discussion

RQ: Is it feasible to measure the individual collaboration skills performance of 11–12-year-old students using the “Collaboration Skills Benchmarking Test 1” (CSB Test 1) assessment tool?

This work is a genuine effort in trying to produce one more drop of valuable information in the vast and interdisciplinary literature for collaboration and collaboration skills. As it was mentioned earlier that even the term ‘collaboration’ has many definitions, resulting in a number of operationalizations. This highly complex nature of collaboration skills paired with their importance in both personal and professional life acted as a springboard to start and continue this research. The main focus and purpose of this work was to develop a performance test that can measure the collaboration skills of an individual student.

For this purpose, CSB Test 1 focused primarily on the contributions of the individual within the group, while taking into account their differences in performance in an individual task. The results showed that the individual students who increased their performance within the first group test increased their performance in the second test as well, regardless of the group composition. Similarly, the students that performed lower or low at the test phase performed low and lower in the retest as well. The findings showed us that individual students, regardless of the group compositions, express a relatively stable level of collaborative performance.

In order to understand if the individual expresses stability in their collaborative performance, it was important to conduct a test–retest validation process. The results of this first large-scale experiment show that CSB Test 1 produced a promising outcome. Using the Spearman correlation coefficient for the data analysis revealed a strong positive correlation between the grades of the individual students in the test and retest phase of 0.623. This result is significant because it is paired with a lack of correlation between student performance and group average performance. The results did not show any difference in terms of gender or populations from different schools. Additionally, there is no correlation between the individual student and grade and the remaining three students in the group. Consequently, the proposed assessment tool produces validated results.

An important contribution of this work is the ability to reveal a clear view of collaboration skills without other skills blurring the view. It was important to create an instrument where all students could contribute as much as possible without other skills (writing, typing, etc.) enhancing or degrading their participation into the assessment. As Lench et al. [49] stated, “Collaboration requires communication…”, so it was important not to restrict communication to written communication. Additionally, according to Evans C. [9], “…it is impossible to collaborate without cognition or self-regulation”. Consequently, the performance and tangible evidence of the individual’s efforts within the proposed assessment, to our understanding, summon the utterances of collaboration skills effectively.

An additional contribution of this work is the fair treatment of all the students that have different levels of introversion and extroversion. If, for example, the assessment would measure the quantity of spoken words or sentences, it would again discard the internal processes of introverted students that would and could contribute through their actions in a short period of time. In a sense, this type of assessment gives a fair space to introverted students to contribute through actions. In the same manner, it is also fair for students of different ethnicities and with different levels of spoken language skills to be fairly assessed on their actual collaboration skills performance.

One of the most important factors in designing CSB Test 1 was the ability to trace the evidence of the individual’s contributions within a group activity without the use of highly complex and technologically demanding tools. Additionally, this issue was significant because there was high reluctance from parents to consent with the experimental procedures if they included the use of video recorders. For example, it is worth stating that out of 75 consent forms handed to students (in one school), there were only 7 consent forms that permitted even the use of photos. Consequently, and in order to have the ability to gather as many participants as possible, the instrument had to deliver a solution that would both comply with the strict public school regulations while gathering as much as possible information from the students’ actions.

The assessment’s novelty and uniqueness, in determining the performances of individual students’ collaboration skills, creates difficulty in comparing it with other assessments. To our knowledge, there is a lack of any similar instruments that attempt to assess collaboration skills through similar gamified performance assessments in the particular age of 11–12-year-old students. Comparing CSB Test 1 to existing assessments [17,18,19,20] that are compatible to the age range of 11–12, it is important to mention that, due to the relative immaturity of the age, students are susceptible to socially desirable answers. Moreover, the grading system that is commonly used in Greek regular school settings, throughout all the classes of primary school, creates high levels of difficulty for students and parents in consenting to participating in such assessments. Therefore, the rationale of this research, as previously mentioned, was to primarily focus on students’ reflective actions in a covert and gamified assessment activity.

Regarding the contents and the perspective of the existing assessments [17,18,19,20], Lowers at al.’s [17] approach was more focused on creating teamwork scales that measured the perceptions of teamwork in students than on the collaboration skills expressed in an activity. Additionally, Marks [18] focused on the element of perseverance, which, in the ATC21S [40] framework, is part of the collaboration spectrum of skills. In parallel, Duckworth et al. [19] measured the elements of grit and perseverance. On the other hand, Sperling et al. [20] focused on another section of collaboration and, more specifically, on self-regulating learning.

5.1. Stakeholders’ Perspectives

From the perspective of a primary school educator, the proposed approach can act as a supportive tool to understand whether a certain pedagogical intervention had any effects on the collaboration skills of the students involved. The assessment is relatively easy to implement and does not require special equipment or knowledge to be handled. Usability is a very important issue when the aim of an assessment is to be launched in public use in typical school settings. The cost-effective nature of the assessment, paired with its low labor intensity, also makes this proposal a flexible solution for wide use. Additionally, the assessment’s timed nature (two 45 min tasks) is specifically designed in order to fit perfectly into two didactical hours in a typical daily school schedule. Across all of the experiments conducted, the students behaved in a playful manner and treated the assessment as an enjoyable activity. This is another aspect that educators seek in assessment tools in order to face the students’ reluctance to participate in an otherwise unenjoyable procedure.

From the perspective of policymakers, the proposed approach can act as a useful tool in measuring the effectiveness of programs that aim in developing collaboration skills to students. The assessment can be utilized by the general remedial policy that the Greek Ministry of Education aims in creating equal opportunities for all students [50]. Additionally, it can be utilized as a tool that can be supportive to the new focus of the Greek Ministry of Education in enhancing students’ 21st century skills.

From the perspective of a researcher, the proposed assessment can reveal valuable information about collaboration skills in relation to collaborative performance. The assessment can be used as a pre-post assessment tool to determine whether or not a methodological approach is successful in developing collaboration skills. Additionally, it would be useful to understand the relationship between personal perceptions gathered through surveys and questionnaires and their actual collaborative performance. This is an issue that Evans [9] pinpointed when trying to portray the demanding endeavor of developing performance assessments for measuring collaboration skills. Due to its flexible intercultural nature, it can also be implemented in diverse populations and ethnicities to determine similarities or differences among participants.

5.2. Limitations and Future Research

As with all research, this work has several limitations. There was significant reluctance among parents to return the consent forms, which resulted in a limited number of participants. However, since this first large-scale experiment was implemented successfully without any problems and was informed in the local school communities, it is expected that the following years will be received with a larger number of participants. This will help the current research to reveal even more information through deeper and more refined data analysis of the various factors that determine the individual’s collaboration skills.

Another limitation of this research is the age restriction, as it only includes 11–12-year-old students. Research shows that there is a need to more comprehensively understand the trajectories of students’ development of collaboration skills. The fact that CSB Test 1 does not rely on subject knowledge hierarchies makes it a useful tool for shedding more light on the pool of knowledge regarding the development of collaboration skills relating to age. For this reason, it is decided to implement CSB Test 1 to ages ranging from 9 to 12, keeping each age range (9–10, 10–11 and 11–12) in different cohorts without mixing them.

6. Conclusions

In this research, we attempted to create a performance-based assessment tool for measuring the collaboration skills of 11–12-year-old primary school students while being engaged in both individual and group puzzle activities. The proposed approach was tested and retested for validating that the individual students express stable grading among two different group compositions (mixed groups of four). Using the Spearman correlation coefficient, we identified that the individual student performance between the test and retest phases was 0.623, indicating a strong correlation. Additionally, we also found that the group average performance and the rest of the students within the group were not correlated with the performance of the individual’s collaboration skills. This means that the assessment instrument successfully measures the performance of collaboration skills for individual students.

This research presents a new approach in assessing collaboration skills through performance measures for primary school students. It is playful, cost-effective, intercultural, and highly usable with stealth attributes, presenting a unique contribution in the vast and interdisciplinary field of collaboration and more specifically collaboration skills assessments. The evidence-based approach sheds more light on students’ actions and creates, to our understanding, a sound basis for further research in the coming years. Due to parental reluctance to consent to the experiments, student participation was more limited than we expected. Higher student participation would have resulted in more detailed and refined statistical analysis. However, the first step has been made, and once the school communities became more accustomed to the nature of the experiments, we expect larger numbers of participants in the coming years.

In conclusion, the unique design of CSB Test 1, which does not rely on subject knowledge, presents a valuable opportunity to uncover deeper insights into the development of collaboration skills throughout students’ lives. This highlights the need for further exploration and refinement of methodologies aimed at nurturing these essential skills in primary school students. The next phase of this research will involve implementing CSB Test 1 across a broader age range with the expectation of revealing critical data on how collaboration skills evolve at different stages of early education.