Exploring the Features of Educational Robotics and STEM Research in Primary Education: A Systematic Literature Review

Abstract

STEM education programs with educational robotics are frequently used in formal or informal education, with participants ranging from kindergarten children up to university students. The widespread implementation of these programs in schools and the growing interest of researchers in the field has led several authors/researchers to review and summarize the characteristics of STEM research. However, the literature on the features of STEM research in primary education (kindergarten and primary school) is limited. Therefore, this article is a systematic literature review that tries to enrich the STEM agenda by answering the questions: (a) which study designs are commonly used in STEM interventions, (b) what the characteristics of the sample are (number/age of the students), (c) which equipment and user interfaces (tangible/graphical) are used, and (d) what are the characteristics of the studies (duration, intervention objectives, activities) and how studies’ data were recorded. For this review, 36 out of 337 articles were analyzed and emerged from eight databases, three search-keywords and six exclusion criteria. The examination of the reviewed articles showed, inter alia, that non-experimental design is usually used, that in half of the cases written evaluations are used and the sample size is almost equal between girls and boys. Finally, long-term research is restricted, therefore it is not safe to generalize the findings of these studies.

1. Introduction

The foundations of educational robotics (ER) and science, technology, engineering and mathematics (STEM) lie in the learning theory of constructivism, by Piaget. According to Piaget, knowledge is an experience constructed by interaction with the environment [1]. Papert extended the theory of constructivism, stating that when real-world content is used, learning is more effective [2]. Moreover, some researchers claim that the use of robots in STEM teaching and learning might help students to understand related topics in more depth and engage them in complex problem solving [3].

Over the past several years, ER and STEM activities have entered into many schools around the globe. This is because some studies have shown that ER activities appear to increase students’ interest and motivation, fostering the learning process [4]. Moreover, ER activities might support teachers in an effort to make their lessons easier and more enjoyable [5,6]. The variety of ER activities seems to attract students and, in some cases, might have cognitive, social and metacognitive benefits at all levels of education [7,8,9,10]. Similarly, other studies argue that students might develop several skills such as computation thinking (CT), problem solving, collaboration and self-efficacy through ER [11]. These skills are essential, as they will help children cope with the challenges of their adult life [12].

On the contrary, there are studies such as Konijn and Hoorn [13], in which students who participated in ER activities did not show statistically significant differences compared to students who attended the standard curriculum. In other words, ER did not have a positive effect compared to classical teaching.

Based on the above, it becomes clear that in order to be able to draw safer conclusions about the value of using robots in STEM teaching, it is necessary to perform an in-depth analysis that will present the characteristics of the studies that have been undertaken in the specific scientific domain. Consequently, this article aims to summarize the features of STEM research in primary education (kindergarten and primary school). In this way, it provides researchers and educators with useful information for the implementation of STEM programs with ER.

Background

Many articles have been written about STEM education and its effect on the educational process. Hence, there are several reviews that have examined and analyzed the impact and benefits of STEM education. For instance, some authors focused on a specific topic (e.g., special education), while others dealt with specific features (e.g., CT assessments).

Benitti [14] used six databases from which he selected 10 articles, and he aimed at primary, middle and high schools. Benitti’s review considered that an article would be excluded if it presented only qualitative learning assessment. His research questions were: “(a) What topics (subjects) are taught through robotics in schools? (b) Is robotics an effective tool for teaching?”. The results showed that most studies focused on mathematics and physics topics. In addition, ER usually enhances learning, though there are several factors that can affect the outcome, such as the pedagogical approaches, the activities and the working group size.

Xia and Zhong [15] used 22 articles from one database and three rounds of snowballing approach (a snowballing approach uses the reference list of an article to find other articles), aimed at K-12 education. Their research questions were: “(a) How have robotics been incorporated into K-12? (b) What implications for teaching are indicated by these empirical studies?”. The results showed that most of the studies were conducted in elementary schools, lasted 8 weeks and up to 40 students were involved, using Lego. Finally, the design of most studies was non–experimental, while observations and questionnaires were used as measurement instruments.

Anwar et al. [16] used five databases from which they selected 147 articles published between 2000–2018. The authors’ aim was to determine the specific benefits of K–12 STEM education for student and teachers. Thus, they categorized the studies as “(a) general benefits of educational robots, (b) learning and transfer skills, (c) creativity and motivation, (d) diversity and broadening participation, (e) teachers’ professional development”. Their findings show that ER can be used as a learning tool even with students who do not show much interest in science and technology. In addition, ER enables a multidisciplinary approach, which helps students develop connections for STEM concepts. Students’ creativity and motivation can be enhanced with activities related to everyday life. Moreover, groups like minorities can benefit from ER by developing knowledge and skills. Lastly, the findings show that teachers rely on Massive Open Online Courses (MOOCs) for professional development.

Cutumisu et al. [17] used seven databases from which they selected 39 articles published 2014–2018, and they aimed at K-12 education. The review focused on CT, and especially on the feature classification of CT assessments. The results showed that most assignments aim at algorithmic skills, problem decomposition and logical thinking. In addition, assignments were aligned with STEM subjects, while most studies adopted a quasi-experimental design. Finally, the findings showed that most studies used selected-response items such as multiple-choice questions.

Gao et al. [18] aimed to review the assessment of student learning in STEM education. More specifically, they used 49 articles published between 2000–2019 and they did a double analysis: (a) they categorized the assessments as mono/inter/trans-disciplinary; (b) they categorized the learning objectives as knowledge, skill, practice or affective domain. Their research showed that although several programs aimed at interdisciplinary education, most assessments were monodisciplinary and targeted at knowledge.

Tlili et al. [19], based on activity theory [20] aimed at robot-assisted special education. Authors used eight databases from which 30 articles were selected for their analysis. In particular, they categorized aspects of studies such as disability types, use of robots, learning domains, activity types and type of performance measures. Their analysis showed the necessity for a stronger link between design/implementation and the needs of students with disabilities. On this basis, recommendations are provided to minimize omissions in future designs.

According to the above, the previous reviews dealt with ER and STEM research without focusing on primary education. Specifically, they include studies from kindergarten children up to university students. In addition, some reviews used articles from a specific time period, or examined a specific topic like CT assessments. Therefore, the purpose of our article is to deliver a systematic literature review of STEM research for young students in primary education. However, we do not focus on articles with specific topics (such as special education) or features (such as including quantitative evaluations).

Subsequently, this review tries to deepen the existing STEM literature providing useful information on current trends for educators and researchers. This way, a more detailed view will arise for the features of STEM research for students under 12 years old. To achieve this, multiple aspects of studies that were used in our review have been recorded. In more detail, we have recorded: the sample characteristics (age of students, number of girls/boys, prior experience), the school, the country, the subject of study, the intervention objectives, the study design, the duration of the intervention, the activities/tasks, the size of groups in collaborative learning, the user interface, the data type, the data source, the equipment type and the results of the intervention.

The rest of the paper is organized as follows: first we list the methodology and the process of searches in databases. Following this, with the appropriate criteria, we are led to the articles that we use. Finally, our results are grouped and the findings of STEM research in primary education are discussed.

2. Methods

According to Okoli [21] there are four major stages to conduct any kind of systematic literature review:

• Stage 1. Planning

• Identify the purpose
• Draft protocol and train the team

• Stage 2. Selection

• Apply practical screen
• Search for literature

• Stage 3. Extraction

• Extract data
• Appraise quality

• Stage 4. Execution

• Synthesize studies
• Write the review

Planning and Selection (stage 1 and stage 2): The aim of this review is to investigate the features of STEM research in primary education, as the literature is limited for these students. More specifically, the research questions (RQs) are:

• Research Question 1 (RQ1). Which study designs are commonly used in the STEM interventions?
• Research Question 2 (RQ2). What are the characteristics of the sample?
• Research Question 3 (RQ3). Which equipment and user interface are used?
• Research Question 4 (RQ4). What are the characteristics and how were studies’ data recorded?

For the purpose of our article, a systematic literature review was undertaken in 2021. Eight databases were used: Scopus, IEEEXplore, ACM, ScienceDirect, ERIC, DOAJ, JSTOR and SpringerLink. The literature was searched with three different keywords, as shown in

Table 1. Searches.

Search	Keyword
1	“primary school” AND “educational robotics”
2	(primary OR elementary) AND (educational OR education) AND (robotics OR robots) AND (STEM) AND (experimentation OR experiment)
3	(primary OR elementary) AND (educational OR education) AND (robotics OR robots) AND (STEM)

.

After each search, six exclusion criteria (EC) were used:

• EC1. Study for teachers
• EC2. Paper/study for framework
• EC3. Study for older students than 12 years old
• EC4. General paper for STEM (without intervention)
• EC5. Review
• EC6. Irrelevant paper

Extraction (stage 3): The articles were examined on the basis of their title, abstract and content. The articles had to be written in English and published in a journal. In total, 255 out of 337 articles were excluded. From the 82 included journal articles, 46 were duplicates. Therefore, 36 unique/selected articles dealing with STEM research in primary education emerged, as shown in

Table 2. Excluded/included articles.

Search	Articles	EC1	EC2	EC3	EC4	EC5	EC6	EC Sum	Included	Duplicated	Selected
1	69	8	12	7	5	10	0	42	27	12	15
2	65	7	12	21	5	3	4	52	13	7	6
3	203	26	35	49	23	10	18	161	42	27	15
Total	337	41	59	77	33	23	22	255	82	46	36

.

In detail, the search results for each database are shown in

Table 3. Result of databases.

Database	Articles				Exclude				Include
	Search 1	Search 2	Search 3	Sum	Search 1	Search 2	Search 3	Sum	Search 1	Search 2	Search 3	Sum
Scopus	15	4	40	59	7	3	29	39	8	1	11	20
IEEEXplore	2	0	5	7	1	0	5	6	1	0	0	1
ACM	1	0	0	1	1	0	0	1	0	0	0	0
ScienceDirect	18	1	8	27	14	0	6	20	4	1	2	7
ERIC	7	7	64	78	4	6	50	60	3	1	14	18
DOAJ	9	2	2	13	4	2	2	8	5	0	0	5
JSTOR	4	0	1	5	3	0	1	4	1	0	0	1
SpringerLink	13	51	83	147	8	41	68	117	5	10	15	30
Total	69	65	203	337	42	52	161	255	27	13	42	82

.

In order to be able to organize and analyze the findings, we categorized some aspects of the studies. In detail, the subject of the study refers to the content to be taught, such as: physics, technology, mathematics, bioengineering, history, computer science or robotics (combination of computer science and engineering). The intervention objectives refer to the offered knowledge (such as programming), skills (such as CT, collaboration) and attitudes (such as motivation) that students can acquire. The activities refer to the nature of the tasks students are called to complete, which can be: mathematics, programming and/or engineering. The robotic system is programmed via tangible user interface (TUI), or graphical user interface (GUI). The type of data is qualitative, and/or quantitative. Quantitative data emerged from written evaluations like multiple choices, whereas qualitative data emerged from observation and/or interviews. The data source refers to how the data were recorded, like videos, observations, interviews and/or questionnaires. The study design according to Campbell and Stanley [22] can be represented with Χ for treatment/intervention, R for randomized assignment and O for observation/measurement. However, in order for qualitative and quantitative research to be presented in the same table, we propose that the representation for observation/measurement be made as follows:

• O = measurement through questionnaires (quantitative)
• Oq = observation through video, interview or observation by researchers (qualitative)
• Om = mixed measurement/observation through quantitative and qualitative data

In addition, according to the taxonomy of Trochim and Donnelly [23] and Benitti [14] we classified the study design into three categories:

• (true) experimental: a design with random assignment to groups
• quasi-experimental: a design with no random assignment to groups
• non-experimental: a design without groups

Finally,

Table 4. Examples for study design.

Category	Design	Representation *
Non-experimental	One-shot post-test/observation only	X Om
(true) Experimental	Randomized post-observation only	R X Oq R Oq
Quasi-experimental	Nonequivalent comparison group	N X O N O

* X: treatment/intervention, O: quantitative measurement, Oq: qualitative observation, Om: quantitative and qualitative measurement/observation, R: random assignment, N: non-random assignment.

shows three examples for each study design category and its representation.

3. Findings

Execution (stage 4): The results of the systematic literature review were sorted into tables, so that we could analyze and discuss them more efficiently. In particular,

Table 5. Study design.

ID	Article	Category	Design	Representation **
1	Chevalier et al., 2020 [24]	Experimental	Randomized post-observation only	R X Oq R Oq
2	Vicente et al., 2020 [25]	Non-experimental	One-shot pre/post-test	O Χ O
3	Angeli and Valanides, 2020 [26]	Experimental	Randomized pre/post-test control group	R O X1 O R O X2 O
4	Muñoz et al., 2020 [27]	Non-experimental	One-shot pre/post-test	O Χ O
5	Cervera et al., 2020 [28]	Non-experimental	One-shot post-test/observation only	X Om
6	Chiazzese et al., 2019 [5]	Quasi-experimental	Nonequivalent comparison group	N X O N O
7	La Paglia et al., 2018 [29]	Experimental	Randomized pre/post-test	R O X O R O
8	Julià and Antolí, 2016 [12]	Experimental	Randomized pre/post-test	R O X O R O
9	Di Lieto et al., 2017 [30]	Non-experimental	One-shot pre/post-test	O Χ O
10	Konijn and Hoorn, 2020 [13]	Experimental	Randomized pre/post-test control group	R O X1 O R O X2 O
11	Arfé et al., 2020 [31]	Experimental	Randomized pre/post-test	R O X O R O
12	Ioannou and Ioannou, 2020 [32]	Non-experimental	One-shot pre/post-test/observation	Om Χ Om
13 *	Bargagna et al., 2019 [33]	Non-experimental	One-shot pre/post-observation	Oq Χ Oq
14	Zviel-Girshin et al., 2020 [34]	Non-experimental	One-shot post-test/observation only	X Om
15	Sullivan and Bers, 2018 [35]	Non-experimental	One-shot mid/post-test/observation	X Om Χ Om
16	Strawhacker et al., 2020 [36]	Non-experimental	One-shot pre/post-test/observation	Om Χ Om
17	Sullivan and Bers, 2016 [37]	Non-experimental	One-shot post-test only	X O
18	Boeve-de Pauw et al., 2020 [38]	Non-experimental	One-shot pre/post-test	O Χ O
19 *	Jung et al., 2020 [39]	Non-experimental	One-shot post-observation only	Χ Oq
20	Cho et al., 2017 [40]	Non-experimental	One-shot post-observation only	Χ Oq
21	Metin, 2020 [41]	Non-experimental	One-shot pre/post-test	O Χ O
22	Master et al., 2017 [42]	Experimental	Randomized post-test only	R X O R O
23	Casey et al., 2018 [43]	Non-experimental	One-shot pre/post-test	O Χ O
24	Newton et al., 2020 [44]	Non-experimental	One-shot pre/post-test/observation	Om Χ Om
25	Baek et al., 2019 [45]	Non-experimental	One-shot pre/post-test/observation	Om Χ Om
26	Popa, 2020 [46]	Experimental	Randomized post-test/observation only	R X OmR Om
27	Moore et al., 2020 [47]	Non-experimental	One-shot post-observation only	Χ Oq
28	Üşengül and Bahçeci, 2020 [48]	Experimental	Randomized pre/post-test	R O X O R O
29	Sisman et al., 2020 [49]	Non-experimental	One-shot pre/post-test	O Χ O
30	Nemiro, 2020 [50]	Non-experimental	One-shot post-observation only	Χ Oq
31	Sullivan and Bers, 2019 [51]	Experimental	Randomized pre/post-test	R O X O R O
32	Ching et al., 2019 [52]	Non-experimental	One-shot pre/post-test/observation	Om Χ Om
33 *	Taylor, 2018 [53]	Non-experimental	One-shot pre/post-observation	Oq X Oq
34	Sung et al., 2017 [54]	Experimental	Randomized pre/post-test	R O X O R O
35 *	Taylor et al., 2017 [55]	Non-experimental	One-shot pre/post-observation	Oq X Oq
36	Sullivan and Bers, 2013 [56]	Non-experimental	One-shot post-test only	Χ O

* Special education. ** R: random assignment, N: non-random assignment, X: treatment/program, O: quantitative measurement, Oq: qualitative observation, Om: quantitative and qualitative measurement/observation.

lists the 36 studies, their category and design.

Twenty-eight studies (78%) were published from 2018 onwards, while only one study was published before 2016. Twenty-four studies (67%) use non-experimental design. Eleven studies (30%) use experimental design, while only in one case (3%) is quasi-experimental design used, as shown in Figure 1.

From the non-experimental category, we can observe that the researchers in 16 studies (67%) out of 24 preferred a design that contained a test and/or observation before and after the intervention. From the experimental category, we can observe that the researchers in 8 studies (73%) out of 11 preferred a design that contained a test and/or observation before and after the intervention. For the quasi-experimental category, there is only one study and it is not safe to draw conclusions.

Table 6. Sample characteristics.

ID	Continent	Country	School(s) **	Sample
				Age	Girls	Boys	Sum	Prior Experience
1	Europe	Switzerland	P	9–10	13	16	29	Yes
2	Europe	Spain	P	10–11	17	13	30	Yes
3	Europe	Cyprus	K	5–6	26	24	50	No
4 *	North America	Panama	K and P	4–7	ns	ns	240	No
5	Europe	Spain	P	7–8	ns	ns	33	Yes
6	Europe	Italy	P	8–10	30	53	83	ns
7	Europe	Italy	P	10	12	18	30	ns
8	Europe	Spain	P	12	10	11	21	ns
9	Europe	Italy	K	5–6	7	5	12	No
10	Europe	Netherlands	P	8–10	41	45	86	No
11 *	Europe	Italy	P	6	92	87	179	No
12	Europe	Cyprus	P	8–9	17	17	34	Yes
13	Europe	Italy	P	8,5	2	6	8	No
14 *	Asia	Israel	K and P	4–7	102	95	197	No
15 *	Asia	Singapore	K	3–6	ns	ns	98	No
16	North America	USA	K and P	4–7	6	19	25	No
17	North America	USA	K and P	4–7	ns	ns	60	ns
18	Europe	Belgium	P	10–12	718	778	1496	No
19	North America	USA	P	8	0	1	1	No
20	North America	USA	P	7–8	12	12	24	ns
21	Asia	Turkey	K	5	16	8	24	No
22	North America	USA	P	6–7	48	48	96	ns
23 *	North America	USA	P	9–11	ns	ns	257	ns
24 *	North America	USA	P	8–11	37	56	93	ns
25	North America	USA	P	7–8	9	13	22	Yes
26	Europe	Romania	P	10–12	ns	ns	72	ns
27	North America	USA	P	7–8	2	1	3	No
28	Asia	Turkey	P	10–11	ns	ns	36	ns
29	Asia	Turkey	P	8–12	5	34	39	Yes
30 *	North America	USA	P	9–12	97	97	194	ns
31	North America	USA	K and P	5–7	ns	ns	105	ns
32	North America	USA	P	9–12	6	12	18	Yes
33	North America	USA	K and P	4–7	2	1	3	ns
34 *	North America	USA	K and P	5–7	30	36	66	No
35	North America	USA	P	6–8	2	1	3	No
36	North America	USA	K	5–6	25	28	53	ns

* More than one school participated in the program. ** K: kindergarten, P: primary.

lists the sample characteristics for each study. In particular, we recorded the continent, the country, the school and the students’ age, gender and prior experience.

The studies were conducted in primary schools (64%), kindergartens (14%) or a combination of both (22%). A total of 49% of the studies were conducted in North America, 37% in Europe and 14% in Asia. The dispersion of the studies on the world map is shown in Figure 2.

The age of the students who took part in the studies is shown in Figure 3.

We can observe that 66% of the participants were older than 7, while 20% are kindergarten children. More specifically, from the total number of participants in all studies, 45% were girls and 55% were boys. Additionally, we can observe that in seven studies (19%) students had previous experience with ER. The distribution of the sample size is shown in Figure 4. We can see from Figure 4 that most of the studies had about 60 students.

Table 7. Materials and User Interface.

ID	Equipment Type	Object	User Interface
1	Wheeled	Thymio	Graphical (block-based)
2	Wheeled	PrintBot Renacuajo	Graphical (block-based)
3	Wheeled	Bee-Bot	Tangible
4	Wheeled	Bee-Bot	Tangible
5	Wheeled	Bee-Bot	Tangible
6	Modular	Lego® Education WeDo 2.0	Graphical (block-based)
7	Modular	Lego Mindstorms EV3	Graphical (block-based)
8	Modular	Fischertechnik sets	Graphical (block-based)
9	Wheeled	Bee-Bot	Tangible
10	Humanoid	NAO	Tangible
11	Software	Code.org	Graphical (block-based)
12	Wheeled	Bee-Bot	Tangible
13	Wheeled	Bee-Bot	Tangible
14	Modular	Lego robotics equipment	Graphical (block-based)
15	Wheeled	KIBO	Tangible
16	Game Board	CRISPEE	Tangible
17	Wheeled	KIWI	Tangible
18	Wheeled	Techno Trailer	Tangible
19	Modular	Cubelets	Tangible
20	Wheeled and Modular	Bee-Bots and Cubelets	Tangible
21	Wheeled	Cubetto	Tangible
22	Wheeled	Animal robot	Graphical (block-based)
23	Wheeled	Roamer	Tangible
24	Modular and Software	EV3 and NXT robotics kits	Graphical (block-based)
25	Modular	Mindstorms EV3	Graphical (block-based)
26	Modular	Mindstorms EV3	Graphical (block-based)
27	Wheeled	Code and Go™ Robot Mouse Activity Set	Tangible
28	Modular	Lego® Education WeDo 2.0	Graphical (block-based)
29	Modular	The Robotis Dream ER kits	Graphical (text-based)
30	Modular	Lego robotics equipment	Graphical (block-based)
31	Wheeled	KIBO	Tangible
32	Modular	Lego Mindstorms	Graphical (block-based)
33	Wheeled	Dash	Graphical (block-based)
34	Software	Scratch Jr	Graphical (block-based)
35	Wheeled	Dash	Graphical (block-based)
36	Modular	Lego Mindstorms	Tangible and graphical

lists the materials and user interface used in each study. It is noteworthy that the equipment type in 18 studies (50%) was wheeled, in 12 studies (33.3%) was modular and in 2 studies (5.6%) a software application was used. In the remaining four studies, different equipment types were used. In addition, Lego robotic systems were used in 10 studies (27.8%), Bee-Bot in 6 studies (16.7%) and in 2 studies each (5.6%) Cubelets, KIBO and Dash were used. In the remaining 14 studies different systems were used.

Furthermore, in 50% of the studies GUIs were used, in 47% TUIs and in 3% a combination of the two, as shown in Figure 5.

Table 8. Study characteristics.

ID	Subjects	Intervention Objectives *	Duration			Activities **	Group Size
			Min	Sessions	Sum Min
1	Computer science	K	45	1	45	P	2–3
2	Physics	K, A	50	15	750	E, P	3–4
3	Computer science	S	40	2	80	P	individual
4	Mathematics	K	2019 school year			P, M	3–5
5	Computer science	S, A	45	2	90	P	2–3
6	Robotics	K, S	120	4	480	E, P	4
7	Robotics	S, A	120	10	1200	E, P	3–4
8	Robotics	K, S	60	10	600	E, P	3
9	Computer science	S, A	75	13	975	P	3–4
10	Mathematics	K	5	3	15	M	individual
11	Computer science	K, S	60	8	480	P	individual
12	History	K, A	80	1	80	P	individual, pairs
13	Computer science	S	45	8	360	P	individual
14	Technology	A	October to May once a week			E, P	2–4
15	Robotics	K, S	60	7	420	E, P	pairs or small groups
16	Bioengineering	A	180	3	540	E, P	2–4, 4–6
17	Robotics	K, S	60	8	480	E, P	2–3
18	Technology	A	1-day intervention at school			E, P	maximum 15
19	Robotics	K	35	9	315	E, P	ns
20	Robotics	S	35	9	315	E, P	4
21	Computer science	K	75	8	600	P	2–5
22	Computer science	A	20	1	20	P	individual
23	Robotics	A	2016–2017 school year			E, P	4, 8
24	Robotics	S, A	120	20	2400	E, P	individual, 2–3
25	Robotics	S	60	8	480	E, P	5
26	Mathematics	K	2017–2019 school years			M	individual
27	Computer science	K	60	4	240	P	2
28	Robotics	S, A	fall semester of 2018–2019 academic year			E, P	individual
29	Robotics	S, A	240	31	7440	E, P	3
30	Robotics	S	120	60	7200	E, P	ns
31	Robotics	A	60	7	420	E, P	ns
32	Physics	K, A	90	16	1440	E, P	3
33	Computer science	K	ns	ns	ns	P	individual
34	Mathematics	K	60	5	300	P, M	individual
35	Computer science	K	ns	ns	ns	P	individual
36	Robotics	K	180	6	1080	E, P	4

* K: knowledge, A: attitudes, S: skills. ** P: programming, E: engineering, M: mathematics.

lists the study characteristics: subject, intervention objectives, study duration, activities and group size. The subject in 15 studies (42%) was robotics, in 11 studies (31%) computer science and in 4 studies (11%) mathematics.

The intervention objective in 11 studies (31%) was knowledge, in 6 studies (17%) attitudes and in 5 studies (14%) skills. In six studies (17%), there was a combination of skills and attitudes, in five studies (14%) a combination of knowledge and skills and in three studies (8%) a combination of knowledge and attitudes.

The distribution of the study duration is shown in Figure 6. The study duration peaks before 500 min, while only seven studies (19%) exceed 1000 min.

For the study activities, in 20 studies (56%) the activities were a combination of engineering and programming, in 12 studies (33%) they were programming, in 2 studies (6%) mathematics and in the remaining 2 studies (6%) programming and mathematics.

Moreover, Figure 7 shows the number of participants within a group. It can be observed that the children worked in groups of 3–4 in half of the studies.

Table 9. Intervention objectives and studies.

Intervention Objectives	ID
Knowledge	1, 2, 4, 6, 8, 10, 11, 12, 15, 17, 19, 21, 26, 27, 32, 33, 34, 35, 36
Skills	3, 5, 6, 7, 8, 9, 11, 13, 15, 17, 20, 24, 25, 28, 29, 30
Attitudes	3, 5, 7, 9, 12, 14, 16, 18, 22, 23, 24, 28, 29, 31, 32

presents the intervention objectives of each study. We can see that in 19 cases (53%), the objective was knowledge, in 16 cases (44%) skills and in 15 cases (42%) the objective was attitudes.

Table 10. Data and results.

ID	Data Type *	Data Source **	Results
			Proved	Non-Proved
1	q	V	Knowledge	-
2	Q	Q	Knowledge, Attitudes	-
3	Q	Q	Skills	-
4	Q	Q	Knowledge	-
5	Q + q	Q, O	Skills, Attitudes	-
6	Q	Q	Knowledge, Skills	-
7	Q	Q	Skills, Attitudes	-
8	Q	Q	Skills	Knowledge
9	Q	Q	Skills	Attitudes
10	Q	Q	-	Knowledge
11	Q	Q	Knowledge, Skills	-
12	Q + q	Q, O	Knowledge, Attitudes	-
13	Q + q	Q, O	Skills	Skills
14	Q + q	Q, V, I	Attitudes	-
15	Q + q	Q, I	Knowledge, Skills	-
16	Q + q	Q, O	Attitudes	-
17	Q	Q	Knowledge, Skills	-
18	Q	Q	Attitudes	-
19	q	V, I	Knowledge	-
20	q	V, I	Skills	-
21	Q	Q	Knowledge	-
22	Q	Q	Attitudes	-
23	Q	Q	Attitudes	-
24	Q + q	Q, V, I, O	Skills, Attitudes	Attitudes
25	Q + q	Q, I	Skills	-
26	Q + q	Q, I	Knowledge	-
27	q	V, I	Knowledge	-
28	Q	Q	Skills, Attitudes	-
29	Q	Q	Skills, Attitudes	-
30	q	I, O	Skills	-
31	Q	Q	Attitudes	-
32	Q + q	Q, I	Knowledge, Attitudes	-
33	q	O	Knowledge	-
34	Q	Q	Knowledge	-
35	q	V, O	Knowledge	-
36	Q	Q	-	Knowledge

* Q: quantitative, q: qualitative. ** V: video, I: interview, O: observation, Q: questionnaire.

lists the data type and source and the intervention results for each study. The data type in 19 studies (52.8%) was quantitative, in 8 studies (22.2%) qualitative and in the remaining 9 studies (25%) quantitative and qualitative, as shown in Figure 8. As a source, questionnaires were used to record data in 52.8% of cases. Interviews were used in 18.9%, observations of researchers were used in 15.1% and video was used in 13.2% of cases.

4. Discussion

In this section, the results of the systematic literature review are examined, in an effort to find answers to our RQs.

Research Question 1. (RQ1). Which study design is commonly used in the STEM interventions? The majority of the researchers (67%) chose to use a non-experimental design. The review of Xia and Zhong [15] ended in similar conclusions, as in their findings 59% of the studies for ER in K–12 education were non-experimental. This may be related to the characteristics of the sample since we should not overlook the fact that in a study with young students, various difficulties and problems arise. For example, young students are easily distracted, thus, the use of a simpler design can facilitate research and lead to the successful completion of a study. On the contrary, the use of a more sophisticated and complex study design may lead to misleading results and study failure.

Based on our analysis, we have found that 23 out of the 36 studies included measurements and/or observations before and after the intervention. In these cases, there was a direct comparison of the impact of the intervention on students adding enhanced reliability to the findings. However, 13 out of the 36 study designs included measurements and/or observation only after the intervention. Therefore, in these studies, the effect of the intervention cannot be easily justified and supported since there is no corresponding data before the intervention.

Based on the above, it seems that the researchers avoided experimental design by not dividing the participants into groups. This prevented them from investigating the effects of the intervention on each group. To overcome this gap, they used pre/post-test/observation in order to give credibility to the results.

Research Question 2 (RQ2). What are the characteristics of the sample? We can see that the examined articles did not take place in South America, Africa or Oceania. In addition, Asia plays a small part, with only three countries. In contrast, most studies were implemented in North America. For several years, the USA has been investing in new technologies and education [57], and perhaps that is a reason for the flourishing of STEM programs in that country. However, this conclusion may be related to the keywords we used in our searches. In other words, through new searches and keywords in the same databases, articles from other countries might have emerged.

From the findings, it can be concluded that 2/3 of the participants were 7 years old or older. This is expected, as according to the intellectual developmental stages of Piaget, children from the age of 7 enter the concrete operational stage and can think in a more organized and logical way [58]. Consequently, STEM activities seem to be more suitable for these students. In any case, it looks like more research is needed to clarify this assumption.

Furthermore, the sample size did not exceed 50 students in the majority of the studies. The number of boys and girls shows a similar distribution with small variations. Although in some cases “boys reported significantly higher motivation than girls” [42], according to Sullivan and Bers [51] “the robotics curriculum impacted girls’ interest in engineering enough that they were just as interested as boys by the end of the intervention”. In addition, according to Zviel-Girshin et al. [34] “the majority of both boys and girls consider robotics education as fun and want to continue their robotic education in the next school year”. Therefore, it is important that both genders have the same opportunities in STEM education, without exclusion due to prejudices or stereotypes. However, the distribution of the sample size raises concerns about the reliability and depth of the studies, since a small sample size cannot lead to generalized conclusions. The review of Xia and Zhong [15] ended with similar conclusions as in their findings, up to 40 students participated in the interventions.

Finally, one out of five primary education students re-participated in similar STEM programs, which indicates the penetration of such technologies in education.

Research Question 3 (RQ3). Which equipment and user interface are used? Our findings show that GUI and TUI were used almost equally. Nevertheless, recent studies show that students express their preference and a positive attitude towards TUIs, as GUIs might create boundaries in their cooperation [59,60]. Based on our results, only in one case article were both interfaces used. In this unique case, using the creative hybrid environment for robotic programming (CHERP) children created tangible physical programs using interlocking wooden blocks and simultaneously were able to create programs onscreen using the same icons that represent commands to control their robots [56]. In the literature, studies combining interfaces are quite limited, yet it may be worthwhile to include such approaches in future designs, to gain a deeper knowledge of the effect of different interfaces on the STEM field.

In addition, from the results, we can observe a preference for the use of wheeled robotic systems. This is probably because researchers using cars or moving mechanisms might easily attract students’ interest and motivate them to engage with STEM topics.

Research Question 4 (RQ4). What are the characteristics and how were studies’ data recorded? According to the findings, we see that the main studies’ subjects are robotics and computer science. This finding contradicts Benitti [14] as his results “show that most of the studies (80%) explore topics related to the fields of physics and mathematics”. In addition, the use of robotic systems for teaching subjects that are not traditionally related to STEM topics (science, technology, engineering, mathematics), such as history (e.g., [32]), is noteworthy. This finding agrees with Anwar et al. [16], in which it is conceded that ER can be used as a learning tool for other sciences and domains like language and argumentation thinking.

Moreover, an important element of any educational process is the cooperation and learning of students in groups. In the majority of the studies, the students worked in groups, although in a significant proportion of studies (24%), the children participated in individual activities. As a result, they may have missed the opportunity to gain the benefits of collaborative learning [61].

The researchers, in more than half of the cases, preferred to record quantitative instead of qualitative data. Questionnaires were used as a measurement tool, although in some cases might have not been validated. In this way, researchers collected sufficient data to make statistical analyses. These analyses showed that in most cases, students who attended the ER activities had significantly improved results compared to students who participated in traditional activities. Therefore, it seems that the use of robots in short-term studies can offer significant benefits, at least in the first stage of the educational process. This conclusion might not be valid in long-term research. We argue that the sample size and duration of the examined studies are too limited to safely generalize the results, so long-term research is recommended in order to collect results with more depth and quality.

Finally, we observe that the three intervention objectives (knowledge, attitudes and skills) are investigated by researchers almost equally. A meta-analysis would be an interesting and challenging future research proposal, to find out which intervention objective is the most beneficial with the use of ER.

5. Conclusions

STEM research has been part of education research for years. Each researcher aims to explore specific topics or features. Thus, from the plethora of studies, several reviews have emerged. However, there is limited knowledge about STEM research in students under the age of 12, so the purpose of this article was to explore the features of STEM research in primary education. Therefore, a systematic literature review was conducted and our findings showed that the study design usually contained pre/post-intervention evaluation. Researchers also seem to prefer non-experimental designs. Most studies were conducted in primary schools and the number of girls and boys did not differ significantly. However, the overall sample was quite limited in size. Likewise, the duration of the studies was limited. The use of GUI and TUI was equal, while wheeled robotic systems were preferred. The researchers also preferred to form groups of 3–4 students; however, in several cases individual activities were used. Finally, in half of the cases, the data were recorded with questionnaires.

The findings of our work can be used in future research designs. More specifically, all of the above should be taken into account by a researcher to prepare their own intervention. Additionally, educators can use ER as an educational tool based on this article, enriching their teaching approaches.

Finally, although studies on STEM research in primary education have increased in recent years, there is still the need to increase the sample size and study duration, so that the findings can more easily be generalized.