Validation and Psychometric Properties of the Gameplay-Scale for Educative Video Games in Spanish Children

Abstract

The knowledge of evaluation instruments to determine the level of gameplay of schoolchildren is very important at this time. A systematic review has been carried out in this study. The aim of this paper is to investigate the psychometric properties of a study of a sample of Spanish gamers. Two hundred and thirty-seven children (mean age: 11.2± 1.17 years, range: 10-12 years, 59.5% female) completed the Gameplay-Scale to discover their opinions after a game session with a serious educational game. The final scale consisted of three factors. The fit for factor 1 (usability) was 0.712, the fit for factor 2 was 0.702 (satisfaction), the fit for factor 3 was 0.886 (empathy) and the overall fit was 0.868. A positive and direct relationship could be observed between all the dimensions of the developed scale. The greatest correlation strength is shown between satisfaction and empathy (r = 0.800; p < 0.005), followed by satisfaction and usability (r = 0.180; p < 0.05) and the association between empathy and usability (r = 0.140; p < 0.05). In summary, the results of the present study support the use of the Gameplay-Scale as a valid and reliable measure of the game experience of youth populations. These results demonstrate strong psychometric properties so that the Gameplay-Scale appears to be a valid instrument for children in different contexts where an educational video game is used, analyzing its usability/“playability” in terms of learning to use it, game satisfaction, and empathy.

1. Introduction

In recent years, there have been many changes in society, most of which are related to the digital phase that has invaded all areas of life. This is especially marked in the younger sectors of the population, resulting, in some cases, in new forms of fun and entertainment. In particular, video games are a popular type of entertainment software that incorporates principles from multimedia and gameplay, which are being used with the additional purpose of improving certain areas of knowledge and skills. Specifically, in education, serious games are becoming important instructional tools and are increasingly being integrated into courses in many different areas to help individuals concentrate on the subject matter and enjoy learning [1]. Serious games offer new learning opportunities, providing challenging tasks that promote more autonomous and exciting instruction. Their flexibility in use and scalability makes them an attractive pedagogic instrument, but it is necessary to check the effectiveness of serious games compared to more traditional formats. With this aim, in Dankbaar et al. [2], a comparison with e-modules was performed, concluding that video-lectures (in a game) and text-based lectures (in an e-module) can be equally effective, although serious games are more motivating in the long term.

Numerous efforts have been made to evaluate the effectiveness of serious educational games and a large number of educational advantages has been revealed [3,4,5,6,7]. However, as the demand for this type of game increases, game developers face the challenge of creating games that are not only effective but also attractive to play. Since there are different opinions on the essential elements that constitute a “good game” [8], it is not easy to determine which criteria should be analyzed to assess the suitability of a video game and, especially, of a serious educational game. It is exceedingly hard to adequately describe and measure a gaming experience because there is a great variety of games and the kinds of experiences that users will have when playing these games will differ considerably [9].

As a consequence, in order to adequately judge the impact of the design of the game on the user experience, traditional usability metrics must be redefined to incorporate the game elements in an overlay model.

The systematic review of the literature, described in the next section of this work, shows that, although many creators of video games and in particular serious educational games have conducted evaluations with significant user populations, there is currently no standard questionnaire to extract the opinion of the users about their experiences during a game session. Existing questionnaires are partial, dependent on the area of application, and are often too complex. To fill this gap, we propose a very simple questionnaire (Gameplay-Scale) that can be used by children and which has been used to evaluate a serious game for comprehensive reading in an educational experience carried out with 237 students between 10 and 12 years old.

Regardless of the effectiveness of the game (which will require evaluation methods specific for the purpose of the game, in our case pre-tests and post-tests of comprehensive reading), the opinion questionnaire about a serious educational game should consider several factors, such as: The difficulty perceived by the users in learning to play the game, the ease and pleasantness of the game environment, the satisfaction produced in the player about each element of the game, etc. With this aim, a questionnaire (Gameplay-Scale) to discover the opinion of the users of an educational game in terms of learning about the game, ease of use, satisfaction about graphics, sounds, interaction mechanisms, story and narrative, characters, challenges, rewards and empathy is presented, applied, and validated in the rest of this paper. Additionally, two questions were included in the evaluation experience to analyze the player immersion understood as the degree to which the serious component of the game, that is, the learning process, is implicit, and therefore, goes unnoticed by the player.

2. Related Work

We conducted a systematic search in the main collection of the Web of Science to identify the efforts made in the last ten years (2008 to 2018) in relation to the evaluation of video games and serious games. Finally, our interest will focus on serious educational games, but this broader vision will help. Concretely, the search for studies comprising a “questionnaire”, “poll”, or “survey” to discover the opinion of the players of the games retrieved over 621 results (within the categories Science Technology and Social Sciences).

This search was later refined to select studies including some of the following topics: a) “Human computer interaction” (or “human–computer interaction”), b) “validation” and/or “evaluation” of a proposal, c) quality attributes of the video game, such as: “Usability”, “ease of use”, “learnability”, “effectiveness”, user “satisfaction”, “playability”, “empathy”, and “awareness of learning”, and d) elements of the game, such as: “Graphics”, “sound”, “story”, “characters”, “avatar”, “challenges” and “scoring” (or “score”). The number of studies taking into account some of these topics (or a combination of them) was reduced from 621 to 252 papers. The title of each paper in this set of 252 papers was analyzed to determine its suitability to be part of our review. This filtering process allowed us to discard all those studies where, through the title, it seemed clear that the contribution of the paper was not concerned with evaluating a specific game nor a set of them nor even the concept of a video game itself.

Studies focused on the analysis of physical or psychological aspects of the player were also ignored. As a result, 84 papers were selected and their abstracts were read. During this systematic search of the scientific literature, some topics unrelated to our object were excluded: Video game addictions, violence, health, teachers’ opinion about the educational use of the game, etc. Exactly 34 papers were selected based on the abstract. Finally, after a complete reading, 30 studies were found that deal with the evaluation of a video game or serious game from the perspective of discovering the opinion of its players after the experience of playing the game.

Of the 30 reviewed studies, more than half (17 papers) refer to serious educational games. Nevertheless, all proposals of evaluation questionnaires for video games and serious games were analyzed to get a broader view. During our study, it was detected that there is a great diversity of proposals to evaluate the experience of using a game, but there is no standard. Since there is no widely accepted evaluation test, we tried to classify the different approaches found. The aim of this effort of classification is to identify common points among the existing proposals and to facilitate the work of future researchers/educators who need to face this task.

Table 1. Proposals of evaluation for serious educational games (focused on learning).

Work	Main Objective	Population	Evaluated Attributes	How is the Usability Evaluated?
Johnsen et al. [3]	To obtain the educational value in terms of face, content, and construct validity	Second-year nursing students	Realism/authenticity (face validity), alignment of content with curricula (content validity), ability to meet the learning objectives (construct validity), usability and preferences regarding future use.	Learning to use, use, engagement and likeability
Tan et al. [5]	To validate the effectiveness of a serious game in improving the knowledge and confidence of nursing students on blood transfusion practice	Second-year nursing students	Effectiveness of learning with the game, immersion, teacher-learner interaction, learner-learner interaction, imagination, motivation, enhanced problem-solving capability.	-
Dankbaar et al. [2]	To evaluate the effectiveness of serious games compared to more traditional formats such e-modules	Fourth-year medical student	Knowledge, self-efficacy, motivation, stress and patient safety awareness	-
Yap,Yap, Bin Abdol Ghani, Yap, and Athreya, [10]	To develop and evaluate an in-house role-playing game simulating various patient encounters in a futuristic post-apocalyptic world	Pharmacy undergraduates	Effectiveness of learning, comparison with lectures, preferences of game elements (storyline, resources, item grants, plot animations, vitality and life bars, experience points), and person perspective (collaborative and competitive aspects)	-
Blažič, Cigoj and Blažič [11]	To develop and evaluate a game for digital forensics training, incorporating learnability properties	Postgraduate students	Competency and cognitive abilities (knowledge, comprehension, application), expectations and motivation (confidence, satisfaction, relevance) and interaction and control	Interaction and control (attractiveness, being informed about the learning objectives, stimulating, guidance and feedback)
Callies, Sola, Beaudry, and Basque, [12]	To evaluate a simulation game developed following a proposed architecture for automatic adaptation	College students	Learning and motivation (feeling of challenge, curiosity, control, feedback, focus, immersion, relevance)	Challenges (difficulty of challenges, sensation of game adaptability, sensation of pressure and fair challenges) and control (if the game permits making different choices)
Martin-Dorta et al. [13]	To present and evaluate a novel spatial instruction system for improving spatial abilities of engineering students	Engineering students	Improvement in spatial skills, interest generated by the game, recommendation to others and usability	Help, system speed, navigation through scenes, easy to learn, capable of solving task, interface friendliness
Knight et al. [4]	To evaluate the effectiveness of a serious game in the teaching of major incident triage by comparing it with traditional training methods	Medical students	Effectivity of the learning (tagging accuracy, step accuracy and time)	-
Duque, Fung, Mallet, Posel and Fleiszer [14]	To create a serious game to teach Geriatric House Calls to medical students	Medical students	Improvement in students’ knowledge using the game, students’ perceptions about the geriatric home visit and the use of video gaming in medical education	-

and

Table 2. Proposals of evaluation for serious educational games (focused on experience of use).

Work	Main Objective	Population	Evaluated Attributes	How the Usability is Evaluated?
Wang et al. [1]	To determine the relations between the perceived usefulness, the perceived ease of use and the perceived goal clarity regarding concentration and enjoyment	College students	Usefulness and ease of use (based on Malhotra and Galletta [15]), goal clarity (based on (Guo and Klein [16]), concentration (based on (Koufaris [17]) and enjoyment (based on Agarwal and Karahanna [18]).	Ease of learning to use, flexibility of interaction, ease of use, ease of achieving objectives, clarity and comprehensibility of interaction (based on Malhotra and Galletta [15])
Johnsen, Fossum, Vivekananda-Schmidt, Fruhling, and Slettebø [19]	Design, development and usability evaluation of a video-based serious game for teaching clinical reasoning and decision	Nursing students	Electronic Health Records Usability (based on Malhotra and Galletta [15]).	Usefulness (accomplishment of goals), usablility (easy to learn, easy to use, and error-tolerant) and satisfying (subjective impression of the users)
Butler and Ahmed [20]	To compare the user enjoyment in learning through a serious game for learning Computer Science as opposed to traditional classroom learning	Computer science students	Preferred learning methods, difficulty of Computer Science concepts, motivation using the serious game and fun using the game	-
Kleinert et al. [21]	To design and evaluate a simulator for training in surgery	Medical students	Acceptance (fun, frequency of use, use of computers, preferred learning medium), effectiveness and applicability	Ease of learning, ease of use and overall impression
Lino et al. [22]	To design and evaluate a serious educational game that aims to assist inexperienced software project managers to be trained	Software development professionals	Relevance, confidence, satisfaction, immersion, challenge, skills and competence, fun and knowledge (phases of the game and matters of software project management, motivation)	Difficulty in understanding the game, difficulty of activities, difficulty of materials and content (confidence)
Lorenzini, Faita, Barsotti, Carrozzino, Tecchia, and Bergamasco [23]	To provide and evaluate a technological tool for both evaluating and training ethical skills of medical staff personnel	Hospital residents	Sense of presence (control, sensory, distraction, realism) (based on (Witmer and Singer [24]) and Usability	System Usability Scale (SUS) (Brooke, [25])
Dudzinski et al. [26]	To identify a successful game design for a serious multiplayer game to be used in learning	Pharmacy students	Game preferences, games usage, study patterns, views about educational games and learning, and opinion about a prototype of a serious game about pharmacy challenges	Usefulness, interactivity, confidence, speed and feedback
Canals, Font, Minguell and Regàs [27]	To create and evaluate a serious game called “Legends of Girona” to be used in the classroom as a teaching resource	Young people (between 10 and 16 years)	Aspects of the game: images, graphics, scenes, characters, game plot, graphics, voices, music, texts; and use of the game	How the game works, playability, game stability, use of commands and controls, instructions given, speed, treatment of failures

summarize the performed classification, focusing only on the proposals of evaluation for serious educational games. Table 1 describes the proposals of evaluation questionnaires that put the emphasis on the learning generated as a result of the use of the game. Table 2 describes the proposals of evaluation questionnaires that deal very closely with the user experience during the game. In both cases, the following information is included: Main objective of the proposed evaluation questionnaire, type of users who have answered the questionnaire (population), attributes considered in the items of the questionnaire, and attributes used in the questionnaire to evaluate the usability of the game (Table 1 and Table 2).

During the complete review of the 30 papers, we found some previous models that can be useful to define certain aspects of an evaluation questionnaire for video games (or serious games). For example, in Agarwal and Karahanna [18], an interesting, multidimensional conceptualization of cognitive absorption about information technologies usage is presented. This construct is exhibited through five dimensions: Temporal dissociation, immersion, enjoyment (pleasurable aspects of the interaction), control over the interaction, and curiosity. In addition, a nomological net is defined, trying to capture relations among the cognitive absorption and other factors, such as personal innovativeness, playfulness, self-efficiency, perceived usefulness, perceived ease of use, and behavioral intention of use.

This model has been used (with adaptations) in Wang et al. [1], however, the multidimensional construct is focused on cognitive absorption and is not specific to video games. In a similar line, in Malhotra and Galletta [15] a multidimensional commitment model of volitional systems adoption and usage behavior is presented (also used in Wang et al. [1]). In this model, the commitment to system use appears related to perceived usefulness, perceived ease of use, and attitudes toward using and behavioral intent. Because voluntary systems require users’ volitional behavior, this model may be applicable to video games. However, again this proposal does not cover the particularities of this type of ludic system. A complementary approach is the presence questionnaire proposed in Witmer and Singer [24] to measure the subjective experience of being in one virtual environment. This questionnaire has been adopted in Lorenzini et al. [23] and its items are organized into six subscales: Sensory exploration, involvement, interface awareness, control responsiveness, reality/fidelity, and adjustment/adaptation. The questionnaire addresses the sensation of presence during the user experience in the virtual environment in a comprehensive manner, but does not include the rest of the elements of a video game: Story, challenges, characters, etc.

Additionally, a few existing models to evaluate usability were found during the review. For example, TURF [28] is a unified framework of EHR (Electronic Health Records) usability. Under TURF, usability is understood through three dimensions: Usefulness, usability, and satisfaction. This approach, followed in Johnsen et al. [19], is structured as a set of representative measures: Across-model domain function saturation, within-model domain function saturation, learnability, efficiency, error prevention and recovery, and user impression of how useful, usable, and likable the system is. Another interesting proposal in this line is the System Usability Scale (SUS) in Brooke [25], which is followed in Lorenzini et al. [23], Boletsis and McCallum [29], and Vallejo et al. [30].

SUS is a simple, Likert ten-item scale giving a global view of subjective assessments of usability. Evaluated aspects in SUS are: Easy to learn to use, easy to use, integration of functions and consistency, like using, and confidence. Again, none of these two approaches is specific to video games, so they need to be adapted to take into consideration the characteristics of this type of software.

Regarding models specific for video games, very few contributions can be mentioned. For example, in Norman [31], a review of two very interesting questionnaires is presented: Game Experience Questionnaire and Game Engagement Questionnaire. The Game Experience Questionnaire (GEQ) [9] involves usability, flow, and immersion and consists of three modules: Experiences during gameplay, social presence, and experiences once a player has stopped playing the game. Questions in each module deal with how the user feels (successful, bored, frustrated, challenged, etc.) and the only element of the game explicitly referenced is the story. The In-Game Experience Questionnaire (iGEQ) is the shorter in-game version of the Game Experience Questionnaire and contains 14-items, rated on a five-point intensity scale distributed between seven dimensions: Immersion, flow, competence, tension, challenge, negative affect, and positive affect [28]. For its part, the Game Engagement Questionnaire [32] aims to provide a measure of an individual’s potential for becoming engaged in video gameplay at differing levels, taking into account presence, flow, absorption, and dissociation.

In conclusion, during the systematic search, we reviewed very different proposals to evaluate the user experience during the utilization of a video game and also of a serious game (proposals for serious educational games are summarized in Table 1 and Table 2). Although we found a few already accepted models that can be used to define certain dimensions of the required questionnaires, there is no standard model or test for this evaluation task. In addition, most of the proposals found during the performed systematic review process are partial and are not specific to video games (much less for serious educational games). One of the most complete revised questionnaires is GUESS (Game User Experience Satisfaction Scale) [8]. GUESS includes attributes to evaluate: Usability/playability, narratives (characters, story, etc.), play engrossment, enjoyment, creative freedom, audio aesthetics, personal gratification, social connectivity, and visual aesthetic. In addition, this proposal defines usability centered on the game (for example, it includes ease of navigation, the manual to play the game, etc.). However, GUESS is not specific to educational games and consists of 55 items with nine subscales, so it does not seem appropriate for children. More precisely, only three papers were included in the systematic map focus on children: Canals et al. [27], putting the emphasis on aspects of the game [33] putting the emphasis on enjoyment and Harrington and O’Connell [34] with the focus on empathy and prosocial behavior.

Accordingly, a new questionnaire to evaluate the user experience during the game, Gameplay-Scale, is proposed in this paper. This proposal takes into account the specific elements of the game (story, characters, challenges, etc.) and the player’s perception during their interaction with the game, including how he/she identifies with their own representation within it (that is, the avatar). In addition, the level of awareness of the user about their own educational process (awareness of learning during the game) was consulted to the players to analyze the ludic–educational balance of the game. The questionnaire has been specifically designed to be suitable to be answered by children.

Thus, the literature shows the need to work with new tests in order to quantify the player’s experience during the game, knowing the degree of gameplay of video games and, accordingly, improving the attitude towards them. We aimed to perform confirmatory analysis (CFA) to confirm the factor structure obtained via exploratory factor analysis (EFA), and so have an assessment instrument in which validity is tested. Consequently, the main purpose of this study was to develop a valid and reliable Gameplay-Scale for child players.

3. Materials and Methods

3.1. Participants and Design

School students from three schools in Granada (Spain) participated in this descriptive and cross-sectional study. We analyzed 237 Spanish schoolchildren between 10 and 12 years old (11.2±1.17). This sample is representative of children in this age group from Granada (n = 20.606), finding the sample error at 0.04 with 95% confidence. Of these, 40.5% were male and 59.5% were female participants. Conglomerate sampling (a probabilistic sampling unit in the conglomerate) was used to select participants.

3.2. Measures

The 12-item Gameplay-Scale was used to assess the user experience of games in children through play “Invasores del tiempo” (versión PC the game: http://bios.ugr.es/~nmedina and versio Android App Store of Google Play: https://play.google.com/store/apps/details?id=com.UGR.LosInvasoresDelTiempo). This questionnaire was previously cited by Zurita-Ortega, Medina-Medina, Chacón-Cuberos, Ubago-Jiménez, Castro-Sánchez, and González-Valero [35]. This questionnaire of twelve items of closed questions is valued by means of a Likert scale of five options where 1 is totally in disagreement and 5 is totally in agreement. However, our target population requires a prior adaptation and validation as these validation tests do not exist for this cultural group and subjects.

Additionally, during the evaluation experience, two questions were used to analyze the extent to which the educational process is hidden from the player (awareness of the learning process), which is key to increasing the motivation, and therefore, the effectiveness of the game. These questions were: “Indicate to what degree you have noticed that you were learning” and “What have you learned with the game?” Only 8.01% of children detected that they were working on “comprehensive reading”, although the average degree of awareness of learning was 3.39. These results suggest an adequate ludic–educational balance and an adequate degree of immersion during the game.

3.3. Procedure

The inclusion criteria for the participants were that they were enrolled in Primary Education. A total of 273 students sent their informed consent through their parents. The questionnaires were administered during May 2017 (after they finished a game session with a graphic adventure for comprehensive reading by Lope, Arcos, Medina-Medina, Paderewski, and Gutierrez-Vela [36]). We eliminated 26 participants who did not complete the questionnaires properly. During the data collection, the investigators were present, and the participants were informed that the information was completely confidential and anonymous. Previously, authorization was obtained from the ethics committee of the University of Granada and authorization from educational centers, and informed consent was requested from all participants.

3.4. Statistical Analysis

First, the SPSS 24.0 program was used to determine the descriptive statistics of the data (mean, kurtosis, asymmetry, and standard deviation). Once these were verified, the FACTOR 9.3.1 program was applied by Lorenzo-Seva and Ferrando [37] to determine the psychometric properties and establish the confirmatory factorial analysis. Finally, the AMOS version 22.0 was used to establish the model of equations. To develop the statistical model, confirmatory factor analysis was first used to check the reliability and validity of the research tool. Next, relationships between all the latent variables and the categories of the model were identified.

Structural equation analysis was employed in order to analyze the relationships between satisfaction, usability, and empathy. The structural model was composed of three latent variables and twelve observed variables that enabled causal explanations to be obtained of the associations between both types of variables and their indicators. In addition, circles constitute the error terms and squares are the observed variables, as can be observed in Figure 1.

In this sense, satisfaction, empathy, and usability are exogenous variables and they are composed of different indicators that are the items of the instrument (Usability: V01, V02; Satisfaction: V03, V04, V05, V06, V07, V08, V09, V10; Empathy: V11, V12). It is important to highlight that the method of maximum likelihood (ML) was employed in order to estimate the parameters of the structural models. This method is invariant to types of scale and allows to establish that all variables have a normal distribution.

Finally, the model’s fit was tested in order to check the adjustment of the structural model with the empirical information obtained. For this aim, several indices were used: P, CFI, AGFI, GFI, and RMSEA. Following the statements established by Hu and Bentler [38] and Schmider, Ziegler, Danay, Beyer, and Bühner [39], values higher than 0.05 for P provide a good fit. For RMSEA, values below 0.1 show an acceptable model fit, while values below 0.8 indicate an excellent model fit. For the other indices, values higher than 0.95 indicate an excellent model fit, while values higher than 0.90 show an acceptable model fit.

4. Results

Table 3. Descriptive of “Gameplay-Scale”.

Items Gameplay-Scale	M	SD	V	A	K
V01. Indicate the degree to which it has been easy for you to learn to play	3.56	0.898	0.807	−0.185	0.250
V02. Indicate the degree to which it has been easy for you to play	3.46	0.875	0.766	−0.112	0.536
V03. Indicate to what extent you liked the artistic part of the game (graphics)	3.49	1.072	1.149	−0.315	−0.553
V04. Indicate to what extent you liked the artistic part of the game (sounds)	3.20	1.132	1.281	−0.229	−0.527
V05. Indicate how much you liked the story told in the game	3.73	1.118	1.251	−0.633	−0.340
V06. Indicate how much you liked the characters in the game	3.25	1.110	1.232	−0.308	−0.516
V07. Indicate how much you liked the challenges of the game	3.68	1.072	1.149	−0.589	−0.320
V08. Indicate to what extent you liked the way you interact with the game (pick and use objects, dialogue with characters, etc.).	3.34	1.056	1.114	−0.318	−0.318
V09. Indicate how much you liked the way the game is scored (using stickers)	3.33	1.090	1.188	−0.308	−0.476
V10. In summary, indicate to what extent you liked the game	3.81	1.046	1.095	−0.664	−0.025
V11. Indicate to what extent you understood how the protagonist feels	3.52	0.972	0.946	−0.416	−0.126
V12. Indicate to what extent you identified with the actions of the protagonist	2.97	1.219	1.486	0.014	−0.910

Note 1: M, Mean; SD, Standard Desviation; V, variance; A, Asymmetry; K, Kurtosis.

shows the descriptive findings from the test “Gameplay-Scale”, the normality of the data was determined through the Kolgomorov–Smirnov test. After following expert recommendations [38,39], none of the variables was removed (Table 3).

The psychometric properties of the 12 Gameplay-Scale items are shown in

Table 4. Rotated Factor Matrix and Load Factor Dimensions of Gameplay-Scale.

Variables	Rotated Factor Matrix			Variables	Load Factor Dimensions of GAMEPLAY-SCALE
	F1	F2	F3		F1	F2	F3
V 01	−0.104	0.842	−0.027	V 01		0.842
V 02	−0.022	0.660	−0.013	V 02		0.660
V 03	0.217	0.023	0.559	V 03			0.559
V 04	−0.079	0.071	0.623	V 04			0.623
V 05	0.074	0.035	0.712	V 05			0.712
V 06	0.134	0.020	0.566	V 06			0.566
V 07	−0.128	−0.081	0.831	V 07			0.831
V 08	0.052	0.081	0.606	V 08			0.606
V 09	−0.007	−0.083	0.699	V 09			0.699
V 10	0.233	−0.029	0.632	V 10			0.632
V 11	0.469	0.046	0.127	V 11	0.469
V 12	1.051	−0.028	−0.311	V 12	1.051
				Alpha (0.868)	0.712	0.702	0.886

. To obtain the results, the FACTOR Analysis program proposed by Lorenzo-Seva and Ferrando [37] was used. The KMO (Kaiser–Meyer–Olkin test) was 0.89, Bartlett index (1112.6 (df = 66; p < 0.001)), both values indicated a good fit for factor analysis. Three factors were rotated, obtaining a 63.03% variance. Likewise, the indexes show a GFI of 1.00, CFI of 0.97, AGFI of 0.99, and the RMSR of 0.029. All these data establish a very good fit for the analyzed data.

To establish the relationships between the three factors (usability, satisfaction, and empathy) a model of structural equations was made for the analysis of gameplay by children (

Table 5. Weights and standardized regression weights.

Relationship between Variables			R.W.				S.R.W.
			Estimate	S.E.	C.R.	P	Estimate
I.1	←	USA	1.000	-	-	***	0.681
I.2	←	USA	1.136	0.399	2.847	*	0.794
I.3	←	SAT	1.000	-	-	***	0.760
I.4	←	SAT	0.783	0.065	12.113	***	0.564
I.5	←	SAT	1.091	0.062	17.710	***	0.795
I.6	←	SAT	0.935	0.062	15.015	***	0.686
I.7	←	SAT	0.893	0.060	14.832	***	0.679
I.8	←	SAT	0.851	0.059	14.315	***	0.657
I.9	←	SAT	0.856	0.062	13.913	***	0.640
I.10	←	SAT	1.071	0.057	18.682	***	0.834
I.11	←	EMP	1.000	-	-	***	0.656
I.12	←	EMP	1.320	0.124	10.605	***	0.690
EMP	↔	SAT	0.414	0.046	9.076	***	0.800
SAT	↔	USA	0.090	0.036	2.524	*	0.181
EMP	↔	USA	0.056	0.030	1.867	*	0.144

Note 1: R.W., Regression Weights; S.R.W., Standardized Regression Weights; S.E., Estimation of Error; C.R., Critical Ratio. Note 2: EMP, Empathy; SAT, Satisfaction; USA, Usability. Note 3: *, p < 0.05; **, p < 0.01; ***, p < 0.001.

and Figure 2). The chi-square test was not significant at 0.05 (χ2 = 142,403; df = 51; p < 0.001), which suggests that the model fit was acceptable. Other data, such as the CFI = 0.959 and the IFI = 0.959, also indicate an acceptable adjustment, as occurred with the NFI = 0.938 and the RMSEA = 0.062. Therefore, once all the indexes were analyzed, a good adjustment of the empirical data towards the model could be made.

Analyzing the structural model, it was possible to observe that all items were positively associated with their dimensions, giving significant relationships at level p < 0.005. Likewise, a positive and direct relationship could be observed between all the dimensions of the developed scale. The greatest correlation strength is shown between satisfaction and empathy (r = 0.800; p < 0.005), followed by satisfaction and usability (r = 0.180; p <0.05) and the association between empathy and usability (r = 0.140; p < 0.05) (Table 5 and Figure 2).

5. Discussion and Conclusions

In this study, which had as its main objective the analysis of the psychometric properties and structure of the Gameplay-Scale, a new questionnaire to evaluate the player’s experience during the utilization of a serious educational game was adapted and applied. The adaptation of the Gameplay-Scale has been adequate, obtaining figures that show a very good factorial adjustment, good stability, and excellent internal consistency.

In addition, these data indicated the validity of the instrument to establish and analyze the gameplay in students. Based on a sample of 237 children using an educational video game, Cronbach’s alphas were all satisfactory. This is important because the period of childhood is considered to be essential for human development and behavior as patterns of behavior followed during this stage are often repeated in adulthood. It is, therefore, a vital period for developing adaptive types of motivation towards video games [40]. Moreover, weekend video gaming was positively associated with the academic performance [41]. Children and adolescents with a greater acceptance of new technologies are more likely to experience repeated satisfaction and pleasure from their engagement and will continue to engage in the future.

Additionally, this item was used previously to demonstrate the analysis of play in people who practice sports non-professionally [35]. Video games are elements that promote intrinsic and extrinsic factors in children. In particular, they promote motivation and the strengthening of relationships within the family. Thus, the findings suggest that exergames (i.e., active-play video games) could be a potentially viable option for families, especially during bad weather in winter. Intervention efforts could benefit from promoting the social aspects of family exergaming and attitudinal factors [42]. Others have discussed how the computer game-based methods that have been analyzed made it possible to distinguish the most significant features that can promote the development of executive functions at the preschool age [43].

The three dimensions (usability, satisfaction, and empathy) load properly in the three factors of the Gameplay-Scale. The structural model developed to discover the relationships between the factors of the scale revealed positive and direct relationships between all dimensions.

The greatest strength of relationship was found between satisfaction and empathy in the same way as found by Harrington and O’Connell [34] or Hainey, Connolly, Boyle, Wilson, and Razak [44], who show how positive moods and the feeling of well-being produced by satisfying a need are associated with affective participation closer to other people’s realities. In the same way, both empathy and satisfaction were directly associated with usability. It seems that ease of use and learning in the game directly influence satisfaction and wellbeing, where it is essential to consider the level of the player to avoid negative experiences with the serious educational game in order to promote adherence to it, the degree of learning, and to avoid premature abandonment due to demotivation [34,45]).

Consequently, this study makes contributions to the literature. In this way, it is important to indicate the promotion of the level of playability of a video game in the learning and practice of it. The satisfaction that videogames bring together with autonomy and personal improvement must be the objective of educational interventions.

This is linked to what Chen, Sun, Harrington, and O’Connell [46] and Thompson [47] propose with the acquisition of adherence to videogames to generate changes in behavior through the promotion of patterns of motivation in learning processes.

The study is not without its limitations. The sample was self-selecting and the data that were utilized were collected from Spanish children. Consequently, the findings are not representative of Spanish children and may not be generalizable to other countries. For validation studies, representativeness is not a major issue, and arguably, over-sampling excessive video game players is more important. However, the present authors were unable to find a way of over-sampling, such a group in the way permission was sought to recruit participants, so this is another limitation that needs to be taken into account.

Furthermore, the scale was designed to be used in epidemiological studies rather than in a clinical context, but given the nature of the instrument, in-depth exploration using items in the scale could be used in a clinical context to get a detailed picture of the psychosocial impact of video game playing on children’s lives.

Future investigations should examine other groups. It would also be interesting to try to establish links between students who are video game players and athletes. It would also be interesting to analyze it together with educational, emotional, or psychosocial variables.

In summary, the Gameplay-Scale is a valid questionnaire for the analysis of the user experiences during the game in Spanish schoolchildren and the psychometric properties of the same are a valid instrument for children in different contexts (cultural or socioeconomic). The validated scale is an effective instrument to assess some basic dimensions in educational video games for school-age children, such as usability, satisfaction, and empathy. Specifically, it shows how ease of use and learning in the game directly influence satisfaction and wellbeing, where it is essential to consider the level of the player to avoid negative experiences with the serious game in order to favour adherence, as well as to build affective participation closer to other people’s realities.