Personalized Exergames Language: A Novel Approach to the Automatic Generation of Personalized Exergames for Stroke Patients

Abstract

Physical rehabilitation of stroke patients is based on the daily execution of exercises with face-to-face supervision by therapists. This model cannot be sustained in the long term, due to the involved economic costs, the growing number of patients, and the aging population. Remote rehabilitation tools have emerged to address this unmet clinical need, but they face the double challenge of motivating patients and ensuring an effective remote rehabilitation. In this context, exergames allow patients to play while performing repetitive therapeutic tasks in a safe and ecological environment. This work proposes the design of Personalized Exergames Language (PEL), a language whose sentences can be processed via software in order to automatically generate exergames. The definition of exergames through PEL, guided by an effective methodology of the design and generation of personalized exergames, will include both game mechanics and the necessary metrics to monitor, guide, and adapt the rehabilitation of each patient. The integration of authoring tools are considered to visually guide the therapist when designing exergames. A study has been carried out with stroke patients and therapists from a hospital and two community centers, in order to evaluate several exergames, automatically generated using PEL, in terms of usability, understanding, and suitability.

1. Introduction

Therapeutic exergames enable patients to exercise by playing, thereby relieving the burden of repetitive tasks through the use of motivating and engaging game dynamics [1]. In this context, an exergame can be understood as a game that tries to mimic, within a virtual or augmented world, a physical exercise that is associated to a therapeutic treatment.

When it comes to using technological tools, based on exergames, either for remote or face-to-face rehabilitation, the therapists assign routines, which are composed of rehabilitation exercises, to patients. These exergames, previously created, do not usually allow the therapists to customize or adapt them directly. This can be due to a number of factors, including the dependency that is introduced by the used rehabilitation systems, the complexity of the process, or the issues raised should therapists have to deal with the technical aspects of exergame development.

In this context, this work addresses the automatic generation of personalized exergames for the physical rehabilitation of patients that are affected by stroke [2]. Although the proposal discussed in this work can be generalized to other neurological diseases, the focus is on stroke, which remains an unmet clinical need nowadays [3]. The incidence of this disease is increasing worldwide, especially in low- and middle-income countries [4]. Additionally, the ideas of personalized and universal access to telerehabilitation [5] are part of the core of the proposal, which aims at contributing to personalized healthcare technologies and improving the flexibility that is provided in terms of physical location, used hardware devices, interaction mechanisms, and multimedia content [6].

A formal language, named Personalized Exergames Language (PEL), has been designed and developed along with a software system thta is able to process PEL sentences. PEL allows for the specification of exergames, while the software system is capable of analyzing PEL sentences and generating exergames automatically. The use of PEL is based on a methodology that is also devised in this work. Such methodology is aligned to the idea of adapting the physical rehabilitation process for each patient. This methodology, which guides the workflow when using PEL to define exergames, consists in the following steps: (i) definition of an exercise from the therapist’s point of view, (ii) definition of an interaction scheme on behalf of patients, (iii) definition of feedback mechanisms for guiding and keeping patients motivated when they make rehabilitation, and (iv) the definition of the metrics that the therapists use to monitor the patients’ progress. All these definitions can be specified by means of PEL. Thus, the whole proposal aims at laying the foundations to build software systems that are able to automatically generate exergames and dynamically customize the patient’s rehabilitation routine as they improve their performance.

In order to implement PEL, the GL Transmission Format (glTF) specification [7] has been adopted. glFT is an open standard, which is based on the JSON format, conceived to store and transmit 3D model information. glTF was chosen to its maturity level as an interoperable and extensible format when handling and integrating 3D contents. Its capacity to describe scenes with the JSON format and easily extend it were strong reasons for this decision. glTF syntax not only allows to define exergames, but it also enables interactions between the virtual nodes of an exergame that has been customized for rehabilitation. Additionally, the glTF extensions let properties, semantics, and even additional formats be added. On the other hand, the use of JSON messages allows for minimizing the size of 3D assets and the impact on the efficient transmission and loading of 3D models and scenes into applications.

The use of PEL as the core component to define exergames is proposed in order to provide a resource that therapists can use to create exergames. It must be noted that, even though the current version of PEL is simple enough to understand how to specify exergames, its direct use on behalf of therapists is not currently recommended without the support of developers. Nevertheless, PEL has been designed to be integrated with high-level authoring tools that facilitate the generation of personalized exergames. These tools will integrate a software module to generate the PEL sentences that represent the graphic definition of the exergame made by the therapist. This work also addresses the initial design of a high-level graphical tool to provide this support for therapists.

The research work carried out in this article is focused on the benefits that the definition of a language to personalize exergames, which can be automatically generated, may have in the physical rehabilitation process of stroke patients. Particularly, the generated exergames must engage and motivate them. At the same time, these exergames must be useful, easy to use, and increase the intention of use of the developed rehabilitation system. From the therapist’s point of view, a benefit should be also achieved thanks to the customization and monitoring capabilities that are offered by PEL. To validate the proposal, and with the aim of evaluating the current capabilities of PEL as a basis for the generation of personalized exergames, a study has been conducted with 17 patients and six therapists. Thus, the main contributions of this article are as follows:

• Firstly, a methodology is proposed for the design and generation of customized exergames. This methodology addresses the therapeutic goals of the rehabilitation process, the enabled interaction schemes for the patients to interact with the exergames, the feedback and motivation mechanisms to engage patients by means of gamification techniques, and the definition of metrics to keep track of the patients’ progress.
• Secondly, a language, named PEL, is designed and developed for the creation of personalized exergames, whose sentences can be processed to automatically generate exergames. The lexical, syntactical, and semantic aspects of PEL are presented.
• Thirdly, the foundations for building computing systems that are able to process PEL sentences are established. This base can be used not only to automate the generation of exergames, but also to facilitate the automatic adjustments of rehabilitation exercises that are defined by means of PEL. A design proposal of a graphical tool to support the definition of exergames by therapists is also addressed.

The rest of the paper is structured, as follows. Section 2 provides a description of relevant related work within the scope of the proposal. Subsequently, Section 3 introduces the proposed methodology and describes in depth PEL, the devised language for personalizing exergames. Section 4 deals with a case study in which PEL is used to define a number exergames for stroke patients. These exergames are evaluated by stroke patients and therapists from one hospital and two community centers. Section 5 validates the proposal and discusses the major conclusions that were obtained after evaluating the use of PEL as a core component to define exergames and the rehabilitation system that runs the generated exergames. Finally, Section 6 presents the final conclusions and outlines future research lines.

2. Background and Significance

At present, stroke remains a global challenge that must be tackled from a prevention perspective [8]. For patients, physical rehabilitation is essential to regain mobility and improve their quality of life. Within this framework, tertiary digital prevention, by means of technology, attempts to mitigate the effects of neurological diseases, such as stroke. By way of example, in Europe alone, there are several million stroke survivors who need rehabilitation and, unfortunately, this number is only expected to rise in the coming years.

Traditionally in health care, rehabilitation consists pf daily sessions of exercises, in which therapists supervise patients in real time. Accordingly, in financial terms and in accordance with the European Brain Council (https://www.braincouncil.eu/wp-content/uploads/2018/04/Brain-Mission-Final-v2.pdf), the estimated cost to European healthcare systems has risen to 800 billion euros per year. In countries, like Spain, the average cost of treating a patient with ictus by specialist units is estimated at 27,711 euros per year [9]. This problem can be extrapolated to a worldwide scale and it is especially relevant in low- and middle-income countries [4]. Thus, this situation has led to medical needs that have yet to be covered [3].

That is why telemedicine tools, especially those that are designed for remote rehabilitation, have attempted to meet this challenge. In this context, the exercises performed by patients can be analyzed by a machine [10], using physical sensors for tracking joints [11,12] or computer vision techniques [13]. The latter have been traditionally based on using Kinect [14,15], a low-cost hardware solution whose effectiveness has been proved in the field of physical rehabilitation [16,17,18]. Using entertainment and fitness-orientated commercial games has also been tried out in order to provide home rehabilitation [19,20], although some studies indicate that there are a few adversary effects of this as the games may lack customization in terms of physical rehabilitation [21]. Some approaches are even focused on specific brain disorders, such as Parkinson’s disease [22].

The aim of a study into the psychological effects of playing exergames is to measure how committed players feel [23]. Because the patient motivation is essential for rehabilitation at home, some authors have put forward gamification and serious games as a way of motivating them [24]. In this respect, the design phase is crucial, not just because they can be supervised appropriately that way, but also to guarantee their efficiency and safety [25].

Regarding the specification of methodologies for creating and designing therapeutic exergames, in [26] a four-stage approach is discussed: (i) choosing an appropriate set of exercises for therapy, (ii) implementing the primary goals in virtual exercises, (iii) transforming virtual exercises into exergames, and (iv) including secondary objectives that are characterized by the game mechanics. This methodology is based on a game engine that has been specifically designed for supporting and integrating computer intelligence methods when monitoring patients [27].

Additionally, the state of the art covers more general-purpose solutions aimed at making it easier to generate or automatize customized exergames, such as the framework that is shown in [28]. In this research, the authors envisaged designing tools to support experts in the field, such as doctors or therapists, so that they can adapt and customize games-based training programs for elderly and disabled people. One of the strong points of the research discussed in [29], in which a platform with a range of functions for motor and social cognitive rehabilitation for hospitalized children is discussed, is that these contents can be edited by means of high-level graphic tools. Likewise, controlling emotions appropriately and feedback provided to patients is also dealt with in research that focuses on designing rehabilitation frameworks [30]. Plus, the process for creating contents within the exergames themselves (such as, for example, the dynamic appearance of game elements according to how patients are performing [31]) has also been addressed in the literature, mainly for the dynamic adaptation of the level of difficulty in the game to patient performance [32].

Ontologies have also been devised for describing exergames. The work discussed in [33] is a recent example, in which OWL Web Ontology Language is used for such a description and for defining the key components of exergames. Within this context, the authors also developed a framework in order to apply the created ontology to online exergames. On the other hand, grammars may also play an essential role to simplify the process of creating rehabilitation exercises. Thus, it is possible to define specific languages for specifying body postures and movements [34], which can be augmented thanks to editors that facilitate therapists’ work. In this contribution, an exercise-specification grammar is employed to guide the virtual avatar and provide quality feedback by comparing the user’s gestures against those performed by the coaching avatar. Related to these works, there are more general approaches, such as those discussed in [35,36], where the models may be used for guiding game designers to specify game design elements.

The work proposed in this article aims at providing the therapists with tools that can be used to personalize how patients employ rehabilitation systems and how therapists run the rehabilitation process. However, instead of designing and developing ad-hoc tools, we have devised a language that can work as the core component in multiple rehabilitation tools. This language provides sentences that can be used to design personalized exergames, and these sentences can be processed in order to automatically generate exergames for specific platforms and game engines. This is the major contribution of this work, which opens the door to deepening the idea of precision physical rehabilitation. On this basis, high-level authoring tools can be built that allow therapists to visually create exergames. Assuming that these tools export such a creation to a set of PEL sentences, then it is possible to cover the whole cycle of design and automatic generation of exergames. On the other hand, the definition of a language, and the capacity to process its sentences by a computer program, would facilitate the automatic adaptation of the rehabilitation routines that are defined by therapists.

3. Personalized Exergame Language

Before presenting the proposed language, named PEL, the devised methodology that guides the design and generation of customized exergames is introduced. An exergame, as previously defined in Section 1, can be understood as a game that mimics a physical exercise that is part of a physical rehabilitation routine. The exergame includes game dynamics to motivate and engage patients when making rehabilitation. PEL will be presented after discussing the adopted methodology, paying special attention to the lexical, syntactical, and semantic aspects of the language.

3.1. Proposed Methodology for Designing and Generating Exergames

Figure 1 graphically shows the steps that are required to effectively apply the proposed methodology for the design and generation of customized exergames. The major reference for our proposal is discussed in [26]; however, the one proposed in this work is particularly tailored when considering PEL, the designed language for defining exergames.

• Definition of exercises. The therapist establishes the therapeutic objectives of the rehabilitation process, considering the parts of the patient’s body involved in this process and the skill to be trained (muscle strength, balance or mobility, among others). At this point, the therapist establishes, at a high level, the temporal sequence of actions or steps that make up the rehabilitation exercise. Each element of the temporal sequence of actions can be evaluated independently and globally (when considering the exercise as a whole). This allows for the patient to obtain feedback on the exercise progress from both points of view. At the same time, and for each exercise, the therapist outlines the metrics that will be used to measure the patient’s performance when executing an exercise. Finally, the definition of the exercise is completed with the specification of constraints. This element allows the therapist to represent very common situations in physical rehabilitation processes, where patients try to compensate for the limitations of the body part to be rehabilitated with other fully functional parts of their body.
• Definition of the interaction scheme between the patient and the rehabilitation system regarding the exergame definition. The therapist establishes aspects, such as the position of the virtual avatar, the position of the virtual camera, the set of virtual nodes that make up the trajectory, or the specific actions of the rehabilitation movement. All of these elements are associated with the type of rehabilitation exercise and the affected parts of the body. Currently, a finite set of positions of the virtual avatar representing the patient (standing, seated, and lying-down) is assumed. On the other hand, the virtual path of the movement to be executed by the patient in the exergame consists of a set of ordered lists of virtual nodes or points in the three-dimensional (3D) space. The interaction in each exergame is defined by associating a node or a path with one or several joints. It is also possible to specify whether the interaction between a joint and a node is simple (simple virtual collision) or the patient must hold the position for a certain amount of time. In this work, the use of Azure Kinect DK (https://www.microsoft.com/en-us/p/azure-kinect-dk as a body tracking device is used in the experimental tests in Section 4.
• Definition of feedback and motivation mechanisms. The definition of every exergame has associated gamification techniques that maintain the level of commitment of the patient when making the exergame. Thus, this step aims at relieving the burden of traditional rehabilitation, in which the same exercise is made over and over again. On the contrary, fun and entertainment become more important. This is related to defining the game objectives. These objectives may be directly aligned with the patient’s specific movements or may be reached as a result of the evolution of the game state in the exergame. Feedback, from a general point of view, must be provided to both guide the patient when doing the exercise and offer a feeling of immersion in the game. The therapist will be able to assign scores that are based on the patient’s performance. The current version of this work only considers visual feedback, while using a color code, so that the patient can easily know, in real time, if they are performing the exercise correctly or not.
• Definition of metrics or key performance indicators (KPIs) to measure the progress of the patient. The therapist defines what aspects he/she needs to measure when the patient does the exercise. The ultimate pursued goal consists in monitoring the progress of the patient. One of the reference metric considered in the current version is the similarity between the execution of the exercise performed by the patient and the gold standard (specified by the therapist). If the similarity is high, then the value of this metric, which is called performance, will be positive. This metric can be complemented with more specific metrics or KPIs, which depend on the exercise to be performed. In contexts where mobility is the skill to be trained, a different metric can be defined, which serves to study the achieved degree of amplitude in a joint. This value can be easily calculated from the information that was obtained by the device that tracks the bones of the patient’s skeleton. Another set of metrics that can be integrated into the rehabilitation system and whose information can be obtained relatively easily are temporal metrics.

These definitions can be specified by means of a language. The existence of this language will allow the automatic exergame customization by means of computing systems. These systems could additionally make slight variations and gene rate adapted exergames according to the patient’s behavior and evolution during the rehabilitation process. However, this would require the inclusion of a processor that would generate the exergame, in the rehabilitation system, from its specification using the language.

3.2. Design and Development of PEL

This section discusses in depth how the language has been designed, so that it can be understood by expert personnel and, most importantly, automatically handled by computing systems. It should be noted that the syntactic structure of PEL is strongly linked to JSON, since this is the format used in the GL Transmission Format (glTF) specification [7]. This specification has been adopted in order to implement PEL.

In the following subsections a detailed specification of the components that form PEL is given.

3.2.1. Basic Exergame Structure

From a general point of view, an exergame defined with PEL integrates three major components:

• Scene. This component comprises the different views or parts of the exergame, understood from the point of view of the patient who performs it. Three basic views are considered: (i) tutorial view, which will usually reproduce an animation of the virtual avatar recreating the exergame that the patient must execute, (ii) participation view, which is associated to the real-time execution of the exergame, where the patient must reach a specific objective (for example, performing a certain number of repetitions), and (iii) results view, which will offer visual feedback to the patient, depending on what the therapist has specified when designing the exergame (for example, the obtained score by the patient).
• Actors. This component groups those elements of the exergame that maintain some type of behavior, such as those associated with the 3D animations or three-dimensional (3D) transforms. Every exergame has at least one actor: the virtual avatar that recreates the movements of the patient.
• Gameplay. This component defines the set of actions that the patient must perform to make a repetition of the exergame. The correct execution of the game mechanics associated to the exergame will trigger a sequence of actions, such as updating the score obtained by the patient.

The current version of PEL is particularly focused on the participation view, where the actual exergame definition is involved.

At content structure level, an exergame that is defined in PEL makes use of several content exchange files that are referenced to each other. However, all of them are governed by a single specification file containing the exergame meta-information. PEL relies on glTF, so it is possible to encapsulate, in self-contained files, all of the graphic resources of the exergame and, at the same time, to use the glTF extensions as a basis for the definition of PEL itself. As a practical example, all of the resources and statements specified in PEL are encoded (i) internally using base-64 data URIs (in a single .gltf file) or (ii) externally through references (using a .gltf file and another .bin file).

3.2.2. PEL Specification

The design of a language involves the definition of the three following aspects:

• The lexicon, which establishes the vocabulary of the language.
• The syntactic rules, which describe the form of the language sentences.
• The semantic rules, which define the meaning of the syntactically correct sentences of the language.

Because glTF uses the JSON format and PEL will be integrated into it by means of extensions, PEL lexical and syntactic aspects are conditioned by the use of data objects structured in attribute-value pairs and arrays. In this sense, the use of special symbols to delimit values and the use of list and dictionary structures are noteworthy. The reader is referred to review the JSON formal grammar [37].

The lexical component of PEL establishes the low-level language constructions (tokens), such as keywords, strings, punctuators, numbers, and special characters. With regard to keywords, they will be well-known terms that have been taken from the therapeutic area, the gamification area, the game mechanics area, and from the 3D graphics area. To describe the rules that define each one of the tokens, the Extended Bakus–Naur form (EBNF), a notation for specifying languages, is used. The definition of the lexicon will be shown together with the specification of the language syntax.

Next, PEL syntax will be presented, while using the EBNF notation. Moreover, the meaning and semantic constraints of each one of the language constructions, which is, their semantic aspects, are simultaneously explained.

General structure of an input

At a global level, the participation view of an exergame defined in PEL comprises the following general components: (1) description, (2) setup, (3) virtual_nodes, (4) trajectories, (5) gameplay, (6) constraints, and metrics.

Therefore, the structure of any PEL string is as follows:

PEL	::=	DESCRIPTION ‘,’ SETUP ‘,’ VIRTUAL_NODES ‘,’
		TRAJECTORIES ‘,’ GAMEPLAY ‘,’ CONSTRAINTS ‘,’
		METRICS

Each of these blocks will be specified below, highlighting lexical (especially keywords), syntax (phrase structure), and semantic (meaning) aspects. This description will be supported by the listings shown in this paper, which can be potentially used for the specification of personalized exergames.

Description component

In the description block, a descriptive information of the exergame will be included. The EBNF grammar for description block is the following:

DESCRIPTION	::=	“‘’ description ‘”’ ‘:’
		BLOCK_DESCRIPTION
BLOCK_DESCRIPTION	::=	‘{’ “‘’ text ‘”’ ‘:’ string ‘}’

The terms description and text are PEL keywords. With the attribute text, a textual description of the exergame is stored. string is a token defined, as follows:

string	::=	“‘’ { alphanumeric ∣ whitespace   }   ‘”’
alphanumeric	::=	alphabetic ∣ numeric
alphabetic	::=	lower_case ∣ upper_case
lower_case	::=	a ∣ b ∣ c ∣ d ∣ e ∣ f ∣ g ∣ h ∣ i ∣ j ∣ k ∣ l ∣ m ∣ n ∣ o ∣ p ∣ q ∣ r ∣
		s ∣ t ∣ u ∣ v ∣ w ∣ x ∣ y ∣ z
upper_case	::=	A ∣ B ∣ C ∣ D ∣ E ∣ F ∣ G ∣ H ∣ I ∣ J ∣ K ∣ L ∣ M ∣ N ∣
		O ∣ P ∣ Q ∣ R ∣ S ∣ T ∣ U ∣ V ∣ W ∣ X ∣ Y ∣ Z
numeric	::=	0 ∣ 1 ∣ 2 ∣ 3 ∣ 4 ∣ 5 ∣ 6 ∣ 7 ∣ 8 ∣ 9
whitespace	::=	‘’

Setup component

In the setup block, the information about the exergame configuration will be included, such as the used avatar, the camera information, the clock restrictions, the required repetitions, and the score to address the integration of this simple gamification technique to reward patients.

The EBNF grammar for the setup block is specified above, where the terms setup, avatar, id_avatar, posture, positions, rotation, camera, id_camera, repetitions, max_number_repetitions, score, clock, activated, and countdown_time, are keywords; and, the tokens type_posture, real, integer, and boolean are defined, as follows:

type_posture	::=	standing ∣ seated ∣ lying-down
real	::=	numeric { numeric } ‘.’ numeric { numeric }
integer	::=	numeric { numeric }
boolean	::=	true ∣ false

With respect to the component setup, avatar allows for specifying what virtual avatar will be used in the exergame, through the attribute id_avatar. Moreover, the avatar placed in the 3D space by means of the attributes posture (standing, seated or lying-down), positions, and rotation.

The element camera allows to define, in a similar way, what camera will be used (id_camera), along with its position and orientation. On the other hand, the element clock contains the attribute activated, whose boolean value makes it possible to activate the timer in the exergame. If that is the case, the attribute countdown_time represents the maximum time in seconds that the patient will have to perform the exergame. Secondly, the object repetitions allows to establish whether the patient must perform a certain number of repetitions, specified in max_number_repetitions, or whether, on the contrary,

SETUP	::=	“‘’ setup ‘”’ ‘:’ BLOCK_SETUP
BLOCK_SETUP	::=	‘{’ LIST_ATTRIBUTES ‘}’
LIST_ATTRIBUTES	::=	AVATAR ‘,’ CAMERA ‘,’ CLOCK ‘,’
		REPETITIONS ‘,’ SCORE
AVATAR	::=	“‘’ avatar ‘”’ ‘:’ BLOCK_AVATAR
BLOCK_AVATAR	::=	‘{’ DEFINITION_AVATAR ‘}’
DEFINITION_AVATAR	::=	ID_AVATAR ‘,’ POSTURE ‘,’ POSITION
		‘,’ ROTATION
ID_AVATAR	::=	“‘’ id_avatar ‘”’ ‘:’ string
POSTURE	::=	“‘’ posture‘”’ ‘:’ type_posture
POSITION	::=	“‘’ position ‘”’ ‘:’ VECTOR3
ROTATION	::=	“‘’ rotation ‘”’ ‘:’ VECTOR4
VECTOR3	::=	‘[’ real ‘,’ real ‘,’ real ‘]’
VECTOR4	::=	‘[’ real ‘,’ real ‘,’ real ‘,’ real ‘]’
CAMERA	::=	“‘’ camera ‘”’ ‘:’ BLOCK_CAMERA
BLOCK_CAMERA	::=	‘{’ DEFINITION_CAMERA ‘}’
DEFINITION_CAMERA	::=	ID_CAMERA ‘,’ POSITION ‘,’ ROTATION
ID_CAMERA	::=	“‘’ id_camera ‘”’ ‘:’ string
CLOCK	::=	“‘’ clock ‘”’ ‘:’ BLOCK_CLOCK
BLOCK_CLOCk	::=	‘{’ DEFINITION_CLOCK ‘}’
DEFINITION_CLOCK	::=	ACTIVATED ‘,’ COUNTDOWNTIME
ACTIVATED	::=	“‘’ activated ‘”’   ‘:’ boolean
COUNTDOWNTIME	::=	“‘’ countdown_time ‘”’ ‘:’ integer
REPETITIONS	::=	“‘’ repetitions ‘”’ ‘:’
		BLOCK_REPETITIONS
BLOCK_REPETITIONS	::=	‘{’ DEFINITION_REPT ‘}’
DEFINITION_REPT	::=	ACTIVATED ‘,’ MAXNUM_REPT
MAXNUM_REPT	::=	“‘’ max_number_repetitions ‘”’ ‘:’
		integer
SCORE	::=	“‘’ score ‘”’ ‘:’ BLOCK_SCORE
BLOCK_SCORE	::=	‘{’ DEFINITION_SCORE ‘}’
DEFINITION_SCORE	::=	ACTIVATED

the exergame simply contemplates the possibility of the patient performing as many repetitions as possible in the time stored in countdown_time. Finally, the object score allows tfor controlling the activation of the exergame score through the attribute activated.

Listing 1 in Section 4 shows an example of PEL definitions regarding the description and setup components.

Virtual_nodes component

In the virtual_nodes block, the collision spheres in the 3D space are defined. They will be used as trigger events in certain types of exergames, specifically in those where the patient interacts with with elements of the scene displayed in fixed positions. An example would be the classic exergame, where targets appear, in predetermined positions, which move towards the player, so he/she must take his/her hand to one of those positions.

The EBNF grammar for the virtual_nodes block is the following:

VIRTUAL_NODES	::=	“‘’ virtual_nodes ‘”’ ‘:’ BLOCK_VNODES
BLOCK_VNODES	::=	‘[’ LIST_NODES ‘{’ NODE ‘}’ ‘]’
LIST_NODES	::=	{ ‘{’ NODE ‘}’ ‘,’ }
NODE	::=	ID_ACTOR ‘,’ CLASS_ACTOR ‘,’
		POSITION ‘,’ RADIUS
ID_ACTOR	::=	“‘’ id_actor ‘”’ ‘:’ string
CLASS_ACTOR	::=	“‘’ class_actor ‘”’ ‘:’ type_actor
RADIUS	::=	“‘’ radius ‘”’ ‘:’ real

The terms virtual_nodes, id_actor, class_actor, and radius are keywords. id_actor uniquely identifies the associated virtual node, while type_actor is a token defined, as follows:

type_actor	::=	up ∣ upper-left ∣ upper-right ∣
		bottom ∣ bottom-left ∣ bottom-right

The component virtual_nodes hosts a list of 3D nodes, and each one of them contains, in turn, a unique identifier (attribute id_actor), the values of a coordinate in the 3D space (attribute position), and the radius of the collision sphere associated to the node (attribute radius).

Trajectories component

The component trajectories is used to store the trajectories associated with the movements that a patient will make when performing an exergame. In this case, and unlike the component virtual_nodes, the points or nodes in the 3D space do not need a unique identifier, since the identity of the element is represented by each path (id_trajectory), and not by individual points in 3D space. As an example, an element defined by the component trajectories would be the route or path associated to a shoulder abduction exercise. Thus, the component trajectories stores a list of individual paths, each one of them being identified by the value of the attribute id_trajectory and defined by the attribute 3d_path. This attribute will contain an ordered list or sequence of points in 3D space that form the path. Single nodes can be used as part of a path.

The EBNF grammar for trajectories block is the following:

TRAJECTORIES	::=	“‘’ trajectories ‘”’ ‘:’
		BLOCK_TRAJECTORIES
BLOCK_TRAJECTORIES	::=	‘[’ LIST_TRAJECTORIES
		‘{’ TRAJECTORY ‘}’ ‘]’
LIST_TRAJECTORIES	::=	{ ‘{’ TRAJECTORY ‘}’ ‘,’ }
TRAJECTORY	::=	ID_TRAJECTORY ‘,’ PATH ‘,’ RADIUS
ID_TRAJECTORY	::=	“‘’ id_trajectory ‘”’ ‘:’ string
PATH	::=	“‘’ 3d_path ‘”’ ‘:’
		‘[’ VECTOR3 { ‘,’ VECTOR3 } ‘]’

The terms trajectories, 3d_path, and id_trajectory are PEL keywords.

The Listing 1 in Section 4 shows an example of the two components associated to the elements of the 3D space used to represent virtual nodes or paths: virtual_nodes and trajectories.

Gameplay component

The gameplay component contains the information related to the game dynamics, which is based on the interaction between joints and virtual nodes in the 3D space. A game dynamics establishes the relationships between three fundamental aspects in an exergame:

• Trajectory or virtual path associated to the movement of the exergame.
• Interaction mechanism used by the patient to perform the exergame, associated, in turn, to (i) the parts of the body that the patient must exercise and (ii) the interaction mode.
• Feedback provided to the patient as he/she performs or completes a repetition of the exergame.

The EBNF grammar for gameplay block is the following:

GAMEPLAY	::=	“‘’ gameplay ‘”’ ‘:’
		BLOCK_GAMEPLAY
BLOCK_GAMEPLAY	::=	‘[’ LIST_GAMES_DYNAMIC
		‘{’ GAME_DYNAMIC ‘}’ ‘]’
LIST_GAMES_DYMAMIC	::=	{ ‘{’ GAME_DYNAMIC ‘}’ ‘,’ }
GAME_DYNAMIC	::=	ID_GAME_DYNAMIC ‘,’
		ID_TRAJECTORY ‘,’
		MAXTIME ‘,’ INVOLVED_JOINTS ‘,’
		INTERACTION ‘,’
		ON_REPT_COMPLETED ‘,’
		ON_REPT_FAILED

The terms gameplay, id_game_dynamic, max_time_between_path_nodes, involved_joint, interaction, type, interaction_time, on_repetition_completed, repetition_increment, score_increment, on_repetition_failed, and action, introduced next, are keywords. The tokens type_joint, type_interaction, and type_action are defined, as follows:

type_joint	::=	hip-center ∣ spine ∣ shoulder-center ∣ head ∣
		shoulder-left ∣ elbow-left ∣ wrist-left ∣ hand-left ∣
		shoulder-right ∣ elbow-right ∣ wrist-right ∣
		hand-right ∣ hip-left ∣ knee-left ∣ ankle-left ∣
		foot-left ∣ hip-right ∣ knee-right ∣ ankle-right ∣
		foot-right
type_interaction	::=	on-touch ∣ hold
type_action	::=	retry ∣ skip

ID_GAME_DYNAMIC	::=	“‘’ id_game_dynamic ‘”’ ‘:’ string
MAXTIME	::=	“‘’ max_time_between_path_nodes ‘”’ ‘:’
		integer
INVOLVED_JOINTS	::=	“‘’ involved_joint ‘”’ ‘:’ type_joint
INTERACTION	::=	“‘’ interaction ‘”’ ‘:’
		BLOCK_INTERACTION
BLOCK_INTERACTION	::=	‘{’ DEFINITION_INTERACTION ‘}’
DEFINITION_INTERACTION	::=	TYPE ‘,’ INTERACTION_TIME
TYPE	::=	“‘’ type ‘”’ ‘:’ type_interaction
INTERACTION_TIME	::=	“‘’ interaction_time ‘”’ ‘:’ integer
ON_REPT_COMPLETED	::=	“‘’ on_repetition_completed ‘”’ ‘:’
		BLOCK_ORC
BLOCK_ORC	::=	‘{’ REPETITION_INCR ‘,’
		SCORE_INCR ‘}’
REPETITION_INCR	::=	“‘’ repetition_increment ‘”’ ‘:’
		integer
SCORE_INCR	::=	“‘’ score_increment ‘”’ ‘:’ integer
ON_REPT_FAILED	::=	‘“’ on_repetition_failed ‘”’ ‘:’
		BLOCK_ORF
BLOCK_ORF	::=	‘{’ ACTION ‘}’
ACTION	::=	“‘’ action ‘”’ ‘:’ type_action

The attribute id_game_dynamic identifies each one of the game dynamics available in the exergame. id_trajectory and max_time_between_path_nodes, contained in the each game dynamic, are related to its trajectory. The attribute id_trajectory contains the unique identifier of a virtual path previously defined in PEL (trajectories component). PEL allows to set what should happen if this restriction is not met. To do this, the element on_repetition_failed allows to set an callback action (attribute action), such as informing the patient that he/she must start the repetition again (value retry).

The attribute involved_joint and the element interaction are associated with the interaction mechanism. Thus, involved_joint allows for registering the joint that the system must monitor, while interaction is used for specifying the type of interaction (on-touch for a simple interaction, or hold for one that requires the patient to hold the position during a time defined in the attribute interaction_time).

Finally, PEL allows to define what happens when a repetition is completed or not in a game dynamic, thus providing feedback to the patient. The elements on_repetition_completed and on_repetition_failed specify it. The current PEL version considers two possible values for on_repetition_completed: repetition_increment to indicate the update of the number of repetitions in the exergame, and score_increment to indicate the update of the patient’s score. On the other hand, on_repetition_failed, as introduced above, determines what happens when the patient does not perform a repetition in a correct way.

Listing 1 in Section 4 shows an example of the use of this component gameplay.

Constraints component

The constraints components stores information regarding the defined restrictions in the exergame, typically associated with those parts of the body that the patient should not ideally move to compensate for the lack of mobility or strength of the body part to be rehabilitated.

The EBNF grammar for constraints block is the following:

CONSTRAINTS	::=	“‘’ constraints ‘”’ ‘:’
		BLOCK_CONSTRAINTS
BLOCK_CONSTRAINTS	::=	‘{’ JOINTS ‘}’
JOINTS	::=	“‘’ joints ‘”’ ‘:’
		‘[’ LIST_JOINTS ‘]’
LIST_JOINTS	::=	type_joint { ‘,’ type_joint }

The terms constraints and joints are keywords. Although this component may seem to be very specific and there is a change in the abstraction level with respect to the other components, the fact is that most virtual rehabilitation commercial tools allow for explicitly indicating which parts of the body a patient should not move when doing rehabilitation. In this sense, the current PEL version considers the element joints as a list of specific joints associated to this restriction.

Listing 1 in Section 4 shows an example of the component constraints.

Metrics component

The metrics block stores information about those aspects of the patient’s exergame execution that need to be assessed by the therapists. This component represents a list of key performance indicators (KPIs) to monitor the performance and evolution of the patient. Depending on the indicator, PEL contemplates the definition of some attributes or others. Currently, the KPIs that are considered in the current PEL version are five:

• performance, which allows the definition of a patient’s performance in an exergame, using the values specified in the attribute labels and the ranges contemplated in the attribute score. Note that the value of the score in an exergame will be conveniently updated according to the value of the element on_repetition_completed included in the component gameplay. As an example, and internally in the parsing module of the automatic exergame generation system, the label not-bad would be associated to a score lower or equal to 300 points (in this example), the label good to a score higher than 300 and lower or equal to 700, and the label perfect to a score higher than 700 points.
• rehabilitation_time, which allows for recording the time that it takes for a patient to complete certain parts of the exergame when evaluating the exercise. This component defines several KPIs, such as the time that elapses between a limb passes through the virtual nodes, the time employed by the patient in order to complete a repetition, and the time that is needed by a patient to rest after completing a repetition. Another KPI is defined to record the time it takes for a patient to complete the activity, which is added in the listing shown below.
• mobility, which allows for specifying and monitoring a list of joints to measure their degree of abduction, adduction, extension, and flexion, respectively. Furthermore, this component allows to record the velocity of the joints. This list is defined by the attribute points.
• displacement, which allows measuring aspects that are related to exercises that involve gestures of movement. In other words, exercises that mimic the gesture of static walking and exercises that require the execution of lateral steps. The defined KPIs allow to evaluate walking measuring, the amount of static steps or the amount of lateral steps. Furthermore, there are two KPIs for recording both the highest and the widest step.
• balance control, which allows for evaluating aspects that are related to balance. This component defines a KPI to measure and evaluate the base support of a patient. Apart from this, another KPI is established to measure and evaluate the patient’s gravity center.

The EBNF grammar for metrics block is specified next:

METRICS	::=	“‘’ metrics ‘”’ ‘:’ BLOCK_METRICS
BLOCK_METRICS	::=	‘[’ KPI { ‘,’ KPI } ‘]’
KPI	::=	REHAB_TIME ∣ MOBILITY ∣
		BALANCE_CTRL ∣ DISPLACEMENT ∣
		PERFORMANCE
REHAB_TIME	::=	“‘’ rehabilitation_time ‘”’ ‘:’
		BLOCK_REHABTIME
BLOCK_REHABTIME	::=	‘{’ DEFINITION_REHABTIME ‘}’
DEFINITION_REHABTIME	::=	TIME_ACTIVITY ‘,’ TIME_VNODE ‘,’
		TIME_REPETITION ‘,’ TIME_REST
TIME_ACTIVITY	::=	“‘’ time_activity ‘”’ ‘:’ BLOCK_TIME
TIME_VNODE	::=	“‘’ time_vnodes ‘”’ ‘:’ BLOCK_TIME
TIME_REPETITION	::=	“‘’ time_repetition ‘”’ ‘:’ BLOCK_TIME
TIME_REST	::=	“‘’ time_rest ‘”’ ‘:’ BLOCK_TIME
BLOCK_TIME	::=	‘{’ DEFINITION_TIME ‘}’
DEFINITION_TIME	::=	TIME ‘,’ LABELS
TIME	::=	“‘’ time ‘”’ ‘:’ VALUES
VALUES	::=	‘[’ integer ‘,’ integer ‘,’ integer ‘]’
LABELS	::=	“‘’ labels ‘”’ ‘:’ ‘’ LABELS ‘’
LIST_LABELS	::=	type_label { ‘,’ type_label }

MOBILITY	::=	“‘’ mobility ‘”’ ‘:’ BLOCK_MOBILITY
BLOCK_MOBILITY	::=	‘{’ DEFINITION_MOBILITY ‘}’
DEFINITION_MOBILITY	::=	EXTENSION ‘,’ FLEXION ‘,’
		ABDUCTION ‘,’ ADDUCTION ‘,’
		JOINT_VELOCITY
EXTENSION	::=	“‘’ extension_degree ‘”’ ‘:’ real
FLEXION	::=	“‘’ flexion_degree ‘”’ ‘:’ BLOCK_JOINTS
ABDUCTION	::=	“‘’ abduction_degre ‘”’ ‘:’ BLOCK_JOINTS
ADDUCTION	::=	“‘’ adduction_degre ‘”’ ‘:’ BLOCK_JOINTS
JOINT_VELOCITY	::=	“‘’ joint_velocity ‘”’ ‘:’ BLOCK_JOINTS
BLOCK_JOINTS	::=	‘{’ JOINTS ‘}’
BALANCE_CTRL	::=	“‘’ balance_control ‘”’ ‘:’ BLOCK_BCTRL
BLOCK_BCTRL	::=	‘{’ DEFINITION_BCTRL ‘}’
DEFINITION_BCTRL	::=	BASE_SUPPORT ‘,’ GRAVITY_CENTER
BASE_SUPPORT	::=	“‘’ base_support ‘”’ ‘:’ BLOCK_DEF_BCTRL
GRAVITY_CENTER	::=	“‘’ gravity_center ‘”’ ‘:’ BLOCK_DEF_BCTRL
BLOCK_DEF_BCTRL	::=	‘{’ DEF_BASE_SUPPORT ‘}’
DEF_BASE_SUPPORT	::=	2D_POSITIONS ‘,’ LABELS
2D_POSITIONS	::=	“‘’ 2d_coords‘”’ ‘:’ VECTOR2
VECTOR2	::=	‘[’ real ‘,’ real ‘]’

DISPLACEMENT	::=	“‘’ displacement “‘’ ‘:’
		BLOCK_DISPLACEMENT
BLOCK_DISPLACEMENT	::=	‘{’ DEFINITION_BLOCKDISP ‘}’
DEFINITION_BLOCKDIS	::=	STATIC_STEP ‘,’ LATERAL_STEP ‘,’
		HEIGHT_STEP ‘,’ WIDTH_STEP
STATIC_STEP	::=	“‘’ static_steps ‘”’ ‘:’ BLOCK_STEP
LATERAL_STEP	::=	“‘’ lateral_steps ‘”’ ‘:’ BLOCK_STEP
HEIGHT_STEP	::=	“‘’ height_step ‘”’ ‘:’ BLOCK_STEP
WIDTH_STEP	::=	“‘’ width_step ‘”’ ‘:’ BLOCK_STEP
BLOCK_STEP	::=	‘{’ DEFINITION_STEP ‘}’
DEFINITION_STEP	::=	JOINTS ‘,’ AMOUNT ‘,’ LABELS
AMOUNT	::=	“‘’ amount ‘”’ ‘:’ VALUES
PERFORMANCE	::=	“‘’ performance ‘”’ ‘:’
		BLOCK_PERFORMANCE
BLOCK_PERFORMANCE	::=	‘{’ DEFINITION_PERFORMANCE ‘}’
DEFINITION_PERFORMANCE	::=	SCORE_PERFORMANCE
SCORE_PERFORMANCE	::=	“‘’ score_performance ‘”’
		‘:’ BLOCK_DEF_PERF
BLOCK_DEF_PERF	::=	‘{’ DEF_SCORE_PERF ‘}’
DEF_SCORE_PERF	::=	SCORE ‘,’ LABELS
SCORE	::=	“‘’ score ‘”’ ‘:’ VALUES

The terms metrics, rehabilitation_time, time_activity, time_vnodes, time_repetition, time_rest, time, mobility, extension_degree, flexion_degree, abduction_degree, adbuction_degre, joint_velocity, balance_control, base_support, gravity_center, displacement, static_steps, lateral_steps, height_step, width_step, 2d_coords, amount, performance, score_performance, score, and labels are keywords. The token type_label is defined, as follows:

type_label

::=

not-bad ∣ good ∣ perfect

Listing 1 in Section 4 shows an example of the component metrics.

3.2.3. Automatic Exergame Generation and Authoring Tools for Therapists

When the specification of the exergame has been completed, the system can be requested in order to automatically generate the exergame, considering the final deployment hardware platform. This generation depends on a game engine on which to execute the source code generated from PEL sentences. In the current version of this work, the Unity 3D (https://unity3d.com/) game engine is used. This choice is based on its portability and the large amount of multimedia resources available for its integration in exergames.

The automatic generation of an exergame is carried out through a software module of hierarchical analysis of the exergame information in PEL/glTF. This module receives through a URI (Uniform Resource Identifier) the location of the .gltf file with all the content of the exergame, that is, both the multimedia resources and the design itself through PEL sentences. Note that PEL makes use of the glTF extensions, particularly through the field extras, in which the specific behavior linked to the exergame design methodology previously introduced is specified. The automatic analysis process of this field extras is integrated in a sub-module that is responsible for parsing in the form of a syntactic analyzer. This is supported by the implementation of glTF to make the de-serialization of the glTF coded elements as efficient as possible.

The idea of providing a language for the definition of personalized exergames, whose sentences can be automatically processed by a software program, has a great impact on the process of adapting physical rehabilitation according to the needs of each patient. Taking this language as a reference, it is possible to build both software modules that work on top of it and authoring tools that make it possible for therapists to easily design exergames in a visual way.

It should be noted that, although the language proposed in this work is simple enough, the therapist is not intended to be the one that use it. As already mentioned, the idea is to have an intermediate language that can be manipulated by advanced subsystems, according to various aspects, such as the interaction of the patient with the exercise. In this context, the therapist may need a high-level authoring tool that facilitates the design of the exergame. This tool would integrate a software module in order to generate the PEL sentences that are equivalent to the graphic definition of the exergame. Although the development and evaluation of this tool is beyond the scope of this article, a first design has been addressed in order to complement the proposed approach.

For the definition of the exercise and rehabilitation objectives specific to each patient, the therapist will be provided with a graphical definition tool that facilitates the definition of key positions, as well as the specification of the different variables and options available in the exergame. This graphic editor encapsulates the glTF and JSON components that will be automatically exported and used by the other components of the tool. Internally, the graphical editor module would employ a description based on the H-Anim standard (ISO/IEC IS 19774-1:2019). As for the bone hierarchy, each animation chain is composed of several articulated elements that define transformations from the root of the hierarchy.

There are several specific avatars for rehabilitation, such as [38] or [39], which, however, are not focused on the creation of exercises by non-technical staff without using motion capture techniques. The precise configuration of the positions and rotations of each part of the animation hierarchy is essential when defining rehabilitation exercises. The definition of movement must allow for both rapid definition of movements, using in our proposal inverse kinematics (IK), and subsequent fine tuning, using direct kinematics methods.

In the virtual avatar integrated in the proposed tool, every element of the kinematic chain can be positioned and rotated while using realistic human limitations. This is the most time consuming phase in the generation of a new exergame. As detailed next, a new approach has been devised in order to increase the adaptability and the ease of use by the therapist. The graphic editor will allow defining start and end positions in three different ways:

• Use a set of predefined key positions. This option allows to directly input common positioning settings for each limb.
• Use of a high level adjustment by employing high-level reverse kinematics manipulators.
• Definition, if needed, of the position and rotation details of each bone using direct kinematics.

The skeleton that is defined in the graphical editing module consists of 103 bones (see Figure 2), of which 19 are inverse kinematic manipulators and high-level controls (they influence a set of bones). The use of inverse kinematics and spherical quaternary interpolation [40] facilitates the therapist’s work in defining the key poses of the starting and ending positions of the rehabilitation exercises.

In relation to the methods of inverse kinematics, there are three concepts to which special attention is paid: the description of the joints, the angle of rotation, and the degrees of freedom. The physical characteristics of the joints themselves determine the final movement, the angle of rotation describes the rotation allowed for each joint, and the degrees of freedom involve the directions in which each joint moves. In most kinematic configurations, it is essential to define the rotation restrictions in order to avoid constrained configurations and simulate only physically correct positions. There are two ways of dealing with IK: analytical or iterative methods. Analytical methods require prior analysis of the animation hierarchy and, in the case of body movements in humans (like in the proposed tool), the resulting equations are very complex. To face this problem, the proposed graphic editor uses the iterative method Cyclic Coordinate Descent (CCD) [41]. CCD is an iterative method for calculating the IK that minimizes the error of the kinematic configuration for each joint.

Thus, each kinematic chain defines its own set of rotation changes, so that the final composition of the animation is obtained using non-linear animation (NLA) techniques [41]. In NLA, each piece of animation (action) is focused on the definition of a certain action (e.g., moving the right arm). These actions are combined with others, each of them defining a certain animation layer. When generating PEL sentences, this independent treatment will allow exporting the restrictions that are defined by the therapist in a simple way. On the other hand, each exergame is defined by several actions (or animation channels). Basic SLERP interpolation [40] will be used to concatenate the different defined sub-actions (e.g., raising the whole arm and then rotating the forearm 90${}^{\circ }$ around the torso).

Obtaining realistic movements is probably the most important element to consider in the representation of rehabilitation exergames. The animation must faithfully represent the exercise to be performed by the patient. The results obtained related to the proposed approach will generate high quality results, which are independent of the final representation. In addition, an approach that is based on keyframe animations produces more accurate, clean, and understandable results than motion capture based techniques, with a lower computational cost in the context of high-demand application domains [42].

The pose editor

One of the main problems related to the creation of rehabilitation exercises is the time needed for their definition and the level of knowledge regarding 3D animation tools. Despite the development of new techniques to facilitate the animation of virtual characters (such as inverse kinematic controls and key poses), an experienced user may require between five and 10 min. to define a realistic movement. Because of the initial knowledge that is required to work with these types of tools, this phase requires special attention in the proposal.

An exercise editing module has been designed (see Figure 3) to facilitate the definition of rehabilitation exergames. The therapist will be requested in order to choose basic settings of shape and orientation of the extremities in a direct way. Firstly, he/she will choose the joint, and then he/she will pick, from a catalog of key poses, the one that is most similar to the one he/she wants to perform. This configuration can be refined later while using reverse kinematics controls and, if necessary, direct kinematics controllers for the final tuning.

In the current proposal, 48 basic positions were defined for the arms and 12 basic positions for the legs. By using an exercise editor like this, the time that is needed to specify a new exercise can be reduced by a 90%, obtaining results of similar quality.

Finally, the exercise definition system has been designed to be handled as a plug-in within the Blender animation suite. Blender existing functionalities can be used for animation, keyframe definition, or playback, among others, which facilitate the subsequent export process to glTF + JSON (see Figure 4).

4. A Case Study Using PEL

In order to evaluate the proposed system, we have conducted two studies to assess different aspects of the system, which is design around PEL, the cornerstone of our software to generate personalize exergames automatically. The first one is related to stroke patients. The second one to therapists.

Regarding the first case, a quasi-experiment with real stroke patients has been conducted in order to evaluate the understanding and adequacy of a set of exergames that are defined with PEL by a therapist with the assistance of a developer. Regarding the second case, and considering that the therapists are responsible for designing routines or exercises with the best therapeutic value, a questionnaire has been designed in order to know how much this system contributes to patients in the context of home rehabilitation and exergames that are generated from PEL sentences.

4.1. Study with Stroke Patients

The study included 17 real associated to the General Hospital Nuestra Señora del Prado (https://sanidad.castillalamancha.es/ciudadanos/centros/hospital-nuestra-senora-del-prado), located in Talavera de la Reina. A positive report by the ethical committee in clinic research of such hospital was issued prior to conducting this study on September 24th, 2019 (project id 30/2019). The group of patients evaluated was randomly selected with ages ranging from 37 to 86 years. When the experiments were carried out, it was explicitly stated at the beginning of the test that the collected data would be treated confidentially and exclusively for this study. The ethic approval and consent form statement, which is available for the reader (https://www.esi.uclm.es/www/dvallejo/PEL_Applied_Sciences), was filled by every single patient before conducting the experiment.

4.1.1. Method

The quasi-experiment was divided into three stages:

• Preparation. An instructor presented the system to each participant for about 10 min. During the explanation, an example was projected on the wall so that the patient could follow the explanation perfectly and understand the activity to be performed.
• Development. Each patient performed an exercise routine included in the system according to their condition, which is, if the stroke caused the patient to lose mobility in the left, right, or both parts of their body.
• Evaluation. Once the patient had completed the previous step, he or she was encouraged to fill in a questionnaire to evaluate the exergames and the software system using Microsoft One Drive Forms (https://forms.microsoft.com/), in order to facilitate their subsequent digital processing.

The exergames that were performed by the patients were based on rehabilitation exercises to train the upper body, thus involving the arms. All of them were automatically generated from PEL language definitions, when considering both left and right arm versions. Particularly, the exercises associated to the generated exergames were those of (i) bringing the right and left hand from the hip to the shoulder, (ii) bringing the right and left hand from the hip to the mouth, and (iii) bringing the right and left hand from the hip to the head. In addition, (iv) right and left shoulder abduction exercises were performed. Figure 5 shows the visual aspect of some of the generated exergames in the initial version of the rehab system, all of them in the context of a virtual gym. On the other hand, Listing 1 shows the full PEL specification of the exercise right shoulder abduction. At this point, it is important to remark that there exists a normalization process so that, independently of the patient’s height and width, the dimensions of the virtual avatar remain the same for every generated game.

Each exercise was projected on a large screen. This allowed the patients to correctly visualize the movement that they had to make, as well as the gamification elements provided by the system.

The purpose of the test was to analyze the use of the generated exergames by real patients in terms of understanding and adequacy. Thus, the steps defined in [43] were followed, while using a template Goal-Question-Metrics.

Firstly, patients were asked if they had already used any system of these characteristics in order to find out the advantages or weaknesses that the proposed approach might have with respect to other rehabilitation systems.

Then, they were asked to complete a questionnaire with a total of 22 questions (see

Table A1. Textual description of the items evaluated by patients.

Dimension	Item	Description
Activity Perception	1. INT1	This activity has been fun for me
	2. INT2	I found this activity interesting
	3. EFF	I have worked to do it well
	4. LEA	It has been easy for me to learn how to use this system
Cognitive Load	5. TD	The activity required a lot of concentration
	6. E1	I have been very concentrated during the activity
	7. E2	I have had to work pretty hard to get the activity done
	8. DD	I have found difficult to perform rehabilitation exercises using this system
Utility	9. PREF	I would rather use this system at home than have to go to a rehab left
	10. CONS	This system would make me more consistent in performing the exercises at home
	11. PSA	I believe that using this system to do rehabilitation exercises can be motivating
	12. GAM	I like the application has the format of a game
Game elements	13. GE1	Avatar
	14. GE2	Repetitions
	15. GE3	Score
	16. GE4	Trajectory to be followed by the avatar
	17. GE5	Decorative elements
	18. GE6	Description
TAM	19. PEU	This system is easy to use
	20. PU	Using this system could help me in performing the rehabilitation exercises
	21. ITU1	If I could borrow this system, I would use it at home
	22. ITU2	I would recommend my friends to use this system to do the exercises at home

in Appendix A), scored with a Likert scale (1: totally disagree; 5: totally agree), grouped in five dimensions: activity perception, cognitive load, utility, game elements, and the TAM (Technology Acceptance Model) framework. These five dimensions are briefly described below.

• Perception of activity. This dimension is composed of a set of questions to measure interest (INT), effort (EFF), and ease of learning how to use the system (LEA).
• Cognitive load. This block includes a series of statements inspired by the Cognitive Load Theory (CLT) [44], which allows measuring the cognitive load imposed by the software, the complexity of the developed task (TD) and the complexity that is required by both the software and the device used (DD). In addition, two questions were added to measure the patient’s effort in relation to the activity performed (E).
• Utility. The third block consists of questions related to the patients’ opinion regarding the exergame-based rehabilitation system, that is, if they prefer to use it at home instead of attending rehabilitation centers (PREF), if the system encourages them to be more consistent when performing rehabilitation exercises (CONS), if the system enhances motivation (PAS), and if they like the application to look like a game (GAM).
• Game elements. The fourth block contains questions related to the degree of suitability of each of the elements included both in the user interface and in the exergame scene itself, which is, aspects, such as the avatar representation, the number of repetitions, the score, or the visually decorative elements.
• TAM. The fifth block consists of questions based on the TAM framework [45], which are used to measure the perception of system usability (PEU), the utility perception (PU), and the intention of use (ITU).

Finally, the patients had the opportunity to provide some comments on the system. In this way, it will be possible to improve specific parts of the exergame-based rehabilitation system in future developments. In fact, Section 5 discusses some of the already integrated improvements regarding the first version of the rehabilitation system.

4.1.2. Results

At the beginning of the survey, the patients answered two questions that were related to the use of systems similar to the one evaluated. 47% of the respondents answered that they had used more than one system in their rehabilitation process, when considering that our proposal is more intuitive and simple to use (see

Table 1. Descriptive statistics of the dimensions evaluated by patients.

Dimension	Item	Mean	Standard Deviation	Mode
Activity Perception	1. INT1	4.41	0.87	5.00
	2. INT2	4.53	0.62	5.00
	3. EFF	4.35	1.06	5.00
	4. LEA	4.65	0.49	5.00
Cognitive Load	5. TD	3.71	0.99	4.00
	6. E1	3.82	0.88	4.00
	7. E2	3.65	1.00	4.00
	8. DD	2.88	1.17	2.00
Utility	9. PREF	4.00	1.22	5.00
	10. CONS	3.88	1.17	4.00
	11. PSA	4.06	1.20	5.00
	12. GAM	4.71	0.59	5.00
Game elements	13. GE1	4.59	0.71	5.00
	14. GE2	4.41	0.80	5.00
	15. GE3	4.24	0.97	5.00
	16. GE4	4.59	0.71	5.00
	17. GE5	4.65	0.79	5.00
	18. GE6	3.24	0.75	3.00
TAM	19. PEU	4.35	0.79	3.00
	20. PU	3.88	1.11	4.00
	21. ITU1	4.06	1.20	5.00
	22. ITU2	4.35	0.79	5.00

).

Apart from the analysis that is shown in Table 1, the existence of correlations between different items has been studied (see

Table 2. Spearman’s rank correlation coefficient. Significant values (p-value < 0.05) are marked in bold.

	E1	INT2	EFF	GE1	GE2	GE3	GE4	GE5	GE6	GAM	PU	ITU1	ITU2	PREF	CONS	PSA
TD	0.82
E1					0.53		0.49			0.36
E2			0.55
DD											−0.41	−0.44	−0.58	−0.49	−0.74	−0.54
EFF					0.72	0.62				0.34
GE2						0.85
GE4										0.56
GE5				0.56					0.29
GE6				0.36
INT1		0.67		0.47				0.44	0.30
INT2											0.59	0.46	0.45		0.46	0.46
PREF															0.66	0.58
PU												0.81	0.75	0.38	0.64	0.69
ITU1													0.81	0.47	0.54	0.51
ITU2														0.57	0.73	0.66
CONS																0.90

). For this purpose, the Shaphiro–Wilk test was applied to contrast the normality of the data set, given that the sample size is less than 50. The result revealed, with a 95% confidence level, that the variables do not fit a normal distribution, which resulted in the use of the Spearman correlation coefficient.

The obtained results show the correlation between INT2 and PU, ITU1, ITU2, CONS, and PSA. This reflects that, as the patients rated the activity as more interesting, they considered that the system would help them perform rehabilitation exercises; they would use it at home; they would recommend it to family members; it would increase their consistency; and, it would motivate them to perform more exercise routines. In addition, the correlation between the variable PREF and variables CONS/PSA indicates that the patients would prefer to use the system at home, because it motivates them and increases their determination.

4.2. Authoring Tool Evaluation with Therapists

The system has been evaluated by six therapists associated to the General Hospital of Nuestra Señora del Prado in order to know its applicability in the context of home rehabilitation. Again, the shown exergames were automatically generated from PEL sentences. The ultimate goal was to evaluate what the rehabilitation system can provide in terms of functionality, which indirectly reflects what PEL can currently offer and usability. The therapists completed with a questionnaire with free text, multiple choice, and single answer questions.

4.2.1. Questionnaire Design

The questionnaire was designed using Microsoft OneDrive Forms both to include multimedia content and facilitate their digital processing. It was divided into two sections:

• System demonstration. A demonstrative video for about 3 min was included to explain the features and functionalities of the system.
• Evaluation of the system. A set of questions were included with different objectives. Some of them were designed so that they provided valuable information that enable continuously improving the system and how PEL can be adapted to address such improvements. Others were added to measure certain aspects of the system.

The second section of the questionnaire was followed using the Goal-Question-Metrics. Firstly, therapists were asked if they knew virtual rehabilitation systems. In this way, we can study the referred systems in order to improve ours. Secondly, they were asked to indicate which group of people could benefit most from the system. Next, a question was introduced for obtaining feedback from therapists to know what information should be collected when a patient finishes an exercise. Subsequently, they had to select with what frequency they would like to receive the information generated by the system after a patient has performed a rehabilitation activity. Subsequently, they were asked to complete a total of 15 questions (see

Table A2. Textual description of the items evaluated by therapists.

Dimension	Item	Description
TAM	1. PEU1	Voice interaction facilitates the use of the system
	2. PU1	The system could help people to perform rehabilitation
	3. PU2	The system is useful to perform exercises autonomously
	4. PU3	It is interesting that the system can personalize exercises based on the needs of patients
	5. ITU1	If I had the system, I would use it with my patients
	6. ITU2	I would recommend other therapists to use this system
	7. ITU3	The system encourages patients to be used because they do not need to wear sensors
System perception	8. MOT	Games can be motivating for people to perform rehabilitation
	9. CONST	The system could increase the constancy of old people to perform exercises
	10. IMP1	The system can improve the rehabilitation in a rehab centre
	11. IMP2	The system could help people to perform rehabilitation at home
Game elements	12. GE1	Avatar
	13. GE2	Repetitions
	14. GE3	Score
	15. GE4	Trajectory to be followed by the avatar

in Appendix B), scored with a Likert scale (1: totally disagree; 5: totally agree) grouped in three dimensions: TAM framework, system perception, and game elements.

• TAM. The first dimension consists of questions to measure the perception of system usability (PEU), perception of usefulness (PU), and intention of use (ITU).
• System perception. The second dimension is composed of a set of questions to evaluate the motivation that the system can produce in the patient (MOT), the constancy the system can generate in old people (CONST), if the system can improve the rehabilitation process in a rehab center (IMP1), and whether it would improve the rehabilitation process at home (IMP2).
• Game elements. The third dimension consists of several questions to measure the adequacy of the user interface from the therapists’ perspective.

Once the therapists completed the part of the questionnaire based on Likert scale questions, they could provide some comments about the system. For example, potential actions the system could perform and their advantages to strengthen them in the near future.

4.2.2. Results

The objective of this study was to know the therapists’ point of view regarding the use of a system focused on home rehabilitation. At this moment, the system developed is in a preliminary version. However, the professionals, who answered the questionnaire, assessed the prototype as a good starting point for achieving one of our goals, which is, making home rehabilitation accessible. Thus, the obtained results from the questionnaire reflect that the therapists considered the system to be useful and easy to use. Apart from this, they positively evaluated the feature of personalizing exergames.

Table 3. Descriptive statistics of the dimensions evaluated by therapists.

Dimension	Item	Mean	Standard Deviation	Mode
TAM	1. PEU1	4.50	0.83	5.00
	2. PU1	4.50	0.54	5.00
	3. PU2	3.33	1.03	4.00
	4. PU3	4.83	0.40	5.00
	5. ITU1	4.67	0.51	5.00
	6. ITU2	4.67	0.51	5.00
	7. ITU3	4.83	0.40	5.00
System perception	8. MOT	3.50	0.54	4.00
	9. CONST	3.83	1.16	5.00
	10. IMP1	4.50	0.54	5.00
	11. IMP2	4.33	1.21	5.00
Game elements	12. GE1	3.83	0.75	4.00
	13. GE2	4.33	0.81	5.00
	14. GE3	4.17	0.98	5.00
	15. GE4	4.50	0.54	5.00

shows the obtained results from a descriptive statistical analysis.

5. Discussion

5.1. Patients

With regard to the results that were obtained from the data collected in the first questionnaire, as reflected in Table 1, the evaluated exergames have been well received by the patients, achieving a good overall score (most items obtained a value of 4 or greater) in all the dimensions. In the first block (activity perception), the patients considered the activity to be fun (INT1) and interesting (INT2). In addition, they considered that they tried to perform the activity well and that the learning with respect to the use of the system was simple (LEA).

Most patients scored high on the cognitive load imposed by the activity (TD), since many of them, due to their condition, needed a high concentration to move the required limb. Furthermore, this implied that they were concentrated during the activity (E1, E2). However, several of the patients considered that the rehabilitation exercises were not complicated for them using this system (DD), since it helped them follow the movement that they had to perform.

On the other hand, practically all of the patients would prefer to use this system at home, rather than having to attend rehabilitation centers (PREF), since most of them had difficulties in being able to move around, either due to the inability to drive a vehicle or to limited economic resources. However, this does not imply, from our point of view, replacing the role of the therapist or reducing the rehabilitation time spent in specialized centers, since they are of vital importance for patients in poor condition or for patients who begin to use rehabilitation systems. On the contrary, the exergames that are generated from PEL sentences can help those patients with good progression, but who present certain logistic difficulties, such as those previously mentioned. On the other hand, most patients considered that the tested system would motivate them (CONS, PSA) to do more exercises because of its game aspect (GAM). This aspect is significant, since one of the drawbacks in the rehabilitation process is the patients’ lack of motivation.

As for the game mechanics, the patients satisfactorily evaluated the avatar design and decorative elements (GE1, GE5). In addition, the information about the repetitions and score achieved in the activity were positively evaluated (GE2, GE3). In these cases, the use of gamification techniques can generate a healthy competition, where patients themselves compete with each other to achieve maximum scores. The element that obtained the highest score was the virtual trajectory composed of 3D spheres that indicate the path the patient should follow (GE4). They found it to be intuitive and helped them to execute the required exercise correctly. However, the description of the activity to be performed did not have a significant score, which may indicate that its design needs to be improved (GE5).

In relation to the last part of the questionnaire, the patients believed that the system is easy to use (PEU) and considered that it can help them improve their physical rehabilitation (PU). Furthermore, most of them would use it at home (UTI1), apart from recommending it to a relative with a similar condition. This fact may be associated with the motivation (PSA) that the system generates, which implies that it increases the frequency with which they exercise.

In addition, the patients had the option of leaving comments regarding the system, of which we highlight the following ones:

• In some cases, it is difficult to interpret the depth of the spheres used to guide the movement of an exercise.
• The system encourages rehabilitation exercises.
• It would be ideal if the system were more portable, that is to say, to remove the dependence on personal computers.

5.2. Clinicians

Our data exploration has consisted in a statistical analysis description given the low number of therapists surveyed (n = 6). In general, the data reflect that the therapists have valued the prototype well, obtaining a good score in all dimensions. This reflects the goodness of the functional capabilities that are offered by PEL.

In the first dimension, the therapists believed that it is useful that the system integrates voice input, so that the patients can interact easily with it (PEU1). They also positively valued that the system prevents patients from wearing sensors on their bodies (ITU3), since it can make difficult the rehabilitation at home. However, they considered that the system needs to be improved so that it can be deployed in a scenario where the patients do not require a professional (PU2), that is, where they can perform rehabilitation autonomously. On the other hand, the therapists considered the system can help patients to perform rehabilitation (PU1). Furthermore, they indicated that they would use the system (ITU1) if they had it, apart from recommending it to other professionals (ITU2).

The second dimension of the questionnaire is related to the perception of the therapists with regard to the system. In this block, the therapists considered that exergames can motivate patients to keep them engaged with their rehabilitation (MOT). This implied that they perceived the tool as a good complement in face-to-face sessions (IMP2). On the other hand, they believed that the system may increase the perseverance of the patients (CONST), being able to help those who present difficulties to go to a rehabilitation center (IMP2) (for example, cases where patients live in small villages far from the city).

Finally, in the third block of the questionnaire, therapists were asked for the game elements of the system. The avatar design was positively evaluated (GE1), as was the component that was related to repetitions and scores (GE2, GE3). The best scored item was the trajectory composed of virtual spheres that indicate the movement a patient’s limb needs to follow (GE4). The result obtained in this block allows for interpreting that PEL is a fundamental component of the system, since this language maintains rules for collecting data to be analyzed and for designing game mechanics that engage a patient during his rehabilitation.

Apart from this, the therapists had the option of providing feedback with the objective of improving the system. Next, we highlight some of them:

• Adapting the avatar position for people in wheelchairs to feel them integrated in the system.
• Adding activities of the daily living (ADLs), for example, taking a glass and making the gesture to drink.

5.3. Improving the Rehabilitation System

The conducted evaluation, both with patients and therapists, allowed for the identification of several aspects that can be improved. Some of these have been already incorporated into the current version of the rehabilitation system, while others are being considered in the context of future lines of research. In this sense, Figure 6 shows the updated version of the user interface that the system offers to patients when carrying out rehabilitation. The main improvements that have been incorporated are summarized below:

• The removal of visual noise so that patients can focus on performing the exergames and only the relevant information is displayed. This improvement has been added to enhance the patients’ concentration when using the system, but maintaining an adequate level of immersion (currently through the virtual gym scenario) to continue motivating them.
• New interaction mechanism based on contextual voice commands (see the upper right part of Figure 6). Thus, the patient can communicate with the rehabilitation system in a more natural and direct way. This functionality is especially important for patients with motor disabilities. This improvement has been integrated when considering the predisposition of the evaluated patients to use the system at home in an autonomous way.
• New visual representation of the 3D path that is associated with the exergame execution, based on the use of a color code and the size of the virtual spheres. The white spheres represent that part of the exercise that has already been done, while the orange ones represent the remaining part of the exercise. On the other hand, the sphere with a radius slightly larger than the rest is associated with the next step considering the current position of the related patient’s joint. This improvement has been added as a result of some patients having difficulty interpreting the position of the spheres. However, it is considered to improve the lighting of the scenes, so that the shadows cast by the spheres can also facilitate the exercise interpretation by the patients.
• The integration of visual components to represent which part of the body is worked in each exergame and show an animation with the correct execution of the exergame (central right part of Figure 6). This improvement has been added, because the analysis of results showed that some patients had difficulties in understanding how to perform the exercise.
• Integration of visual components, so that the patient knows what exergame of the routine he/she is performing at every moment (bottom of Figure 6). The number of circles represents the number of exercises that make up the routine. The orange circle represents the position of the current exercise in the list. This enhancement has been added to facilitate the patient’s guidance between exercises.

The improvements integrated in the latest version of the system will be evaluated in the future through further experiments with patients and therapists.

6. Conclusions

PEL (Personalized Exergames Language) has been defined to allow for the automatic generation of customized exergames within the context of the physical rehabilitation of patients affected by stroke. PEL relies on glTF, an open standard that is devised to efficiently send and load 3D models and scenes in applications. Particularly, the design of PEL was discussed paying special attention to the lexical and semantic aspects, since the syntactic component depends on the JSON format, which is used by glTF. The underlying software system is able to generate and execute exergames from PEL sentences.

A methodology for the effective design and generation of personalized exergames has also been proposed. This methodology follows a five-step process: (i) exercise definition, so that the therapist can establish the therapeutic objectives, (ii) definition of the interaction scheme between the patient and the exergame, typically through the definition of the trajectory of the rehabilitation movement, (iii) definition of feedback and motivation mechanisms, so that patients are engaged when performing the exergames, (iv) definition of metrics to measure the patients’ performance and monitor their progress, and (v) generation of the exergame, an automatic process that transforms the exergame definition in PEL into an actual exergame that can be executed.

A number of exergames, defined while using PEL, were generated in order to assess how patients and therapists from one hospital and two community centers perceived the exergames understanding and suitability. From a general point of view, the feedback they provided was very positive, when considering that one of the potential uses of the system consists in performing the exergames at home. This feedback has been studied to improve specific aspects of the rehabilitation system, which have been integrated in the current prototype.

This first version of PEL, and the underlying software that automatically generates personalized exergames from PEL sentences, opens the door to potential improvements, such as the integration of authoring tools that can be used by therapists in order to graphically define exergames. These tools, like the one introduced in Section 3.2, will have to include a software module that is able to translate the visual definition of exergames into PEL sentences. From them, it is now possible to automatically generate exergames. At the same time, representing the exergames with sentences that can be processed by a computer facilitates the automatic personalisation of rehabilitation routines, depending on the dynamic evolution of the patient’s progress. Another potential line of work would involve the integration of the care-giver avatar by expanding the setup component devised in PEL. Thus, the care-giver could physically assist the patient when he/she cannot perform the exergames in an autonomous way. Because the used hardware can track multiple bodies, this would be possible from the technical point of view. However, aspects, such as the impact of partial occlusions and data filtering, should be carefully assessed.