# **New Research in Children with Neurodevelopmental Disorders**

Edited by Dulce María Romero-Ayuso Printed Edition of the Special Issue Published in *Children*

www.mdpi.com/journal/children

## **New Research in Children with Neurodevelopmental Disorders**

## **New Research in Children with Neurodevelopmental Disorders**

Editor

**Dulce Mar´ıa Romero-Ayuso**

MDPI ' Basel ' Beijing ' Wuhan ' Barcelona ' Belgrade ' Manchester ' Tokyo ' Cluj ' Tianjin

*Editor* Dulce Mar´ıa Romero-Ayuso Physical Therapy Occupational Therapy Division University of Granada Granada Spain

*Editorial Office* MDPI St. Alban-Anlage 66 4052 Basel, Switzerland

This is a reprint of articles from the Special Issue published online in the open access journal *Children* (ISSN 2227-9067) (available at: www.mdpi.com/journal/children/special issues/New Research in Children with Neurodevelopmental Disorders).

For citation purposes, cite each article independently as indicated on the article page online and as indicated below:

LastName, A.A.; LastName, B.B.; LastName, C.C. Article Title. *Journal Name* **Year**, *Volume Number*, Page Range.

**ISBN 978-3-0365-4392-5 (Hbk) ISBN 978-3-0365-4391-8 (PDF)**

© 2022 by the authors. Articles in this book are Open Access and distributed under the Creative Commons Attribution (CC BY) license, which allows users to download, copy and build upon published articles, as long as the author and publisher are properly credited, which ensures maximum dissemination and a wider impact of our publications.

The book as a whole is distributed by MDPI under the terms and conditions of the Creative Commons license CC BY-NC-ND.

## **Contents**



## **About the Editor**

### **Dulce Mar´ıa Romero-Ayuso**

Dulce Romero Ayuso completed her doctoral thesis for her PhD degree on neuropsychological and neuromagnetic profiles in children with ADHD. She received the Extraordinary Doctorate Award from the Faculty of Psychology at the Complutense University of Madrid. She has been intensely active in research on neurodevelopmental disorders. She is a researcher and member of the Cognitive Neuroscience Group of the Brain, Mind and Behavior Research Center (CIMCYC) at the University of Granada. She has recently joined the Granada Ibs Research Group TECe2'-Rehabilita-T.

Among her lines of research is the study of cognitive processes that underlie the performance of activities of daily living, occupational therapy and the application of new high- and low-cost technologies for evaluation and neurorehabilitation in childhood. She has directed different research projects, mainly focused on attention-deficit/hyperactivity disorders. She is currently the coordinator of the Degree in Occupational Therapy and is the Director of the Master's Degree in Occupational Therapy in Functional Diversity in Childhood at the University of Granada.

## *Editorial* **Future Challenges in Research in Children with Neurodevelopmental Disorders**

**Dulce Romero-Ayuso**

Department of Physical Therapy, Occupational Therapy Division, Faculty of Health Sciences, University of Granada, 18016 Granada, Spain; dulceromero@ugr.es

The prevalence of neurodevelopmental disorders exceeds 15% worldwide, and often they are associated with other neurological disorders [1]. Neurodevelopmental disorders are characterized by showing different patterns in the acquisition of motor, cognitive, linguistic and socio-emotional skills, which affect the functioning in the different contexts relevant to children. They are, therefore, a relevant topic for clinicians engaged in evaluation and intervention in children. The aims of this Special Issue are to present new approaches and results on evaluation and intervention, emphasizing the importance of the full active participation of the child in the different environments, personal, family and/or academic, that are part of their daily life. In this way, the studies included in this monograph allow us to cross the barrier of the clinical context, changing the focus of evaluation and intervention to other spaces where the main significant activities of children occur, such as play, educational learning, activities of daily life and self-care, rest and social participation. A section has also been dedicated to studies that provide more evidence on new therapies, such as the use of hypnotherapy or virtual reality systems.

Several evaluation tools are presented in this Special Issue. The first of them is a new tool developed from Angel Riviére's autism spectrum inventory [2] of these children, including the response to sensory stimuli [3]. According to the last diagnostic classification of DSM-5, the importance of sensorial hypersensitivity in autism spectrum disorder [4], which may result from a sensory processing disorder, has been recognized [5]. The second tool is a cross-cultural adaptation to the Spanish population of an instrument called My Child's Play [6] that allows the evaluation of the play of children between 3 and 9 years old. This instrument, through the play, provides a cut score that helps to differentiate children with typical development and with neurodevelopmental disorders [7]. Additionally, this study enabled us to determine what factors of the play can be weaknesses or strengths: cognitive flexibility and executive attention, communication and social interaction, the preferences and characteristics of the play or the opportunities of the context. On the other hand, Sewani and Kashef [8] show an innovative proposal with a study that allows us to approach the diagnosis of autism spectrum disorders using a machine encoder based on the learning of neural networks. Finally, within the evaluation tools, a new approach is presented with the aim to develop children's understanding of death, through the EsCoMu scale [9]. This instrument is an important contribution given the lack of research that addresses bereavement in the child. The authors provide a useful tool, with good psychometric properties and four factors underlying children's understanding of death: universality, irreversibility, nonfunctionality and causality.

Regarding interventions, the effects of low-intensity modified Constraint-Induced Movement Therapy on upper limb functionality in eight children with hemiplegia aged 4–8 years are presented. The results of this study show an increase in the spontaneous use of the affected upper limb in bimanual tasks and dissociated and propensity movements [10]. Likewise, the study of Riquelme, Sabater-Gárriz and Montoya [11] delves into the impact on the family of chronic pain in children with cerebral palsy (CP) and its relationship to the quality of life of these children and their ability to communicate. This volume also

**Citation:** Romero-Ayuso, D. Future Challenges in Research in Children with Neurodevelopmental Disorders. *Children* **2021**, *8*, 328. https:// doi.org/10.3390/children8050328

Received: 20 March 2021 Accepted: 22 April 2021 Published: 23 April 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

addresses the effectiveness of two types of emerging therapies: hypnotherapy in children with CP [12] and the use of virtual reality in children with attention-deficit hyperactivity disorder (ADHD) [13]. Both studies allow us to advance the available evidence. Thus, it is concluded that hypnotherapy, for 8–12 weeks, stimulates proprioceptive and balance reactions, reducing muscle spasticity, through rhythmic and symmetrical movement, leading to an improvement in the gross motor function in children with CP, such as lying down, roll, sitting and walking [12]. The meta-analysis of Virtual Reality-Based Interventions for Children and Adolescents with ADHD highlights the need to design randomized controlled trials with virtual reality and concludes that virtual reality may be effective in improving performance in sustained attention tasks in children with ADHD [13].

Three studies address the impact of neurodevelopmental disorders on personal, educational and social functioning. In the first of these, Blanco-Martínez et al. [14] present a cross-sectional study on participation in different everyday contexts, finding differences between children with and without neurodevelopmental disorders and their possible impact on daily life. Among the significant activities in children between the age of 6 and 12 are school activities. In this way, Maciver et al. [15] approach Scotland's experience in developing strategies and models that promote the participation and inclusion of children with support needs at schools, through the involvement and training of teachers, in order to reduce inequalities. The proposed model, CIRCLE, incorporates the concepts of the model of human occupation, with the aim to broaden our perspective beyond deficits, including the strengths, motivation, routines of the child in a comfortable, flexible school, able to innovate, that train teachers, professionals and include families. Likewise, the concern about the care of children with autism at school is also reflected in the study of Taresh et al. [16] conducted in Yemen.

We hope the studies presented in this Special Issue will be useful and of value to the different professionals and researchers in the fields of occupational therapy, physiotherapy, psychology and education and, above all, to help improve the opportunities in the everyday contexts of children with neurodevelopmental disorders. Research along with clinical experience and clinical reasoning will enable the development of practices based on the best available evidence. Additionally, we hope that this volume serves its intended purpose: an advance in two of the challenges of the H2031 Strategy and the 2030 agenda for sustainable development, related to Goal 3 "Good health and well-being" and Goal 10 referring to "reducing inequalities": (1) promoting the dissemination of research on factors that can provide safe, nonviolent, inclusive, and effective learning environments for all and (2) presenting interventions that reduce inequality to enhance or promote social inclusion and the improvement of the self-regulation processes of children with neurodevelopmental disorders. The challenges of the future are stimulating, and we want this Special Issue to be an initial approach to continue advancing and creating the best opportunities for the development, health and well-being of all children.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The author declares no conflict of interest.

### **References**


## **Preschool Teachers' Beliefs towards Children with Autism Spectrum Disorder (ASD) in Yemen**

**Sahar Mohammed Taresh 1,2 , Nor Aniza Ahmad 1,\* , Samsilah Roslan <sup>1</sup> and Aini Marina Ma'rof <sup>1</sup>**


Received: 6 July 2020; Accepted: 8 September 2020; Published: 6 October 2020

**Abstract:** It is perplexing that some preschool teachers not only advise parents who have children with autism spectrum disorder (ASD) to go to religious healers, but also attribute such neurological disorders to The curse of The "evil eye" or vaccines. Although it is now The twentieth century, this behavior simply reflects The concerns of over-protective teachers and The cultural misperceptions about The actual definition of ASD. In Yemen, The term "ASD", with its wide range of symptoms, is still ambiguous among preschool teachers. Thus, in a rather insightful piece for The education community, this study has attempted to look beneath The surface of The beliefs (religious belief–social belief–personal belief) of Yemeni preschool teachers regarding ASD. Based on The data collected from 213 teachers (20–30\31–40-~≥40 age) in The Taiz district, this study found that misconceptions specific to autism spectrum disorder were strongly evidenced among teachers who taught preschoolers. Due to personal ignorance and growing superstitions, these teachers tend to believe The society's perceptions of ASD, thus resulting in The ignorance of scientific views. However, The mass media can increase this group's awareness of ASD by continually assessing The inaccurate views on ASD, and correcting them. And by influencing The teachers to take a more conceptual scientific approach in serving their special needs students, furthermore, by informing preschool teachers of children's rights in normal life in The future through providing children with an optimal chance of development by early intervention.

**Keywords:** autism spectrum disorder (ASD); preschool teachers; religious belief; social beliefs; personal beliefs; Yemen

### **1. Introduction**

The Republic of Yemen's decision in 1991 was The first legislation for The care and rehabilitation of children with disabilities in Yemen. In 1999, The Higher Committee for The Care of Handicapped Rehabilitation was established, which launched The Handicapped Care and Rehabilitation Fund. Unfortunately, The Handicapped Care and Rehabilitation Fund considered autism spectrum disorder (ASD) as one of The disability categories. In their strategy for The years 2004–2018, The first goal was to change society's view towards children with disabilities by raising The level of awareness of their rights and capabilities. During that period, Yemen was rapidly improving towards urbanization, while suffering a deteriorating economy. According to The United Nations Human Development Report ranked in 2007, Yemen was estimated to be one of The poorest countries in The world. Specifically, it was rated number 140 out of 182 (United Nations Development Program-Human Development Report 2009). Low admission to education became an essential poverty-related issue in Yemen. According

to The World Bank, 87% of The poor in Yemen are illiterate or have not completed primary school. All of this was reflected in The individuals' beliefs and attitudes towards children with disabilities in general and children with developmental disorders, as is The case with children with ASD. Thus, these beliefs can be used to explain any strange phenomenon, including ASD. Due to a lack of visible symptoms in children with ASD, many may attribute ASD with stigma and misconception. For example, uninformed people may perceive a child with ASD to be someone who looks "normal" but acts voluntarily in ways that violate social norms [1,2]. In a longitudinal study of families with children diagnosed with autism, it was confirmed that autism spectrum disorder is often characterized by pervasive impairments in social interactions, communication skills, and restricted patterns of behaviors and interests [1]. The severity of The symptoms varied significantly; children with ASD often have intellectual abilities within normal borders [3]. Despite their ordinary intelligent abilities, preschoolers with ASD show a prorate weakness in language, social skills, and executive performance, thereby making it tough for them to learn and to forge relationships with their peers. Consequently, these limited capabilities contribute to an increased misunderstanding regarding this disorder, and such misunderstanding is associated with different positive and negative beliefs [4,5].

In recent years, families and communities have been reported to be The primary cause of ASD misunderstanding [6]. Armstrong and Fitzgerald [5] also confirmed that different cultures often use explanations and observations with significant differences in descriptions and classifications to describe The disability. To depict cultural differences, The Multicultural Disability Advocacy Association (MDAA) of NSW, Australia, suggested that while The disability is often explained as a medical concept in Western cultures, other cultures view it as a punishment or gift from a higher power [7].

The above perceptions strongly affect The behavior of disabled children as well as their guardians [8]. Considering The misconception that The disability is related to specific actions, there is a high risk that families and communities deny children diagnosed with autism their rights to receive adequate help [6]. When explanations for The disability are placed outside appropriate interventions, there is also a high risk that The incapacity of The disabled children will be kept private [9].

Previous studies have found that knowledge of ASD is associated with a lower rate of stigma [10–12]. However, this is not always The case because most ASD information that The public receives comes from The media, which is made up of stereotypical beliefs or convictions that people see as truth [13]. However, these beliefs are not backed by sufficient proof [14]. There are many beliefs about autism spectrum disorder that originate from extreme superstition and cultural views. Hence, several investigations have been carried out to get an in-depth explanation regarding The root of The problem. The results of such investigations point toward a pattern of negative opinions towards autism spectrum disorder, whether as a punishment for The family's previous sins, The mother's negligence, or The work of wicked ghosts [15,16]. Similar to an investigation into South Korean society, The psychiatrists were more likely to deduce that "bad" mothers and pregnant mothers who were depressed and introverted caused their child to become autistic [17]. On The other hand, another study determined how five Muslim parents who had children diagnosed with autism interpreted their religious beliefs to empower them to seek an understanding of their children's special needs [18]. Other published studies support The above works, proving that The beliefs that The family is at fault for causing a children's mental disorder are extremely spiritual and religious [10].

The effect of spirituality is amazingly profound among some believing families. Parents or guardians were able to self-heal by accepting that Allah, The Creator of The Universe, had chosen them to be The caretakers of children diagnosed with autism due to their piety, diligence, nobility, and capacity to support. This notion of Allah's involvement does play a role in enabling The affected families to reach a more profound acceptance of family members with disabilities. In other words, a child with ASD brings them closer to their religion and The Creator. It was also found that Pakistani and Bangladeshi Muslim believers learned that not only did Allah put an autistic child in their care as a result of destiny, but also because Allah wished to test their family and to see if they would continue to be excellent

or unkind to The child. Beyond this, however, it seems that teachers, in general, are still plagued by misconceptions. For instance, one previous study on various cultural groups examined The role of culture in influencing teachers' beliefs within an ecological framework—discussing The nature and causes of childhood disability and The teachers' ideas about treatment. Another study revealed that The teachers held both biomedical and socio-cultural views that reflect duality in their beliefs [19]. The view of both parents and teachers is a relevant factor in The above relationship. With their consent, some researchers sought to determine The cognitive, emotional, and developmental characteristics of children with The disorder by assessing The parents' and teachers' beliefs and knowledge about specific areas of The disorder. The views of The parents and teachers were not interrelated nor accurate, which made cooperation between The two parties difficult [20].

People with less experience and knowledge of ASD often hold inaccurate beliefs about The disorder [21]. Based on The discussion above, a lack of knowledge regarding autism spectrum disorder creates a wide avenue for social stigma, which means that children diagnosed with autism and their families may be more stigmatized than other children with disabilities. In a study in which adults and teens with ASD were asked about their challenges, The respondents reported feelings of being evaluated and rejected by their families and friends. Some even stated that they were victims of bullying at social events [22]. Besides, another study evaluated The knowledge of 15 early childhood preservation teachers about ASD adaptation as an entry questionnaire to autism spectrum disorder. Of The 15 elements with true/false reaction options, The results illustrated that pre-service child teachers did not know The subject matter.

Furthermore, they had some misunderstanding about ASD etiology and The behaviors of children with ASD. Regarding etiology, it was found that 93% of preschool teachers did not identify ASD as a developmental disorder, while 60% of them believed that The children could "overcome" their situation. Only 53.3% of them recognized The genetic contribution to The disorder, and 20% of them wrongly indicated The effect of trauma as a cause for The visible behaviors of ASD.

Moreover, 73.3% of pre-service teachers thought that behavioral treatment was not an effective intervention, and 66.7% of them affirmed that children with ASD were entirely similar to others. The results also indicated that 46.7% of them did not identify with The justification for early interventions to help children with ASD, and 26.7% of The participants considered it an erroneous behavioral intervention for a child with ASD. This finding is similar to The results of Barned et al. [23], which indicated that The pre-service preparation for ASD in preschool is inadequate [24].

Another noteworthy study reviewed The beliefs regarding ASD among The general public, including teachers in The United States and Canada (*n* = 823), as well as people dealing with childcare services in The State of Idaho. The results showed that nearly all participants properly understood The genetic and neurological cause of ASD (not parenting, drugs, or a recent diet) that can be recognized early on in a child's life for prompt intervention. The study also emphasized The correct community knowledge of ASD to facilitate early identification and effective intervention. The results suggest professional development courses for childcare providers as well as effective channels for transmitting accurate information such as broadcast and online media from which The general public, especially followers of ethnic minority groups, are most likely to learn about ASD [25].

Theoretically, through The health belief model (HBM), it can explain The preschool teacher's beliefs based on this theory, as this theory describes health-related behaviors and medical decision-making. The HBM was developed initially in The 1950s to explain why people did not participate in preventive disease programs [26,27]. Preschool teachers' beliefs can be a barrier to healthy behavior toward their children, and preschool teachers' belief in their actions can protect their children in The class and estimate when to take action [28].

Interestingly, this data presented a whole collection of findings that differed from those reported in The Eastern world, specifically The Arab world. Hence, it is critical to correct such misunderstanding and stigma surrounding ASD, mostly in countries where ASD services are scarce. ASD in Arab countries such as Yemen has received relatively little attention from The research community so far.

7

Preschool teachers' judgments of The perceived barriers and The perceived benefits of action define The course of action taken; these two components together form The dimension outcome expectations [28]. This includes preschool teachers' "perceived costs involved in seeking a diagnosis (e.g., time, don't have evidence, social stigma, how to voice their concern to parents, not knowing who to contact, refuse The parents, etc.).

A current analysis of ASD research in Arab countries revealed a total of 75 articles being published between 1992 and 2012 [29]. In contrast, The United States has been producing international ASD research articles for years, including 1040 publications in 2010 alone [30]. Thus, there is an urgent need for further research, as in The case of The current study, to assess teachers' beliefs about ASD in Yemen.

### *1.1. Religious Belief*

Beliefs and behavioral health are rooted in religion and spirituality. Many religious and spiritual traditions observed in cultures around The world are associated with health practices [31]. Aside from having a profound influence on The views of health and diseases in many cultures, these two elements can also have implications for The field of health communication [32]. In essence, religion affects various social, cultural, and personal aspects, such as traditional norms, values, and customs [33]. Consequently, traditional cultural perspectives inevitably influence ASD treatment recommendations. One important aspect to consider when discussing ASD from a cultural perspective is that there could be many weird explanations. While ASD is clearly defined by its visible features in all cultures, The systems of cultural and religious belief vary drastically and, therefore, create a different experience of illness with different religious perspectives.

Among Muslim families, it is generally recognized that Allah puts an autistic child under their care not only because of fate or reincarnation, but also because Allah wants to test that particular family to see if they can care for The child. This concept, quite simply, prohibits any inhumane treatment or immoral behavior towards children with ASD [10]. Some families, on The other hand, embrace The child's disabilities as qadar/kismat (fate) [34], a situation which is similar to that reported by other Muslim families in other studies, especially because religious considerations have long been taken at The family level. For example, Muslim mothers in Turkey mentioned Allah, fate, spells, and evil spirits as The causes for having disabled children, while strong traditional beliefs exist concerning The possibility of a child's restoration, which in turn determines their actions relating to help or treatment [35]. In a small part of The Indian subcontinent, various cultural and ritual superstitions surround The disabled. Some Punjabi Muslims believe that treating people who are either called defective or crazy, will ensure their direct admission to heaven when they die. Some also blame evil crimes for The health deterioration of children [4].

Besides, Pakistani and Bangladeshi Muslims believe that Allah wants The parents of children diagnosed with autism to play a pivotal role in improving their children's life skills through various forms of therapy. The Muslim community here trusts that a suitable therapy is one that improves both The health and The soul [36]. However, some traditional families become angry at The Western point of view on ASD treatments and exclude their children diagnosed with autism from a proper diagnosis [10]. For instance, parents would refuse to work with certified professionals, focusing on The child's weaknesses instead of highlighting The child's potential [10]. This calls for a form of willingness to find religious healers who ignore The negative side and focus on The positive side of ASD instead [10]. As part of their spiritual belief system [4], Pakistani and Bangladeshi Muslims have also resorted to prayers and pilgrimage to seek guidance in helping their children diagnosed with autism reach their maximum potential and mission in life [17].

Religious implications on beliefs concerning children with developmental problems are not only limited to Muslims believers [10] but also include other religious groups. As reported in a previous study, 55% of Latina mothers believe that their autistic child is a sign of Allah's existence [37], while other Latina mothers believe them to be blessings or gifts from Allah that would give them The chance to do good and to surrender parts of their lives to serve others [38]. Furthermore, it was found that

Latin Americans have The option of opting for non-traditional treatments, such as The use of folk healers [39].

Meanwhile, numerous Hindu parents of children with "mental disorders" in The United States believe that Allah has given them The child as a response to sins committed in their previous lives [40]. While white Americans use traditional treatments and professional services [39], many African-Americans would seek advice from friends, families or church members before going to professionals. Asians, on The other hand, are often hesitant to seek professional assistance; they are more likely to "go alone" and get help if they cannot manage their child [38,39].

In contrast, it was found that ultra-orthodox Jewish families often change community dynamics by receiving medical advice from a Rabbi [41]. If a given treatment modality is contradictory, then The advice of The Rabbi is followed [10,42] regardless of whether insights into The disability are deemed as pessimistic or a celebration of life. The underlying concept, however, rests on The fact that religion often plays an important role in understanding The ASD experience [10,43,44], and therefore, families who believe that ASD can be cured tend to follow The mandate of The treatment designed to cure, while unbelieving families are less likely to challenge The course of The disorder [45].

The above discussion highlights The importance of religion. Based on The available knowledge on culture, religion can help families emotionally and socially, while playing an important role in boosting The capacities of parents to care for their child. However, religion may be misinterpreted, thus affecting The way families make decisions about further treatment and help.

### *1.2. Social Belief*

Culture affects an individual's common belief system and serves as an explanatory model for disorders such as ASD [31]. In spite of The contrasting interests with which ASD has to contend, multiculturalists assessed The pressures of ASD differently, and these evaluations offer both negative and positive reviews [31]. On The negative side, culture makes people believe that ASD is a stigma. A stigma is known as The manifestation of a diverse form [46]. It is discriminating in nature and does not correspond to The normative expectations of society, and therefore results in a deteriorating social identity for affected individuals or even groups. Stereotyping against those with disabilities often occurs because of The negative perception of ASD [47]. The stigma surrounding ASD has resulted in discrimination against not only children diagnosed with autism but also their families [48]. It is often negatively perceived that children with ASD would find it hard to achieve success in life. Some parents go to The devastating extent of hiding their children diagnosed with autism from The community to prevent The family from losing respect [31].

In other communities, The disabled group is pushed to social exclusion and often their quality of life is put at risk [31]. Discrimination against children with ASD has been broadly described in The United States and around The globe to acknowledge The burden that stigma can bring to both parents and children with ASD [49].

Because The reason for ASD is still unknown, numerous multicultural groups have propped up their own belief about The reasons for developmental disorders. When people's beliefs about ASD etiology were reviewed, it was found that people, including preschool teachers, believed that genetic, environmental factors, and birth-related events were The contributing factors. Despite having been debunked in multiple extensive studies, anecdotes from parents support The suggestion that vaccines can cause developmental disorders in children. Millions of parents firmly believe that genetics, birth trauma, illness, inheritance, perinatal damage, The environment, or some combination of these disorders could cause children to become autistic [50,51]. However, many parents also believe that genetic factors are The main cause of these disorders [52,53]. Given The vast storehouse of myths and half-truths, more scientific inquiries on various cultural groups and their genetic and biological make-up will help confirm The causes of ASD. Cultures have often perpetuated false beliefs in The field of ASD because different groups may have a specific way of explaining atypical behaviors and beliefs [54].

### *1.3. Personal Belief*

Beliefs inevitably take on a more influential role than knowledge in determining how people organize and identify tasks and problems. Moreover, beliefs are The strongest indicators of people's behavior [55]. Preschool teachers often stigmatize children with ASD, highlighting individual perceptions that may be incorrectly ascribed to people with disabilities. For some people, The regularity of The education system may be The most important factor in determining their beliefs [33].

In a study among special administrative region university students living in Macau, most of The participating students confirmed that The ASD etiology stemmed from negligent and emotional parents. However, only about one third of The students believed in genetic etiology. There was also a significant difference in The strength of each belief. On average, The participants expressed mild-to-moderate agreement with statements describing paternity as The etiology for ASD. Instead, they responded with a slightly neutral opinion or reaction to statements regarding genetic factors as The etiology of ASD [54]. Furthermore, a study by Hoekstra et al. proved that The personal beliefs of preschool teachers influences The transference of information to The parents of an ASD child [56].

In Arab countries, Hasnain et al. (2011) highlighted The commonly used Arabic word for ASD (Ygñ JË@), as individuals with a behavioral, mental, physical, and/or emotional disability. However, for many, The word itself is commonly interpreted as "to introvert" or "withdraw". Therefore, many individuals may have incorrectly interpreted The nature of ASD as introversion [31]. In China, a lack of ASD knowledge has been confirmed among preschool teachers. Similarly, this lack of knowledge has been reported due to The literal translation of both Chinese terms for autism spectrum disorder —zibizheng Gud¯ an—as "loneliness" or "introvert disease", which implies a more misunderstand form ¯ of psychological etiology [54].

The aforementioned findings provide support for The positive belief that children with ASD are talented in art, music, language, and computing [21]. However, figuratively, it is believed that children with or without ASD may possess extraordinary abilities. Moreover, their sophisticated skills may have been confused with general interests in public, social, and scientific discourses. Thus, having in-depth knowledge about ASD is equally important for members of The society, including preschool teachers. Countering misconceptions is best done through professional training that focuses on equipping teachers with The skills they need to improve The experience of children with ASD [33].

Needless to say, researchers have pointed out The importance of conventional directions—what preschool teachers have traditionally believed, their religious, social, and personal views on ASD, as well The relationship between several independent variables such as their education level and teaching experience and one dependent variable, which is a preschool teacher's belief about autism spectrum disorder. Therefore this study aims to answer The following two questions:


### **2. Methodology**

### *2.1. Research Design*

A quantitative descriptive survey research design was utilized in this study to assess The different forms of beliefs of preschool teachers in general education towards ASD in Yemen. This design has been used in some previous studies such those of Qi et al. and Chirico et al. [54,57] to assess belief about ASD among teachers and parents as well.

### *2.2. Instrument*

Overall, The instrument used in this study, specifically The ASD Beliefs Questionnaire (ABQ), consisted of two sections. The items in The developed instrument were adapted from questionnaires

by Harrison et al. [58] and Al-Sharbati et al. [59], with their permission. The first section focused on The demographic information of The respondents (i.e., gender, age, marital status, education level, and teaching experience). The second section consisted of 20 ABQ items to assess The religious (five items), social (seven items), and personal (eight items) beliefs of preschool teachers towards ASD. For this section, The respondents were required to provide their responses according to a five-point Likert scale with The endpoints of " from strongly disagree" (1) to "strongly agree" (5). In addition, a "less agree" option was included in The response scale, instead of "neither agree nor disagree," in order to avoid respondents trying to guess their responses.

Firstly, The religious belief, in this case, represented one's belief on how ASD and behavioral health disorders are rooted in religion and spirituality. Examples of religious belief include "ASD is likely a result of a curse or evil eye put upon or inflicted on The family" or "I think it is possible to treat ASD by consulting with religious therapists". Secondly, The social belief, in this case, represented one's common belief based on The cultural effect as an explanatory model system for ASD. Examples of social beliefs include "My society thinks that vaccination causes ASD" or "ASD holds a social stigma in some communities, such as Yemen society". Lastly, The personal belief, in this case, represented one's perception (maybe correct or incorrect) on children with ASD. Examples of personal beliefs include, "I think The majority of children with ASD suffer from mental retardation" or "I believe ASD can develop due to child maltreatment".

For this study, The developed instrument was back-to-back translated carefully. The equivalence of The translation was first reviewed by a panel of five experts (special educational needs teachers). Meanwhile, The face validity of The instrument was initially verified by four education specialists (two psychologists and two psychiatrists) who were professionally trained in The field of special education. This group of professors rated The clarity and appropriateness of The Likert scale statements. Necessary adjustments, including The rewording of certain phrases, were made according to The group's observations and suggestions. Following The adjustments, The percentage of The agreement from The group achieved 0.94. The face validity of The Yemeni version of The Likert scale statements was subsequently confirmed during The pilot study that involved 45 preschool teachers. All items were revealed to be easily understood, and no changes in wording were needed. The responses during The pilot study were not included in The actual study. Besides, The internal consistency of The instrument recorded Cronbach's alpha coefficient of more than 0.83, which reaffirmed The reliability of the items.

### *2.3. Procedure*

The school administrators in one of The biggest cities in Yemen, specifically The Taiz district, were contacted to obtain their permission to conduct this study that involved their preschool teachers. After obtaining The permission of school administrators to conduct this study, preschool teachers were randomly selected to complete a self-administered ABQ on their beliefs towards ASD. The data collection was conducted during The summer of 2018. A total of 250 preschool teachers were randomly selected to complete a questionnaire on their beliefs towards ASD. All questionnaire sets were then returned to The researcher. However, 37 returned questionnaire sets had missing information. The exclusion of these incomplete questionnaire sets resulted in a final sample of 213 preschool teachers.

### *2.4. Data Analysis*

The categorization of scores in this study represented The different levels of beliefs of preschool teachers, specifically religious, social, and personal beliefs, towards ASD. The recorded scores were interpreted according to three distinct levels: (1) low level of accurate beliefs (mean value ≥ 3.34); (2) moderate level of accurate beliefs (mean value of between 1.67 and 3.33); (3) high level of accurate beliefs (mean value ≤ 1.66) [60]. In other words, a low level of accurate belief implies that The preschool teacher displays disbelief towards ASD; a moderate level of accurate belief implies that The preschool

teacher displays some misbelief towards ASD; a high level of accurate belief implies that The preschool teacher displays an accurate belief towards ASD.

Accordingly, The obtained survey responses were analyzed using IBM SPSS (version 19). First, The demographic information of The participating preschool teachers as survey respondents was descriptively analyzed. Second, The reliability and validity of ABQ were determined based on The results of Cronbach's alpha coefficient. Third, after ensuring that The obtained data were normally distributed, multilinear regression was carried out to see The differences among dependent variables (religious, social, and personal). Two-way ANOVA was carried out to assess The relationship between independent variables (i.e., age, education level, and teaching experience) and The dependent variable (personal belief). Besides this, post-hoc analysis (Scheffé post-hoc criterion for significance) was also performed.

On The other hand, data for The other dependent variables (religious belief and social belief) reported non-normal distribution. Moreover, The homogeneity equality of variance was less than 0.05 (*p* = 0.042). Therefore, The non-parametric test—The Kruskal–Wallis test (K–W test)—was valid and used instead of The ANOVA test. Thus, it was concluded that assumptions for The two-way ANOVA had been violated. The transformation was rather difficult because The specimens were small and difficult to interpret, especially in The two-way ANOVA. Therefore, The non-parametric test (K–W test) was valid and applied for religious belief and social belief [60].

### **3. Results**

The respondents of this study were preschool teachers in normal schools in Taiz. Only 5% of The participants have reported that they have had previous contact with children with ASD, which was expressed through The (Yes, No) open question "Have you ever contacted a child with autism?". Table 1 presents The sample distribution according to gender, age, education level, and teaching experience.


**Table 1.** Sample characteristics.

### *3.1. Preschool Teacher's Belief Regarding Autism Spectrum Disorder*

The first research question focused on determining The beliefs of preschool teachers towards ASD. With respect to this research question, The descriptive statistics (i.e., mean, standard deviation, and weight mean) of The categorical data in this study were acquired. And multilinear regression to see The differences lie between dependent variables (religious, social, and personal).

### 3.1.1. Descriptive Analysis

Table 2 shows The weight means for each belief among The preschool teachers according to three distinct levels for interpretation. In particular, The religious belief regarding preschool teachers' belief towards ASD was found to be at a moderate level, whereas their social and personal belief of preschool teacher's belief towards ASD was located at a low level. In other words, preschool teachers display inaccurate beliefs towards ASD resulting from The religious, social, and personal beliefs of The preschool teachers.


**Table 2.** Preschool teachers' beliefs regarding autism spectrum disorder.

std: standard deviation.

### 3.1.2. Results of Multi Linear Regression

Multiple Linear Regression analysis is an appropriate statistical method that allows us to examine and understand how The relationship among The independent variables are related to The dependent variable and to explore The forms of these relationships (between two or more variables of interest). Regression analysis is also used to understand which among The independent variables are related to The dependent variable, and to explore The forms of these relationships.

One-way ANOVA was employed to determine The effects of The dependent factor on The two independent factors as shown in Table 3. The F statistic was 9.880, indicating that The independent's variables (social belief, personal belief) are statistically significant. The results showed that The two independent factors (social belief, personal belief) have an effect on The dependent factor (religious belief) (*p*-values < 0.000) as illustrated in The same Table 3. This reflects that The alternative hypothesis of 3 variables are accepted, and that both social and personal beliefs were associated with religious belief.


**Table 3.** ANOVA test between religious belief (DV) and both social and personal belief (IV).

df: degrees of freedom.

The structural model assessment, as shown in Table 4, provides an indication of The hypotheses tests or question research of this current study. If hypothesis H1 (β = 0.142, t = 4.234, *p* = 0.000) and H2 (β = 0.089, t = 2.121, *p* = 0.035) are accepted, these could be The correlation relationships between Social Belief and Personal Belief empathy which significantly predicts that The Religion Belief dependent variable holds a strongly positive correlation relationship which is highly significant at The 0.01 level (2-tailed).

**Table 4.** Coefficients of The structural model assessment.


std: standard deviation.

The second research question was to assess The potential differences in The beliefs of preschool teachers towards ASD according to their age, education level, and teaching experience as shown below.

### *3.2. The Interactiont of Age, Education Levels, and Teaching Experience on Religious Believe*

Based on Table 5; The Kruskal–Wallis Test (K–W test) interaction effect results indicated that there was a significant difference in religious belief between age (20–30 and 31–40 and above 40 years) and education levels (Diploma and Bachelor) (*p* = 0.023; *p* = 0.017). Moreover, there was not a significant difference in preschool teachers' belief regarding religious belief between age and education level (high school) (*p* = 0.115). The mean score for The age between 20–30 was less than The mean score for The age between 31–40 years. Thus, it was concluded that The age between 20–30 years and above 40 years had lesser inaccurate beliefs than The age between 31–40 years.

Moreover, The results indicated that there was a significant difference in preschool teachers' beliefs based on religious belief between ages (above 40 years) and teaching experience (above 10 years) (*p* = 0.001). There was not a significant difference in preschool teachers' belief regarding religious belief towards ASD between age and teaching experience (from 5 years and below and between 5–10 years) (*p* = 0.238; *p* = 0.226). This is shown in Table 6 below.


**Table 5.** Kruskal–Wallis Test (K–W) test for religious belief (age with education levels).

df: degrees of freedom.

**Table 6.** Kruskal–Wallis Test for religious belief (age with teaching experience).


df: degrees of freedom.

Based on The K–W test, there is a significant difference between The mean scoring of teaching experiences (from 5 and below and between 5–10 years) and education levels (*p* = 0.023; *p* = 0.000) on preschool teachers' beliefs based on religious belief. However, there is no significantly different between teaching experiences (above 10 years) and education levels (*p* = 0.920) on preschool teachers' beliefs based on religious belief towards ASD, as it follows in Table 7.


**Table 7.** Kruskal–Wallis Test for religious belief (teaching experiences and education levels).

df: degrees of freedom.

### *3.3. The Interaction E*ff*ect of Age, Education Levels, and Teaching Experiences on Social Belief*

The K–W test interaction effect results indicated that there was a significant difference in preschool teachers' beliefs regarding social belief between ages and education levels (diploma) (*p* = 0.006). Moreover, there was not a significant difference in preschool teachers' beliefs regarding social belief between age and education levels (high school and Bachelor) (*p* = 0.130; *p* = 0.116). The mean score for The age between 20–30 was less than The mean score for The age between 31–40 years. Thus, it was concluded that The age between 20–30 years and above 40 years had a lesser effect on preschool teachers' beliefs based on social belief than The age between 31–40 years, as is shown in Table 8 below.


**Table 8.** Kruskal-Wallis Test for social belief (age with education levels).

df: degrees of freedom.

The results also indicated that there was a significant effect of social belief on preschool teachers' belief between age and teaching experience (between 5–10 years and above 10 years) (*p* = 0.002; *p* = 0.002). There was not a significant effect of social belief on preschool teachers' beliefs between age and teaching experiences (from 5 years) (*p* = 0.542). See Table 9


**Table 9.** Kruskal–Wallis Test for social belief (age with teaching experience).

df: degrees of freedom.

According to The K–W test, there is a significant difference between The mean soring of teaching experiences (between 5–10 years above 10 years) and education levels (*p* = 0.000; *p* = 0.006) on preschool teachers' beliefs based on social belief. However, there is no significant difference between teaching experience (from 5 years and below) and education levels (*p* = 0.237) on preschool teachers' beliefs based on social beliefs. See Table 10

**Table 10.** Kruskal–Wallis Test for social belief (teaching experiences and education levels).


df: degrees of freedom.

### *3.4. The Interaction E*ff*ect of Age, Education Levels, and Teaching Experiences on Personal Belief*

Two-way ANOVA was employed to determine The effects of age, education levels, and teaching experience on personal beliefs. An assumption of two-way ANOVA was checked, and The normality test was also checked for each factor. Each factor showed either that it was normally distributed or slightly skewed positively, which was in The accepted range of skewness. Homogeneity equality of variance was checked using Levene's test and it was found that The equality of variance assumption was met (*p* = 0.215). Thus, it can be concluded that it had not violated The homogeneity of variance assumption for 2-way ANOVA.

As seen in Table 11, The age effect (age between 20–30, age between 31–40, and age above 40 years) and working experience have no significant effect on preschool teachers' beliefs based on personal belief (*p* = 0.192; *p* = 0.859). Only The effect of education levels has a significant effect on The scores of personal belief (*p* = 0.002).


**Table 11.** Two-way ANOVA for personal belief.

df: degrees of freedom. F = variation between sample means / variation within The samplest. \* Means it is significant. R Squared = 0.312 (Adjusted R Squared = 0.252).

However, when a two-way interaction effect between each study factor was checked, it was found that there was a significant interaction effect between age factors with education levels and teaching experience on The scores relating to personal belief (*p* = 0.033; *p* = 0.018). It indicated that The effect of The age effect on personal beliefs may differ between education levels and teaching experiences. The results also found that there is no significant effect between education levels and teaching experience (*p* = 0.637).

Regarding The result of The post-hoc Scheffe's test, and based on The personal beliefs of preschool teachers (aged 31–40), there is a significant effect (mean = 2.57; *p* = 0.002) on preschool teachers' beliefs towards ASD. In addition, The personal beliefs of preschool teachers with a high school degree have significant differences (mean = 6.39; *p* = 0.000) on preschool teachers' beliefs towards ASD. Furthermore, The personal beliefs of preschool teachers who have teaching experience below 5 years have a significant effect on (mean = 2.02; *p* = 0.021) preschool teachers' belief towards children with ASD.

### **4. Discussion**

The current study aimed to gain a better understanding of The impact of religious, social, and personal beliefs on one's perception of etiology, symptoms, signs, and socio-demography correlating to ASD. Overall, The survey conducted in this study revealed that many preschool teachers had misconceptions about ASD. They also repeatedly associated ASD with religion and believed that religious healers could treat children with an autism spectrum disorder. Some also believed that traditional therapy might be useful, as ASD was an atonement for previous sins [61]. Family members or relatives have sought help from traditional healers and shamans, including Rabbis [62], in part due to The lack of a known cure for ASD. Another reason why most parents struggle with finding The right help for ASD is that they believe that ASD is a scourge from Allah and that The disorder is a result of The curse of The evil eye. Neni et al. [63] validated this in their research which demonstrated that 18.2% from their sample of members of The public still believed that evil spirits caused The disability. These findings indicate that religion in many types of culture and practices is known to serve as an explanatory system to provide a reason for The causes or treatment of ASD.

Religious beliefs of preschool teachers aged above 40 years with teaching experience above 10 years have a significant effect on preschool teachers' beliefs towards ASD. In other words, The religious beliefs of those aged above 40 with teaching experience above 10 years were The most significant. Their religious beliefs caused inaccurate beliefs toward children with ASD. The reason for that is that older generations (who have more education and more teaching experience) were raised in an era that had strong religious/traditional interpretations of health. These findings indicate that in Yemen, religion plays a major role in identifying The causes and treatment of ASD. Yemen is among The Muslim countries that belief traditional therapy to be useful for treating cases such as ASD. The most common traditional therapy applied is Quran therapy.

On The other hand, The religious beliefs of preschool teachers with an (high school) education level and teaching experiences (below 5 years and between 5–10 years) have no significant effect on their belief toward children with ASD. Based on their religious beliefs, these preschool teachers had accurate beliefs about ASD. This result could be explained by The fact that preschool teachers with low levels of education attempt to seek more knowledge through attending workshops and training programs that can help them deal with children who have problematic behavior instead of relying only on their religious interpretations.

This study revealed many common social assumptions on ASD among preschool teachers. Their contradicting views are debatable. Some teachers with many years of experience had more accurate beliefs on ASD than those with fewer years of teaching. In addition, more experienced teachers did not consider or view vaccination as a cause of ASD when compared to less-experienced teachers. Many of them believed that ASD was a genetic disorder. Mitchell et al. [25] found that most preschool teachers stayed up-to-date through social media, and yet, they still had inaccurate information about ASD; for instance, The assumption that ASD might be linked to vaccines. Most preschool teachers with less than five years of teaching experience claimed that children with ASD are indeed geniuses with distinguished skills, confirming The results reported by Khanna et al. [64] The preschool teachers in The study also believed that children with ASD were susceptible to negative social consequences and general stigma. Similarly, Al Sharbati et al. [59] pointed out that children with disability challenges in Oman were once hidden within The familial household and had limited access to educational or remedial services.

The social beliefs of preschool teachers aged 31–40 with an educational level (diploma) as well as teaching experience (between 5–10 years and above 10 years) have a significant effect on The accurate belief of preschool teachers towards children with ASD. In other words, The social beliefs of preschool teachers aged 31–40 with educational levels (diploma) as well as teaching experience (between 5–10 years and above 10 years) were The most significant. Their social beliefs caused inaccurate beliefs toward children with ASD. This might be due to preschool teachers with more than 10 years of teaching experience being uninterested in updating their knowledge compared to those who have less than 5 years of teaching experience. Moreover, low-experienced preschool teachers tend to update their knowledge to improve their careers by attending more training programs [2]. Furthermore, The Yemeni culture has social restrictions that are reflected in The preschool teachers' interpretations of any phenomenon such as ASD. For instance, some preschool teachers consider ASD to be a stigma, which is very common in Yemeni society [59]. These inaccurate beliefs held by many Yemeni people might shape The preschool teachers' cultural beliefs affecting their career.

On The contrary, based on social beliefs, The Age effect (20–30; 31–40; >40) with The education levels (high school–Bachelor) as well as teaching experience (below 5 years) have no effect on The accurate belief of preschool teachers towards children with ASD. This may be due to those preschool teachers with less than 5 years of teaching experience, and a high school education being more concerned about gaining basic teaching and behavior modification skills [23]. It may be The case that preschool teachers who have a high education level (Bachelor) may have received university courses in special education needs.

The preschool teachers had personal assumptions or perceptions on ASD that they had formed through thoughts and experiences. This study found that some teachers thought cell phones were a reason for The emergence of ASD and that most children with ASD were introverts. Such findings confirm The results of John et al. [21] in that those preschool teachers with less experience might find it difficult to distinguish between ASD and introversion, thus thinking that they are similar. Another common personal belief was that an autistic child is suffering from "loneliness"—an error also prevalent among preschool teachers in China [33]. The Chinese teachers' inaccurate response to ASD may partially stem from The Chinese terms for ASD that translates to "loneliness disease" or "isolation disease'", implying a more psychological etiology. Some Arabic terms like ((Ygñ JË@translated as "loneliness", explains why some of The preschool teachers in this study equated this disorder with The feeling of being lonely. However, The response rates indicate that this disorder could be cured, so training in early childhood development would be adequate to provide teachers with a knowledge base in neurodevelopmental disorders, as recommended by Liu et al. [33]

The personal beliefs of preschool teachers aged (31–40) with an educational level (high school) as well as teaching experience (below 5 years) have a significant effect on The accurate belief of preschool teachers towards children with ASD. In other words, The personal beliefs of preschool teachers aged (31–40) with an educational level (high school) as well as teaching experience (below 5 years) were The most significant. Their personal beliefs caused inaccurate beliefs toward children with ASD. This means that preschool teachers with high teaching experience can change their thinking towards ASD better than preschool teachers with teaching experiences for less than 5 years. On The other hand, The personal beliefs of preschool teachers aged 20–30 (>40) with an education level (Diploma and Bachelor) have no significant effect on their belief towards children with ASD. This indicates that they have accurate beliefs towards ASD based on their personal beliefs. This might be due to personal beliefs being a reflection of individual perceptions gained from life experiences and training programs, which may lead to logical thinking toward any phenomena.

### **5. Limitations**

Despite The importance of The above findings, there some limitations to this study that should be highlighted for future research. The preschool teachers in this study were mostly from normal schools, which might affect The generalizability of The findings of this study. Indeed, The index of beliefs expressed in this study was subjectively informed. There was little evidence to suggest that The beliefs were translated into actions. Therefore, it is difficult to know whether The teachers' beliefs or attitudes would translate into actual behavior. To avoid giving a rather low overall impression of their belief system, The teachers might have responded with what they perceived as favorable answers. A joint effort to rule out such confounding factors is necessary, with a need for large-scale future studies equipped with standardized training techniques that reach a broader demographic of teachers. Furthermore, The respondents were asked to state their choices in The comment section of The questionnaire. This method might have encouraged them to respond positively, even if they were not sure about a particular item, as explained in other studies [65]. It might have been better to use another method, such as a qualitative approach, by asking The respondents to explain their beliefs concerning The different issues surrounding ASD. Finally, only 5% of The preschool teachers of The sample had had contact with children diagnosed with autism, which erodes negative views. Similar to The interaction hypothesis by Brown et al. [66], several works also proposed that having previous interpersonal contacts with a disabled person would likely reduce negative views. This situation is based on The assumption that one side of The personality is uncovered to The other side, while new understanding emerges as prejudice weakens. Thus, if preschool teachers have had more contact, their understanding would have been more positively affected. Future studies could explore The contact versus non-contact factor, which may have a bearing on The teachers' understanding of The condition that is unique to ASD children.

### **6. Conclusions**

The prevalence of children with ASD now exceeds that of Down's syndrome, diabetes, or cancer [67]. Hence, preschool teachers should widen their knowledge on ASD to help parents guide their special-needs children. Educated preschool teachers can help parents obtain proper interventions to ensure that their children reach their full potential. To accomplish this end, preschool teachers need to be correctly educated on this disorder, and any inaccurate information among them in particular and in society generally must be discarded. Perhaps a pre-service training among teachers could address these misconceptions and allow easier access for teachers to obtain such important information. Additionally, special-needs education via lectures, workshops, and courses are urgently needed to improve The mindfulness of people working in schools. Mass media can contribute actively by raising awareness regarding ASD knowledge and on how educators can play their roles effectively.

For The first time, this study has highlighted one of The main educational concerns in Yemen. One conclusion that can be drawn is that incorrect beliefs or misunderstandings of The condition

that is unique to children with ASD are social problems that transcend specific culture, geography, or society. It concedes to individual countries The right to work hand-in-hand with others to make ASD education compulsory and jointly face The ever-rising ASD misconceptions and in addition, to give these kinds of children complete rights for social justice in education and jobs.

**Author Contributions:** N.A.A. is The chair person and main supervisor, S.R. and A.M.M. is The committee member of The project. They contribute to The conception and design of The study, gives a critical revision of The article and give final approval of The version to be published. Writing an article, data collection, data analysis and findings interpretation contributed by S.M.T. and N.A.A. All authors have read and agreed to The published version of the manuscript.

**Funding:** This research received no external funding.

**Acknowledgments:** The authors would like to thank all The preschool teachers who participated in this project. Nor Aniza Ahmad and Sahar Taresh are The joint first-authors of this article.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **Development of a Pathway for Multidisciplinary Neurodevelopmental Assessment and Diagnosis in Children and Young People**

**Marion Rutherford <sup>1</sup> , Donald Maciver 1,\*, Lorna Johnston 1,2, Susan Prior <sup>1</sup> and Kirsty Forsyth <sup>1</sup>**


**Abstract:** There is a variable standard of access to quality neurodevelopmental assessment and diagnosis. People may have negative experiences, encountering lengthy waiting times, and inconsistent practices. Practitioners need guidance on standards and practices for assessment and diagnosis matched to new ways of working. In this paper, we present a new pathway and recommendations for multidisciplinary neurodevelopmental assessment and diagnosis for children and young people (<19 years), developed by the Scottish Government funded National Autism Implementation Team (NAIT). Our research used the Medical Research Council guidance for the development of complex interventions and included several iterative stages. Stage 1: *n* = 44 stakeholders attended an event on developing new practices for diagnosis and assessment. Stage 2: a literature synthesis was completed by the research team of clinical guidelines and diagnosis and assessment tools. Stage 3: an event with *n* = 127 stakeholders included discussion and debate of the data from stages 1 and 2. Recommendations and a draft pathway were written. Stage 4: successive drafts of recommendations and the pathway documentation were circulated among an advisory group, including multidisciplinary clinical experts and people with lived experience, until the final pathway was agreed upon. The finalised pathway includes guidance on terminology, assessment, diagnosis, triage, time standards and engagement of people with lived experience. The new pathway has been adopted by the Scottish Government. The pathway and associated documentation are freely available online for use by others.

**Keywords:** guideline; autism; neurodevelopmental; pathway; assessment; diagnosis

### **1. Introduction**

People with neurodevelopmental differences present with variations in one or more neurocognitive functions towards the extreme or otherwise out of the 'normal' range [1,2]. Common diagnoses include autism, developmental coordination disorder (DCD), developmental language disorder (DLD), intellectual disability (ID), foetal alcohol spectrum disorder (FASD), and attention deficit hyperactivity disorder (ADHD) (American Psychological Association (APA), 2013). These categories frequently overlap, and most individuals present with interrelated needs and requirements for support [3].

Accessing timely assessment and diagnosis is a known area of difficulty for people with neurodevelopmental differences, and there are few guidelines available to support practice. Delayed or missed diagnosis reduces access to help, and has a negative impact on wellbeing, participation and quality of life [4–8]. An overly deficit-focused approach is also unhelpful. Diagnostic assessment should include an understanding of the individual in context, and their strengths and goals [9]. An approach which considers 'difference not deficit' is important [10].

**Citation:** Rutherford, M.; Maciver, D.; Johnston, L.; Prior, S.; Forsyth, K. Development of a Pathway for Multidisciplinary Neurodevelopmental Assessment and Diagnosis in Children and Young People. *Children* **2021**, *8*, 1033. https://doi.org/10.3390/ children8111033

Academic Editor: Sari A. Acra

Received: 29 September 2021 Accepted: 8 November 2021 Published: 11 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

There has been a tradition of single condition guidelines and pathways [6,11,12]. The focus has been on the presence or absence of a single diagnosis, leading to communities of practice with knowledge, skill and capacity in relation to these areas (e.g., autism). As a consequence, services have become overspecialised, drastically underestimating the true degree of overlap across neurodevelopmental differences [13]. A singular focus on one presenting issue leads to the potential omission of other diagnoses (and supports). People may experience repetitive and lengthy assessment, a process which individuals and families find distressing and burdensome [4,14].

Movement from single condition pathways toward neurodevelopmental pathways is desirable. Although there are strong advocates for this, for example the 'ESSENCE' framework [3], bridging the gap into practice remains challenging [15]. There was no neurodevelopmental guideline for assessment and diagnosis in Scotland. In this research, commissioned by the Scottish Government, we present our work to address this gap.

### *1.1. Objective*

With this paper, we present recommendations to guide multidisciplinary practice from pre-referral to diagnosis for children and young people aged 0–18 years. We focus on the 0–18 age range as we wish to develop future materials for post-18-year-old individuals, which will have a greater focus on employment, while this current guidance includes a greater focus on childhood years, covering the periods of primary and secondary education. In this research we have built on the United Kingdom Medical Research Council (MRC) guidance on the development of complex interventions [16] and engaged with stakeholders, professionals and people with lived experience. Recommendations were primarily derived from consultation in Scotland; however, they may be transferability to other countries with similar characteristics or issues.

### *1.2. The Scottish Context*

In Scotland, services report increasing numbers of children and young people meeting the criteria for one or more neurodevelopmental diagnosis. The current prevalence of autism is 1.035% in Scotland [17]. The estimated prevalence of neurodevelopmental disorders (internationally) is around 10% [3]. Other Scottish neurodevelopmental prevalence data is not known; however, an analysis of 2019 school census data found that 15% of primary aged children and 20% of secondary aged children had an additional support need indicative of neurodevelopmental needs, including learning disabilities, learning difficulties, language or communication disorders, autism, social, emotional and behavioural difficulties, and mental health problems [18]. In Scotland, there are defined pathways for autism [19,20] and largely separate pathways for ADHD and other neurodevelopmental differences. Prior to the COVID-19 pandemic, services had begun implementing neurodevelopmental pathways, and others were considering this shift, but little synthesised guidance existed to support this important change in practice.

### **2. Methods**

### *2.1. Team*

This research was completed by the National Autism Implementation Team (NAIT). This team forms part of the 10-year Scottish Government autism strategy [21,22]. NAIT have a leadership role in this strategy, and have a remit to improve autism and neurodevelopmental diagnostic pathways across the lifespan through integration with education (in children) and with the employment sector in adults. The team includes clinical expertise in autism, occupational therapy, speech and language therapy, education, psychiatry, as well as researchers with expertise in complex interventions development and health systems research. The team has completed previous research on autism pathways, diagnosis, wait times, interventions and guidelines [6,7,19,20,23–26].

### *2.2. Medical Research Council Complex Interventions Development Framework*

Intervention development methods were modernised through the publication of the MRC complex interventions development framework [16]. The MRC framework provided a systematic and cyclical approach to the development of new interventions [16]. Whilst quantitative statistical techniques, including the controlled trial or statistical meta-analysis, are important, such methodologies may not always be feasible. Qualitative methods are an important alternative [27,28]. A key aspect of such approaches is stakeholder involvement, which can be used to understand problems, identify priorities, and collaborate to develop solutions [29,30]. The MRC guidance provides a robust standard for intervention development and includes four phases: development, feasibility/piloting, evaluation and implementation [16]. The development phase, as recommended by the MRC framework, focusses on building partnerships, specifying the population, completing a literature review, collaborating with stakeholders, and developing and consulting on materials to support implementation [30].

In following the development phase of the MRC framework, an iterative multi-stage process was undertaken 2019–2021. This began with stakeholder engagement, followed by a literature review, further stakeholder engagement, and recruitment of an advisory group. We attempted to find and synthesise data focused on the research questions as listed below:


### *2.3. Literature Review Methods and Analysis*

The research team completed a rapid review of clinical guidelines and available diagnosis and assessment tools. We began searches in December 2019 of Cochrane, Medline, CINAHL, PubMed and Google for pre-existing clinical guidelines and diagnosis and assessment tools for neurodevelopmental diagnoses: autism, developmental coordination disorder (DCD), developmental language disorder (DLD), intellectual disability (ID), foetal alcohol spectrum disorder (FASD) and attention deficit hyperactivity disorder (ADHD). Search terms included ICD-10 and DSM-5 definitions, acronyms and other terminology alternatives for each neurodevelopmental diagnosis, as well as terms for clinical guidelines. The list of assessments for inclusion was developed by consultation with clinicians about local, national and international practice, and consultation with experts, as well as the reviewing of papers to identify commonly used tools in research. Based on these criteria, we identified assessments most commonly studied, most used in research and most used in practice. The focus was therefore on widely used and studied measures, rather than on providing a systematic overview. We revisited the literature throughout the development of the pathway. We set up automatic alerts to monitor the relevant literature for updates.

Summary tables were developed using content analysis and narrative summary methods [31]. Recommendations and guidance were synthesised across the available clinical guidance/guidelines in relation to the key questions of the study. Summaries of assessment tools were prepared, with overview of each tool, users, population, and constructs. A matrix was constructed to indicate which tools were relevant to which neurodevelopmental diagnosis. Tables of diagnostic factors for clinicians were derived. The tables contained descriptions and likely indicators associated with the DSM-5 criteria

for autism, ADHD, ID, DLD, DCD and FASD. A matrix was developed to display the complex set of observations and information required in relation to diagnostic criteria. The information developed was used to facilitate stakeholder discussion, and eventually formed a key part of the pathway documentation.

### *2.4. Stakeholder Engagement Methods and Analysis*

Stakeholders were identified for involvement through professional and user networks, and represented every health-board area of Scotland. Senior professionals and people with lived experience were included throughout. Stakeholders included people with lived experience, families and local autism advocacy groups. Professionals included psychiatrists, psychologists, teachers, occupational therapists, speech and language therapists, paediatricians, and academics. Stakeholders also included members who were involved in government and regulatory bodies.

We used group-work, workshops, discussion and debate with stakeholders, applying interactive and participatory methods [32,33] over a series of face-to-face events with follow up review, phone calls, emails, individual meetings, small group discussion and videoconferencing. The first group of stakeholders (*n* = 44) were invited to attend an initial event. Initial priorities and directions were identified. The key problem under discussion was improvement of diagnosis and assessment, having identified the service and personal costs of missed or delayed diagnosis. There were initial ideas about improvements which could be implemented, and the format or delivery of the proposed pathway. Knowledge about the scope and nature of the problem and possibilities for improvement were discussed, including personal experiences, the work of practitioners, policymakers, and researchers [29,30]. The research team-maintained notes of the discussions, synthesising key themes and main points as discussion documents, which were further shared with the group via email and videoconference. During this period, the review of evidence was undertaken, and draft pathway documentation produced. Next, a larger group of stakeholders (*n* = 127) attended a face-to-face event. This event included review of the draft pathway, review of the literature review outputs, and identification of themes and feedback. This event helped us to understand issues and generate new ideas concerning the pathway. Additionally, "buy-in" was required from the community, with understanding gained on the feasibility, acceptability and potential future engagement of people with the pathway. Each stage included the careful review of collected information and frequent return to primary documents and stakeholders as necessary. Summaries were derived and synthesised using content and thematic analysis [34]. As the analysis progressed, more refined descriptions and recommendations were created, and detailed prototype pathway documentation and content was written.

### *2.5. Advisory Group*

An advisory group (*n* = 25) was recruited to consult on and finalise the pathway, representing an expert multi-disciplinary group comprising occupational therapy, clinical psychology, psychiatry, child and adolescent mental health services, paediatric medicine, general practice (family medicine), teaching, speech and language therapy, government representatives, and people with lived experience. Most individuals in the advisory group had participated in one or both of the previous stakeholder events. This was a coproduction team, with stakeholders participating extensively with researchers to finalise the content and documentation of the pathway. As a team, we regularly communicated to discuss and agree upon content. Debate and review were completed and supported by email and videoconferencing. Three cycles of detailed review of draft documentation were completed with the advisory group (December 2020, February 2021 and May 2021). As stakeholders had stressed the need for high quality documentation, a graphics design process was undertaken when the content had been finalized. The pathway was then made freely available online [35].

### *2.6. Ethics*

All stakeholders were volunteers, and were free to withdraw at any time for any reason. Stakeholders were provided with information about the project and consented to participate in consultation events and activities. People with lived experience were volunteers who self-identified as having neurodevelopmental differences, they self-selected to participate, and were identified through pre-existing health and social care programmes for public and patient involvement and government engagement. This work was carried out in accordance with the relevant ethical standards of institutional and national practice in Scotland, and in-line with the Declaration of Helsinki.

### **3. Results**

The final neurodevelopmental pathway was designed to cross professional boundaries and facilitate collaboration between medical professionals, mental health professionals, education professionals, allied health professionals, service users and families. Supplementary File S1 contains an overview of the full pathway. The content of the guidance is detailed more fully in the sections that follow.

### *3.1. How Should People with Neurodevelopmental Differences Be Identified for Diagnostic Assessment?*

There was consensus that neurodevelopmental assessment and diagnosis should be supported by a multi-disciplinary team (MDT), with suitable training and mix of skills and professional groups. Clinical views, assessment and observations are mapped to ICD 11 or DSM 5 criteria. Most commonly, assessments are likely to result in more than one diagnosis, including autism, ID, DLD, ADHD, DCD and FASD. It was noted that classification systems focus on those who have a specific aetiology and/or extreme impairment. These perspectives were contrasted with a neurodiversity paradigm which rejects the idea of 'normal' neurocognition. There was strong debate, leading to a consensus that it was also helpful to consider that children may require a neurodevelopmental assessment when presenting with a need as below:


It was noted that referrers, parents or other family members may see the situation differently and describe what they 'see,' and not necessarily in terms of specific diagnoses. The following were agreed upon as indicators in need of need of further assessment:


Rather than seeing an 'in-person problem' the position taken within the pathway (taking account of the views of people with lived experience), supports the right to access diagnostic assessment, even if the individual is relatively unaffected [36]. Whilst this may suggest that the right adjustments are already in place, it may also suggest that the individual is masking or camouflaging [37]. There is a need to balance between efficiency and comprehensiveness [38], and diagnostic assessment should also aim to create a detailed profile of strengths and concerns to inform planning. A lifespan perspective is therefore important. Neurodevelopmental differences are enduring, and understanding which anticipatory adaptations might be helpful at different stages is enhanced when diagnosis is clear.

### *3.2. Which Documents and Guidelines Are Relevant to Implementing a Neurodevelopmental Pathway?*

There was no single Scottish neurodevelopmental guideline. Assessment may result in overlapping diagnoses, and Some, but not all, had a guideline. We identified and summarised the available documentation, and drew on these for the pathway (Table 1).


**Table 1.** Clinical guidelines used to develop the pathway.

*3.3. Which Assessment Tools Are Applicable 'Pre-Referral' to Identify a Need for Neurodevelopmental Assessment, Relevant to All Diagnoses?*

Information gathered at the pre-referral stage supports timely and proportionate professional involvement [23]. Avoidance of duplication is desirable. Before commencing a neurodevelopmental assessment, we identified the following suggestions for professionals:


In line with the available clinical guidelines, we recommend that pre-referral information can be gathered through:


• locally-used approaches/tools (these are not diagnostic but provide qualitative information local teams should agree on tools which are used for all individuals, and tools which are used in specific circumstances).

We reviewed and summarised key tools that are commonly used and recommended either in published clinical guidelines or by expert stakeholders, and were applicable to all diagnoses in the pre-referral stage (Table 2).

**Table 2.** Pre-Referral Tools Applicable to all Diagnoses. Adapted from published NAIT pathway [35].


### *3.4. What Are the Key Considerations for Assessment and Triage?*

Standards for formal neurodevelopmental assessment agreed with stakeholders are presented in Table 3. Recommendations focus on early identification of 'core' and 'complex' cases, presented in Table 4. This categorisation was proposed to support proportionate intervention and avoid duplication. In short, if a team are confident of a straightforward conclusion towards diagnosis (i.e., a 'core' case), there is no need for appointments with several sets of professionals. Where there is less complexity, multi-stage assessment may not be required, and assessments are completed locally by a smaller team. Diagnosis can be completed rapidly in most cases. If there is greater complexity, identification of such is completed promptly. More appointments may be necessary, and enhanced staff skill required, with time allocated for additional standardised diagnostic tools and formulation.

**Table 3.** Standards for Neurodevelopmental Assessment. Adapted from the published NAIT pathway [35].

	- a. 'Core'–which can be completed by the team currently involved or other local team
	- b. 'Complex'–which should be completed by a team identified at triage, who have skills required and time allocated or training in additional standardised diagnostic tools and formulation across different diagnoses
	- a. 'Information for you while waiting for your appointment'
	- b. 'Neurodevelopmental information leaflets and website links'
	- c. 'What to expect in an online or face to face appointment'
	- a. each assessment appointment \*
	- b. diagnostic outcome report

<sup>\*</sup> It is noted that in some circumstances a single assessment may be carried out over more than one appointment.


**Table 4.** Complexity factors which may be used to identify 'core' and 'complex' cases. Adapted from published NAIT pathway [35].

The most 'severe' presentation is not the most complex and nor is the 'milder' presentation the most straightforward. The number of complexity factors may or may not add up to a complex case. In different contexts a particular presentation may or may not be complex depending on the clinical team.

It is desirable, both for those referred and in order to deploy resources, that only those for whom it is relevant undergo diagnostic assessment. Triage is commonly used in healthcare for this reason [6]. See Table 5 for triage outcomes and actions. The triage team should include expert practitioners across the age range of different diagnoses and available local services (e.g., paediatrician, psychologist, OT). Frequency and length of meetings to process referrals depends on referral rate. To support the provision of high quality information prior to referral, a standardised information gathering approach is important, ensuring consistent understanding and documentation. It may be necessary for the team completing triage to complete more information gathering before to the first formal appointment (recommended tools are summarised in Table 6).


**Table 5.** Triage outcomes and actions. Adapted from published NAIT pathway [35].

Before triage, check that the following are available: (1) consent for referral, (2) referral form(s), (3) ESSENCE-Q or similar screener, (4) reports on: professional views, reasons for referral; early development and family history (including prenatal alcohol history); parent/carer views about strengths, concerns, expectations; young person's views; Relevant reports/assessments (e.g., occupational therapy).

**Table 6.** Post-triage further information gathering tools, prior to the face-to-face appointment. Adapted from published NAIT pathway [35].


**Table 6.** *Cont.*


*3.5. Which 'Disorder Specific' Assessment Tools Should Be Considered?*

We identified that a neurodevelopmental assessment should include:


Several tools can be applied depending on presenting concerns (see Supplementary File S2 for a full list of tools, commonly used and recommended either in published clinical guidelines or by expert stakeholders). Clinical expertise and consultation across relevant teams can be used to identify the necessary assessments. The assessments can be completed together or separately. Assessments do not need to be completed in sequence, and may not be targeted to a specific presentation or diagnosis.

### *3.6. What Are the Key Considerations for Making a Diagnosis and Reporting This to Individuals and Their Families?*

Information should be brought together and considered in a procedure termed 'formulation' [67]. This was a term familiar to some of our stakeholders, but not all. One benefit of a multi-disciplinary approach is the spread of shared language and practices. Formulation ideally takes account of ideas, thoughts and experiences of the individual and their family members in relation to diagnosis. Diagnoses are then made with regard to international criteria. The research team developed tables of evidence (see Supplementary File S3 for full tables of evidence for each diagnosis) to support clinicians in reviewing assessment evidence and reaching a diagnosis. The tables contain DSM-5 criteria for autism, ADHD, ID, DLD, DCD and FASD. Criteria for each are located in a matrix designed to allow clinicians to view and compare information gathered from different elements in one location. The tables were designed to facilitate a decision as to whether enough information was present to support making a diagnosis. If diagnosis is not possible, the tables may indicate aspects requiring further information/review.

Stakeholders identified that qualitative and experiential aspects of diagnostic processes were extremely important. People respond to diagnosis in a multitude of ways, and an approach that is personalised is recommended. Several opportunities for discussion should be provided in order that discussion can be completed adequately and clarifications can be made if required. However, people and their families maintain the option of both accepting and/or disclosing a diagnosis to others. Any issues, idiosyncrasies or behaviours might not necessarily be seen as troublesome to people and their families. There are also important factors related to time, and potential changes in people (and families) view of diagnosis over a lifetime. There are therefore instances where it is necessary to review a diagnosis. Relatedly, there is a need to have a process to seek out, access or improve new or existing supports across the lifespan, including anticipatory support in advance of major transitions.

See Table 7 for agreed standards for diagnosis and follow up.

**Table 7.** Standards for diagnosis and follow up. Adapted from published NAIT pathway [35].

	- a. the age and stage of their child
	- b. children with neurodevelopmental differences
	- a. Face to face meetings with health and education professionals with expertise in the 'diagnosed' presentations at this age and stage, and with professionals with up to date experience and understanding of the local planning process
	- b. The opportunity to ask questions
	- c. Information relevant to their family, including support to apply for benefits (where appropriate)
	- d. Local support groups, parent information/education sessions and parent mediated interventions
	- e. Access to recommended supports

*3.7. What Time Standards Should Be Applied to a Neurodevelopmental Pathway?*

We developed time standards as a benchmark to report against (summarised in Table 8). Waiting for diagnosis is commonly raised as a major concern for those seeking support. We identified few published or formal time standards for diagnostic assessment. Time standards were therefore developed through consideration of the similar standards for autism e.g., [38,41,68] and consensus views gained from stakeholders.


**Table 8.** Time Standards. Adapted from published NAIT pathway [35].

\* Offer a first appointment to those who have been identified (e.g., by family doctor) as in need of a neurodevelopmental assessment. Each service or local area will have pre-existing standards for responding to initial or safeguarding concerns.

### *3.8. How Can We Involve Lived Experience in the Development of Pathways?*

There is a need for the co-production and involvement of users of services, experts by experience or 'patient-voices' at all stages [9,69] based on the principle of 'nothing about us without us' [70]. When a diagnosis is made, a new, potentially supportive community of people with similar experiences is opened up. Diagnosis matters to people seeking and receiving assessment. However, the process is experienced differently, and sometimes it is positive, while at other times it may be challenging [14,71]. Based on our consultation with representatives of people with recent experience in diagnostic assessment, key points for reflection are summarised in Table 9 (and Supplementary File S4 for further detail).

**Table 9.** Lived experience themes and reflections for professionals. Adapted from published NAIT pathway [35].



**Table 9.** *Cont.*

### **4. Discussion**

In this paper, we shared the process and outcomes in the development of a new pathway for neurodevelopmental assessment and diagnosis. Stakeholder engagement, along with review of current guidelines and assessment tools, allowed the development of recommendations that are feasible and evidence informed. In this article, we have outlined the reasons why a neurodevelopmental approach is desirable, together with the presentation of solutions and recommendations for practitioners and services undertaking this change.

The role of diagnosis may be controversial; however, a preference for timely diagnosis is well-supported in the literature [72–74]. We strongly advocate the perspective that people's needs should be addressed irrespective of diagnosis or 'label'. However, it is often the case that a diagnosis can help with accessing needed supports. In particular, when people are at key points in their lives, when they are accessing new environments, situations or changing circumstances, a diagnosis is helpful in clarifying how neurodevelopmental differences may be impacting an individual, and therefore, the interventions, supports or adaptations required. The need for evidence-informed intervention, particularly nonpharmacological interventions and environmental adaptations, could be made more explicit by a recognised diagnosis [3].

### *4.1. Implications for Practice*

Key points identified from evidence and consensus are summarised below.


### *4.2. Limitations*

Our recommendations are based primarily on expert consensus, although evidence from guidelines has been utilised where possible, and a review of assessment tools undertaken. Recommendations are aimed at children and young people. Adults may share similar features, but their day to day contexts and roles are different. Although this study was completed in Scotland, the demographic and cultural similarities of Scotland to other countries means recommendations are likely to be transferable. There is an ongoing need for service configuration data, and an understanding of ratio and skill mix of staff recommended for a given population. Work is ongoing in Scotland via NAIT to develop this.

### *4.3. Conclusions*

We have developed a practical resource for neurodevelopmental assessment and diagnosis. The recommendations reported here are supported by the Scottish Government. All materials are freely available and recommended for use by those with relevant skills and experience in undertaking neurodevelopmental assessment in children and young people aged 0–18 years.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/10 .3390/children8111033/s1, Supplementary File S1: Pathway; Supplementary File S2: Standardised Assessment Tools Recommended for Particular Presentations; Supplementary File S3: Summary Tables of Evidence for Each NDD Diagnosis; Supplementary File S4: Views from People with Lived Experience.

**Author Contributions:** M.R.: Conceptualization, Methodology, Writing—Original draft preparation, Writing—Reviewing and Editing. D.M.: Conceptualization, Methodology, Writing—Original draft preparation, Writing—Reviewing and Editing. L.J.: Conceptualization, Methodology, Writing— Original draft preparation, Writing—Reviewing and Editing. S.P.: Conceptualization, Methodology; Writing—Reviewing and Editing. K.F.: Conceptualization, Supervision, Funding acquisition. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Scottish Government.

**Institutional Review Board Statement:** This work was carried out in accordance with the relevant ethical standards of institutional and national practice in Scotland, and following the declaration of Helsinki. Ethical approval was not required for this study as it was a collecting service improvement information from patients, carers and clinicians; no identifiable information was collected from respondents.

**Informed Consent Statement:** All stakeholders were volunteers, and were free to withdraw at any time for any reason. Stakeholders were provided with information about the project and consented to participate in consultation events and activities. People with lived experience were volunteers who self-identified as having neurodevelopmental differences, they self-selected to participate, and were identified through pre-existing health and social care programmes for public and patient involvement and government engagement.

**Data Availability Statement:** Please contact the corresponding author for study data.

**Acknowledgments:** We wish to give thanks and acknowledge the contributions of stakeholders, the Scottish Government, people who are experts by experience of neurodevelopmental difference and their families, health and education professionals from across Allied Health Professions, Community Child Health, CAMHS and Education in the development of this resource for all those involved in, delivering or receiving neurodevelopmental assessments. All materials are available freely online https://www.thirdspace.scot/nait/diagnosis-resources/ (accessed on 11 October 2021).

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

### **References**


## *Article* **A Novel Tool to Assess Basic Activities of Daily Living in Spanish Preschoolers**

**Sabina Barrios-Fernandez 1,\* , Margarita Gozalo 2,\* , Andres Garcia-Gomez <sup>3</sup> , Jorge Carlos-Vivas <sup>4</sup> and Dulce Romero-Ayuso <sup>5</sup>**


**Abstract:** Background: Basic activities of daily living (BADLs) are those related to self-care. Their performance depends on the development of sensorimotor and cognitive skills, as well as social and environmental aspects. A good performance in BADLs is required for independence and social participation, so they play an important role in early education and early care. We aim to create a tool for BADLs assessment for Spanish preschoolers. Methods: The tool was administered to 303 participants (48.5% boys and 51.5% girls) between three and six years of age. Analyses to find out the factorial structure and internal consistency was carried out. Results: The instrument was composed of 84 items in four scales (eating, personal hygiene, dressing, and daily functioning) with nine factors (oral sensitivity, good manners, manual dexterity, brushing teeth, toilet management, hygiene and grooming, dressing, higher-order and core executive function). Reliability values were from acceptable to preferred (0.74–0.94). Conclusions: The instrument could be useful and shows preliminary good indicators in construct validity and reliability.

**Keywords:** activities of daily living; executive function; child; evaluation; assessment

### **1. Introduction**

*1.1. Activities of Daily Living Conceptualization and Development*

Activities of daily living (ADLs) refers to a group of tasks that every person carries out to be independent. Concretely, basic activities of daily living (BADLs) refer to the ones oriented toward taking care of one's own body including mobility, feeding, personal hygiene and dressing. Instrumental activities of daily living (IADLs) are activities to support daily life in home and community, often more complex than BADLs, including home management, taking care of others or community mobility [1,2]. BADLs are gradually acquired during childhood, and through practice they become almost automatic, while IADLs are developed through education and practice, with a greater influence on the individual's life roles [3]. The inability to accomplish ADLs may lead to unsafe conditions, lower participation, caregiver overload and poorer quality of life [4].

ADLs development is related to motor, physical, cognitive and emotional areas [5], but also to practical experience and contextual factors. Thus, their outcomes result from the dynamic intersection of the individual, task/activity and context/environment characteristics [6].

Preschool age is a period of huge growth in ADLs development. Initially, infants are completely dependent on their caregivers in terms of care, even talking of co-occupations. Around 2–4 years, they begin to manage their cutlery, as well as simple clothes. Then, about 5–6 years, most typically developing children perform the most essential tasks/activities

**Citation:** Barrios-Fernandez, S.; Gozalo, M.; Garcia-Gomez, A.; Carlos-Vivas, J.; Romero-Ayuso, D. A Novel Tool to Assess Basic Activities of Daily Living in Spanish Preschoolers. *Children* **2021**, *8*, 496. https://doi.org/10.3390/ children8060496

Academic Editor: Laura Simons

Received: 12 May 2021 Accepted: 8 June 2021 Published: 10 June 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

included in BADLs, although they usually need assistance from their caregivers for safety reasons or to initiate them, according to their parenting styles and socio-cultural factors. Setting milestones by age about BADLs acquisition is complex, due to the numerous factors that influence them. Table 1 displays some reference examples [7,8].


**Table 1.** Examples of activities of basic daily living milestones in typical children [7,9,10].

However, children with neurodevelopmental disorders [11] are often unable to reach these milestones, showing significant challenges, poorer outcomes, delays, and impairments compared to their typical development peers [12–15].

### *1.2. Underlying Factors in Activities of Daily Living*

Several factors are important for proper ADLs performance. On the one hand, brain maturation-associated internal factors, especially to the prefrontal lobe, include processes such as perception, memory, or executive function (EF) [16,17]. EF are a set of cognitive skills necessary for goal-oriented behavior. There is some agreement on considering inhibition and interference control, working memory, and cognitive flexibility as core sub-processes [18–20]. Inhibition means being able to control one's attention, thoughts, emotions, or behaviour, suppressing other stimuli. Working memory let us to briefly maintain information while performing other operations. Cognitive flexibility refers to being able to switch between thoughts or actions depending on the demands of the context [21–23]. From core EFs, higher-order EFs are built, including planning (choosing steps to reach a goal), reasoning, and problem-solving [22,24]. Regarding its development, a first phase happens during the first three years of life, where basic skills emerge, and a second one between the third and the fifth years, when different sub-processes begin to coordinate achieving adaptive goals [20,25]. Thus, EF is essential for all ADLs and to succeed in any daily task [26–30].

On the other hand, social and contextual factors are also essentials for ADLs development, including family and school. During childhood, caregivers must provide opportunities for children to practice ADLs in their communities, encouraging their social participation. This repeated practice promotes the establishment of occupational roles and routines, transmitting cultural values to the child [3,31]. Parenting styles are also relevant, considering that democratic styles are associated with greater independence, while overprotection, overcontrol, persistence in performance, or excessive permissiveness negatively affect children's mental health and sense of competence [32].

### *1.3. Activities of Daily Living in Early Education and Early Intervention Services*

In addition to home and community settings, there are two other contexts in which monitoring ADLs development is essential: at school and, when signs of dysfunction are detected in early intervention services. Regarding school, in Spain, preschool education is divided into two stages: 0–3 years, and 3–6 years. Every stage has its own goals, contents, and evaluation criteria. Both stages are structured in three main areas: environment knowledge, languages, and self-knowledge and functional independence [33,34].

Table 2 shows some of the closest contents related to ADLs, including aspects related to EF required for successful performance.

**Table 2.** Activities of daily living related contents in the early education Spanish curriculum [33].


Monitoring children's development is critical so that appropriate actions can be undertaken as early as possible; either through educational adjustments or referring to early care services [35,36]. These services aim to respond to temporary or permanent needs presented by children with developmental disorders or at risk [37], and in planning and carrying out interdisciplinary interventions.

In early education, ADLs performance is assessed by teachers, mainly through students' behaviours observation. Families play an unquestionable role in children's education [38], so teachers must obtain information about ADLs performance of their students in natural environments through their caregivers [34]. Therefore, observational tools or questionnaires completed by caregivers seem to be an interesting tool [39]. They can also be useful for early care therapists since, although therapists commonly work in clinical settings, they also need to collect information about children's performance in their natural environments [10,40–43].

### *1.4. Assessment of Activities of Daily Living in Preschoolers*

Several observational tools can be considered to assess ADLs in children from three to six years: The Vineland Adaptive Behavior Scales [44,45], The Adaptive Behavior Assessment System [46,47], The Checklist of Adaptive Living Skills [48], The Inventory for Client and Agency Planning [49], The Pediatric Evaluation of Disability Inventory-Computer Adaptive Test [50], The Battelle Developmental Inventory [51], or the Merrill-Palmer-Revised Scales of Development [52]. However, these instruments present limitations to be applied during the ADL's evaluation process: (1) some of them are not focused on ADLs construct, but on the concept of adaptive behaviour. It can be problematic as adaptive behaviour is not synonymous with ADL, including different domains and giving more

or less weight to ADLs according to authors' points of view [53]; (2) some of them do not cover the full range of BADLs; while (3) others are translated into Spanish but, to our best knowledge, without performing a cultural adaptation process.

### *1.5. Aim*

This study aims to present the psychometric properties (construct validity and reliability) of a tool to measure BADLs performance in typically developing Spanish preschoolers aged 3–6 years. We hope this tool will (1) help to characterize the BADLs performance of typically developing children serving as a screening instrument and (2) will be useful to detect deviations from normality in the BADLs development of children with neurodevelopmental disorder diagnoses, helping professionals in early education and early care services.

### **2. Materials and Methods**

### *2.1. Study Design*

Participants were recruited through schools and social events in Extremadura (Spain). Furthermore, a convenience clinical sample of 11 participants with autism spectrum disorders (ASD) aged 3–6 years was included to analyze the classification ability of the questionnaire.

### *2.2. Participants*

Three-hundred and three preschoolers with typical development, aged from 3 to 6 years (3 years = 13.2%; 4 years = 26.1%; 5 years = 32%; and 6 years = 28.7%), participated in the study. The sample was composed of 147 boys (48.5%) and 157 girls (51.5%). All participants provided written informed consent before starting data collection. To be included in the study, participants need to meet the following eligibility criteria: (1) age between 3 and 6 years, (2) attend to ordinary schools, (3) no present disorders according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) [11], and (4) provide informed consent.

### *2.3. Instruments and Procedure*

### 2.3.1. Creation of the Basic Activities of Daily Living Assessment in Preschoolers

A group of five experts from the clinical field (occupational therapists, and specialists in developmental psychology and neurodevelopmental disorders), selected for their experience in childcare (clinical and educational) and development of psychological tests, were recruited. Initially, an exhaustive review of the available instruments assessing children development, sensory integration, cognitive assessments, and ADLs was carried out. A rational criterion was followed for the selection of behaviours represented in most of the instruments (achieved or in process). The selected items were classified, developing a pool of 250 items. An operational proposal for the different dimensions was submitted to the experts' judgement. Thus, the experimental version consisted of 113 items. Subsequently, a pilot study was carried out with the participation of 15 families who were asked to answer the questionnaire, assessing the clarity of each item and allowing them to make proposals about wording. They were also informed about the time required to complete the test. Finally, the relationships between the proposed dimensions were explored and the items that did not fit in the model were removed. Thus, the final version of the tool includes 84 items. Figure 1 shows the steps followed for developing the instrument.

*Children* **2021**, *8*, x FOR PEER REVIEW 5 of 18

**Figure 1.** The Basic Activities of Daily Living Assessment in Preschoolers (BADL-P) creation pro-**Figure 1.** The Basic Activities of Daily Living Assessment in Preschoolers (BADL-P) creation process. 2.3.2. Description of the Basic Activities of Daily Living Assessment in Preschoolers Tool

2.3.2. Description of the Basic Activities of Daily Living Assessment in Preschoolers Tool The Basic Activities of Daily Living Evaluation in Preschoolers (BADL-P), a novel

2.3.2. Description of the Basic Activities of Daily Living Assessment in Preschoolers Tool The Basic Activities of Daily Living Evaluation in Preschoolers (BADL-P), a novel questionnaire created for Spanish preschoolers during this study, was used. The BADL-P included 84 items, distributed in 4 scales with 9 factors that provide a theoretical model to support the instrument. Eating, personal hygiene and dressing are BADLs themselves, as explained, while the daily functioning scale includes information about cognitive skills The Basic Activities of Daily Living Evaluation in Preschoolers (BADL-P), a novel questionnaire created for Spanish preschoolers during this study, was used. The BADL-P included 84 items, distributed in 4 scales with 9 factors that provide a theoretical model to support the instrument. Eating, personal hygiene and dressing are BADLs themselves, as explained, while the daily functioning scale includes information about cognitive skills critical for good BADLs performance (Figure 2). questionnaire created for Spanish preschoolers during this study, was used. The BADL-P included 84 items, distributed in 4 scales with 9 factors that provide a theoretical model to support the instrument. Eating, personal hygiene and dressing are BADLs themselves, as explained, while the daily functioning scale includes information about cognitive skills critical for good BADLs performance (Figure 2).

critical for good BADLs performance (Figure 2).

**Figure 2.** The Basic Activities of Daily Living Assessment in Preschoolers (BADL-P) basic structure. **Figure 2.** The Basic Activities of Daily Living Assessment in Preschoolers (BADL-P) basic structure. **Figure 2.** The Basic Activities of Daily Living Assessment in Preschoolers (BADL-P) basic structure.

Most of the items are written in positive form, and those in negative were recoded. This instrument must be completed by interviewing main caregivers. Response options for every item are always, sometimes, never, or not known/no opportunity. Therapists or Most of the items are written in positive form, and those in negative were recoded. This instrument must be completed by interviewing main caregivers. Response options for every item are always, sometimes, never, or not known/no opportunity. Therapists or educators must obtain evidence that caregivers' answers are as close to reality as possible. Most of the items are written in positive form, and those in negative were recoded. This instrument must be completed by interviewing main caregivers. Response options for every item are always, sometimes, never, or not known/no opportunity. Therapists or educators must obtain evidence that caregivers' answers are as close to reality as possible.

educators must obtain evidence that caregivers' answers are as close to reality as possible.

### *2.4. Ethical Approval*

*2.4. Ethical Approval* The protocol followed in this study adhered to the updates of the Declaration of Helsinki [54], and it was approved by the Committee on Biomedical Ethics of the University *2.4. Ethical Approval* The protocol followed in this study adhered to the updates of the Declaration of Helsinki [54], and it was approved by the Committee on Biomedical Ethics of the University The protocol followed in this study adhered to the updates of the Declaration of Helsinki [54], and it was approved by the Committee on Biomedical Ethics of the University of Extremadura (198/2019).

#### of Extremadura (198/2019). of Extremadura (198/2019). *2.5. Statistics*

cess.

*2.5. Statistics* Microsoft OfficeTM Excel v.16 (Redmond, WA, USA: Microsoft Corporation), FAC-TOR v.10.10.02 (Tarragona, Spain, ESP: Rovira i Virgili University) and IBMTM SPSS v.25 (IBM Corporation, Armonk, NY, USA) were used for data analysis. A semiconfirmatory factor analysis (SCFA) was carried out, that is considered appropriate to prevent errors included in the "Little Jiffy" approach [55,56]. FACTOR performs at the same time an exploratory analysis offering goodness-of-fit indicators, so an additional confirmatory factor *2.5. Statistics* Microsoft OfficeTM Excel v.16 (Redmond, WA, USA: Microsoft Corporation), FAC-TOR v.10.10.02 (Tarragona, Spain, ESP: Rovira i Virgili University) and IBMTM SPSS v.25 (IBM Corporation, Armonk, NY, USA) were used for data analysis. A semiconfirmatory factor analysis (SCFA) was carried out, that is considered appropriate to prevent errors included in the "Little Jiffy" approach [55,56]. FACTOR performs at the same time an ex-Microsoft OfficeTM Excel v.16 (Redmond, WA, USA: Microsoft Corporation), FACTOR v.10.10.02 (Tarragona, Spain, ESP: Rovira i Virgili University) and IBMTM SPSS v.25 (IBM Corporation, Armonk, NY, USA) were used for data analysis. A semiconfirmatory factor analysis (SCFA) was carried out, that is considered appropriate to prevent errors included in the "Little Jiffy" approach [55,56]. FACTOR performs at the same time an exploratory analysis offering goodness-of-fit indicators, so an additional confirmatory factor analysis is not necessary [57–60].

analysis is not necessary [57–60]. Considering the ordinal nature of the data, polychoric correlations using the robust unweighted least squares method with oblique rotation were employed. The Kaiser-Meyer-Olkin (KMO) and Bartlett's sphericity tests were used as indices of sampling adequacy [61,62]. Due to the comprehensive nature of the tool, which is intended to be used as a developmental scale to monitor BADLs acquisition, and in the absence of cross-loadploratory analysis offering goodness-of-fit indicators, so an additional confirmatory factor analysis is not necessary [57–60]. Considering the ordinal nature of the data, polychoric correlations using the robust unweighted least squares method with oblique rotation were employed. The Kaiser-Meyer-Olkin (KMO) and Bartlett's sphericity tests were used as indices of sampling ade-Considering the ordinal nature of the data, polychoric correlations using the robust unweighted least squares method with oblique rotation were employed. The Kaiser-Meyer-Olkin (KMO) and Bartlett's sphericity tests were used as indices of sampling adequacy [61,62]. Due to the comprehensive nature of the tool, which is intended to be used as a developmental scale to monitor BADLs acquisition, and in the absence of cross-loadings, some items with loadings above 0.30 have been included [63].

ings, some items with loadings above 0.30 have been included [63]. To assess the goodness-of-fit, we used the chi-squared probability setting as appropriate non-significant values (*p* > 0.05); the comparative fit index (CFI) and the nonnormed fit index (NNFI); the root mean square error of approximation (RMSEA); and the quacy [61,62]. Due to the comprehensive nature of the tool, which is intended to be used as a developmental scale to monitor BADLs acquisition, and in the absence of cross-loadings, some items with loadings above 0.30 have been included [63]. To assess the goodness-of-fit, we used the chi-squared probability setting as appro-To assess the goodness-of-fit, we used the chi-squared probability setting as appropriate non-significant values (*p* > 0.05); the comparative fit index (CFI) and the non-normed fit index (NNFI); the root mean square error of approximation (RMSEA); and the root mean square of residuals (RMSR) [62,64].

priate non-significant values (*p* > 0.05); the comparative fit index (CFI) and the nonnormed fit index (NNFI); the root mean square error of approximation (RMSEA); and the

root mean square of residuals (RMSR) [62,64].

Ordinal alpha was used to find out the internal consistency of the tool. It represents an alternative to Cronbach's Alpha for ordinal items, being >0.70 values considered as acceptable and >0.80 preferred [65,66]. Ordinal alpha was used to find out the internal consistency of the tool. It represents an alternative to Cronbach's Alpha for ordinal items, being >0.70 values considered as acceptable and >0.80 preferred [65,66].

As external validity criteria, descriptive and contrast results are provided according to sociodemographic characteristics of the sample. Additionally, preliminary data on the classification ability of the questionnaire are presented through the analysis of the receiver operating characteristic (ROC curves), comparing with a sample of 11 subjects with ASD in addition to the sample of 303 typically developing participants. As external validity criteria, descriptive and contrast results are provided according to sociodemographic characteristics of the sample. Additionally, preliminary data on the classification ability of the questionnaire are presented through the analysis of the receiver operating characteristic (ROC curves), comparing with a sample of 11 subjects with ASD in addition to the sample of 303 typically developing participants.

#### **3. Results 3. Results**

#### *3.1. Item Analysis and Internal Structure of the Questionnaire 3.1. Item Analysis and Internal Structure of the Questionnaire*

*Children* **2021**, *8*, x FOR PEER REVIEW 6 of 18

The BADL-P study version was initially composed of 113 items. After performing the analysis, 29 items were deleted, so the final version was finally formed by 84 items distributed in four scales with nine factors (Figure 3). The instrument is created in Spanish (Supplementary Material), but items are provided in English to facilitate the reading of the paper. The BADL-P study version was initially composed of 113 items. After performing the analysis, 29 items were deleted, so the final version was finally formed by 84 items distributed in four scales with nine factors (Figure 3). The instrument is created in Spanish (Supplementary Material), but items are provided in English to facilitate the reading of the paper.

**Figure 3.** The Basic Activities of Daily Living Assessment in Preschoolers (BADL-P) complete structure. **Figure 3.** The Basic Activities of Daily Living Assessment in Preschoolers (BADL-P) complete structure.

The factor structure of the resulting dimensions and the factor loading of each item are presented below. The factor structure of the resulting dimensions and the factor loading of each item are presented below.

#### 3.1.1. Eating Scale 3.1.1. Eating Scale

A KMO value of 0.68, and Bartlett's test, *p* < 0.001 were both good enough to carry out the SCFA. However, 12 items did not reach <0.30 so 16 items formed the final version of the scale. A KMO value of 0.68, and Bartlett's test, *p* < 0.001 were both good enough to carry out the SCFA. However, 12 items did not reach <0.30 so 16 items formed the final version of the scale.

Eating refers to all the tasks or activities that help in manipulating, keeping food or fluids in the mouth and swallowing [2]. We found an interpretable solution with three factors which explores: (1) items related to sensory integration, (2) items associated with social, educational, and cultural behaviours that must be learnt to be considered nicely behaved during mealtime, and (3) items about hand skills with food, fluids, cutlery, or containers to perform self-feeding (Table 3). Eating refers to all the tasks or activities that help in manipulating, keeping food or fluids in the mouth and swallowing [2]. We found an interpretable solution with three factors which explores: (1) items related to sensory integration, (2) items associated with social, educational, and cultural behaviours that must be learnt to be considered nicely behaved during mealtime, and (3) items about hand skills with food, fluids, cutlery, or containers to perform self-feeding (Table 3).


**Table 3.** Factorial solution of the Eating scale.

### 3.1.2. Personal Hygiene Scale

A KMO value of 0.903 and a *p* < 0.001 for Bartlett's test were found. Initially, 36 items formed the scale, but six items were deleted. Thus, 30 items were maintained.

Personal hygiene refers to obtain and use toileting supplies to get or keep clean, including toileting needs, brushing, washing up, bathing and grooming [2]. We got an interpretable solution with three factors: (1) all the items related to brushing teeth, (2) the ones related to toileting needs, and (3) the rest of personal hygiene and grooming activities (Table 4).

### 3.1.3. Dressing Scale

A KMO value of 0.952 and a *p* < 0.001 for Bartlett's test were obtained. The scale had 30 items, but nine items were deleted, so 21 items form this scale.

Dressing refers to being able to select clothes, shoes, and accessories, putting them on and taking them off, and getting dressed and undressed in the right way [2]. This scale is formed only by one factor, as presented in Table 5.

### 3.1.4. Daily Functioning Scale

A KMO value of 0.737 and a *p* < 0.001 for Bartlett's test were found. Only 2 items were deleted, so the final version got 17 items on this scale.

This scale joins cognitive aspects that influence BADLs performance, and it is composed of two factors (Table 6): higher-order EF (eight items) and core EF (nine items).

### **Table 4.** Factorial solution of the Personal hygiene scale.


**Table 5.** Factorial solution of the Dressing scale.



### **Table 6.** Factorial solution of the Daily functioning scale.

### *3.2. Correlations between Factors*

Table 7 provides correlations between the different factors of every scale. All BADLs factors are related to each other. Likewise, EF is related to all BADLs except oral sensitivity. Thus, oral sensitivity seems to function independently, and it is only weakly and negatively related to core EF.

### **Table 7.** Correlations between the BADL-P factors.


\* Significant correlation for *p* < 0.05 \*\* Significant correlation for *p* < 0.01.

*3.3. Goodness-of-Fit Indices*

Table 8 shows that all the indices, calculated with FACTOR software, are acceptable.



CFI = Comparative fit index; NNFI = non-normed fit index; RMSEA = Root mean square error of approximation; RMSR = Root mean square of residuals.

### *3.4. Reliability*

Ordinal alpha (Table 9) was used to find out the internal consistency of the BADL-P. Results are acceptable (>0.70) or preferred (>0.80).



### *3.5. Results According to Sociometric Variables and Questionnaire Structure*

Table 10 shows descriptive and contrast statistics referring to the participants' scores considering sex (boys and girls). Significant differences of moderate magnitude according to sex in the toilet management dimension (*p* < 0.04; d > 0.56), with differences in favour of the girls' group, were observed. Moreover, significant differences of large magnitude are observed in the core EF (*p* < 0.001; d > 1.22) in favour of girls.

**Table 10.** Descriptive and contrasting statistics by sex group factors.


*t*: Two-sample *t*-tests; *p*: statistical signification (\* 0.05; \*\* 0.01); *d*: Cohen'd (Small = 0.2; Medium = 0.5; Large = 0.8).

Table 11 describe descriptive and contrast statistics after grouping participants according to the age of the two stages within early childhood education (3–4 and 5–6 years). Participants' scores indicate significant and high magnitude differences in almost all the dimensions, except in oral sensitivity (*p* < 0.22; *d* > 0.19) and core EF (*p* < 0.001; *d* > 1.22). These findings indicate a strong effect of age on the acquisition of BADLs (Table 11).

**Good Manners** **Manual** 

**tivity**


**Table 11.** Descriptive and contrasting statistics by age group factors. 3–4 5.4 ± 1.2 17.3 ± 2.2 9.0 ± 2.9 12.6 ± 2.8 23.9 ± 3.1 32.9 ± 6.4 46.9 ± 7.3 18.7 ± 2.2 12.7 ± 2.9

**Eating Scale Personal Hygiene Scale Dressing** 

**Dexterity Brushing Toilet Man-**

*Children* **2021**, *8*, x FOR PEER REVIEW 11 of 18

**Table 11.** Descriptive and contrasting statistics by age group factors.

**agement**

Table 11 describe descriptive and contrast statistics after grouping participants ac-

**Scale Daily Functioning Scale**

**Core EF Factor**

**EF Factor**

**Hygiene Dressing Higher-Order** 

cording to the age of the two stages within early childhood education (3–4 and 5–6 years). Participants' scores indicate significant and high magnitude differences in almost all the dimensions, except in oral sensitivity (*p* < 0.22; *d* > 0.19) and core EF (*p* < 0.001; *d* > 1.22).

These findings indicate a strong effect of age on the acquisition of BADLs (Table 11).

**General** 

*t*: Two-sample *t*-tests; *p*: statistical signification (\* 0.05; \*\* 0.01); *d*: Cohen'd (Small = 0.2; Medium = 0.5; Large = 0.8).

#### *3.6. The Basic Activities of Daily Living Assessment in Preschoolers Discrimination Ability between Typically Developing Participants and a Sample of ASD Participants* veloping children, our goal is that it can be used with children with neurodevelopmental

Although this manuscript presents the preschool version of our tool in typically developing children, our goal is that it can be used with children with neurodevelopmental disorders. Thus, a preliminary comparison of 11 typical children compared to 11 children with ASD (not included in our study sample) is presented to test the ability of the BADL-P to discriminate between performance on ADLs between typical development and ASD. disorders. Thus, a preliminary comparison of 11 typical children compared to 11 children with ASD (not included in our study sample) is presented to test the ability of the BADL-P to discriminate between performance on ADLs between typical development and ASD. As illustrated in Figure 4 and Table 12, the area under the curve (AUC) shows that

Although this manuscript presents the preschool version of our tool in typically de-

As illustrated in Figure 4 and Table 12, the area under the curve (AUC) shows that the tool can classify beyond chance between typically developing participants and participants with ASD (*p* < 0.00). The ability to classify between the two groups of greater magnitude is related to the personal hygiene scale and the total score of the questionnaire. the tool can classify beyond chance between typically developing participants and participants with ASD (*p* < 0.00). The ability to classify between the two groups of greater magnitude is related to the personal hygiene scale and the total score of the questionnaire.

**Figure 4.** Graphical representation from the receiver operating characteristic (ROC curves). **Figure 4.** Graphical representation from the receiver operating characteristic (ROC curves).


**Table 12.** Statics from the receiver operating characteristic (ROC curves).

AUC = Area under the curve; CI = Confidence interval; *p* = significance level; *d* = effect size following Cohen criteria [67].

### **4. Discussion**

### *4.1. The Basic Activities of Daily Living Assessment in Preschoolers Theoretical Model*

This study presents the BADL-P theoretical model, a novel tool for Spanish children between 3–6 years, with good psychometric properties according to preliminary data provided, practical and useful for both, early school educators and early care services educators and therapists. It was initially composed of 113 items, and after the study, they were reduced to 84. The model is divided into four scales: eating, personal hygiene, dressing, and daily functioning scales. The last scale offers a screening of cognitive factors which may influence during ADLs performance. In the eating scale, we got a structure with three factors and 16 items: oral sensitivity (three items), good manners (seven items) and manual dexterity (six items). In the personal hygiene scale, three factors and 30 items: brushing teeth (six items), toileting management (nine items) and hygiene and grooming (15 items). The dressing scale is composed of only one factor with 21 items. Finally, the daily functioning scale is composed of two factors and 17 items: higher-order EF (eight items) and core EF (nine items) during ADLs performance.

In our previous study [68], we presented the scholar version for children between six and 12 years (ADL-E). It was formed by a total of 84 quantitative items and six additional qualitative items only for girls about menstruation management. All items were distributed in the same four scales, but with different factors that outlined the progressive specialization in the BADLs from birth through lifetime. Thus, comparing the preschool version (BADL-P) with the school version (ADL-E), we observe how the dimensions are gradually expanded, as the skills are subdivided as the children grow up, which presents an indicator of validity [17,69].

As observed in Table 7, which shows correlations between factors, core EF, is closely related to BADLs performance. Thus, it is important to determine how problems in both of them could be affecting BADLs performance [30,70,71]. Therefore, our tool might help clinicians to determine whether a more in-depth assessment in one or another direction is needed.

### *4.2. The Basic Activities of Daily Living Assessment in Preschoolers and Other Tools*

As exposed, ADLs performance is influenced by both internal and external factors. Occupational development is the result of the dynamic interaction of person, activity and environment [6]. In Spain, some instruments for measuring BADLs are available, but as previously exposed, they present some limitations. On the one hand, sometimes occupational assessment is inferred from instruments that assess adaptive behaviour, described as suitable behaviours for independent living. However, this concept includes some or other areas or activities depending on the classification consulted. For example, Kamphaus (1987) talks about physical/motor, self-help/independence, interpersonal/social, cognitive/communication and responsibility. Meanwhile, Widaman et al. (1993) mention cognitive competence, social competence, social maladaptation, and personal maladaptation [53]. On the other hand, ADLs conceptualization has reached a huge agreement [2,72,73]. The BADL-P structure is focused on ADLs, which is concrete and unambiguous. Some of the most widely used instruments based on adaptive behaviour concept, mix BADLs

performance with other social aspects, not covering the BADLs full range of activities, or being their items divided into different sections or categories not according to their nature. For example, the Battelle Developmental Inventory [51], or the Inventory for Client and Agency Planning [49] include few items on BADLs performance. In the Adaptive Behavior Assessment System II [74], the items are divided into different sections which make them difficult to understand. It is also important to note that some tools, such as the Carolina Curriculum [75] or the Merrill–Palmer Revised Scales [52], do not cover the full preschooler stage. Moreover, many of these instruments are available in Spanish, but we have not found information about their validation; e.g., the Pediatric Evaluation of Disability Inventory [76] or the Vineland Adaptive Behavior Scales II [44], with the risk of not being culturally adapted.

### *4.3. The Basic Activities of Daily Living Assessment in Preschoolers Psychometric Properties*

We have created our instrument after performing an exhaustive review, using an experts' group, carrying out a pilot study with families, conducting the study itself and carrying out the corresponding statistical analysis. In relation to statistics, we have carried out the factorial analysis with FACTOR software, which can perform SCFA what means that while performing an exploratory factor analysis shows goodness-of-fit indices to prove if the factorial solution offers a suitable adjustment. The SCFA has been used to validate instruments in natural [77,78], social [79–85], and health sciences [30,86,87]. Thus, it is a widely contrasted procedure.

As reflected in results section, the goodness-of-fit indices related to the construct validity of the test structure are highly adequate. The descriptive and contrast statistics indicate significant differences related to the two age groups analyzed (2–4 and 5–6 years). It demonstrates that ADLs performance is acquired throughout development and our instrument seems sensitive for analyzing this progression. Furthermore, construct validity is reinforced by additional data on the instrument's ability to classify between typical participants and a small group of ASD individuals. ROC curves show the instrument's ability to discriminate adequately on some of the dimensions (Hygiene = 0.95 and Total = 0.93). As it can be noticed, the scores that most discriminate between typical development and ASD preschoolers are the personal hygiene scale and the total score. Concerning hygiene activities, they have a greater contextual load (noises, smells, visual stimulation by mirrors and reflections) and are longer and more precise than feeding and dressing. Therefore, they have a greater demand of core EF. The reliability index provided by the ordinal alpha [88–91] shows an acceptable level of internal consistency (α > 0.70). In summary, this preliminary study offers promising indicators of reliability and construct validity.

### *4.4. Limitations and Future Lines*

This research has some limitations. We are aware that we need to check the concurrent validity using well-established tools and predictive validity, conducting longitudinal studies to check the capacity of the instrument to detect occupational performance issues development of the children. However, we have not found validated tools that support BADLs construct in Spanish preschoolers. Another limitation is the small sample size of ASD children. This is the reason why we are talking about preliminary results, and we are working to get a larger sample size to increase the power of results.

Several future lines will be developed. In this manuscript, we have presented the BADL-P for preschoolers. In a previous study, the scholar version was tested [68]. We are also interested in exploring the skills required to achieve BADLs during the 0–3 years stage. In this line, IADLs (caring for others, communication management, community mobility, financial and home management, spiritual activities, safety and emergency maintenance, and health management [2]), which are performed at home and in the community, gain importance during adolescence. Thus, it could be interesting to explore an instrument in this scope.

Furthermore, our final goal is testing our tools in children and adolescents with neurodevelopmental disorders diagnosis [11], which includes intellectual disability, ASD, attention deficit and hyperactivity, specific learning, motor, and communication disorders. These children usually have poorer ADLs performance compared with typical development children, and some studies draw different occupational profiles in these individuals [12,13,92,93]. At the stage studied, the development of ADLs is closely linked to maturational processes, in addition to other contextual processes, such as the patterns of parenting in each cultural setting. The translation of the scale into other languages and its adaptation to other cultural contexts will bring greater clarity to the influence of both aspects on the development of ADLs. Although this paper presents preliminary discrimination data with a small sample of ASD subjects, we intend to check the specificity–sensitivity of our tools in clinical samples of children with neurodevelopmental disorders.

Finally, we want to highlight the importance of carrying out practices and policies to support children with disabilities to participate in society, even those with "invisible disabilities" like children with neurodevelopmental disorders. Thus, this project is aligned with objectives 3 ("Good-health and well-being") and 4 ("Quality education") of the 2030 Agenda for Sustainable Development [94].

### **5. Conclusions**

Based on our preliminary results, we conclude that BADL-P is a practical and easy to use tool with good construct validity and reliability properties for assessing BADLs occupational performance in Spanish preschoolers between three and six years.

Although this tool was developed to test BADLs occupational performance in typical development preschoolers aged from three to six years, preliminary results suggest that this tool could discriminate between typically developed children and their peers with neurodevelopmental disorders. However, future studies with larger sample sizes are needed to increase the power of results.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/10 .3390/children8060496/s1. Supplementary materials: The Basic Activities of Daily Living Assessment in Preschoolers (Spanish version).

**Author Contributions:** Conceptualization, S.B.-F., J.C.-V., M.G. and D.R.-A.; methodology, A.G.- G.; software, A.G.-G.; formal analysis, A.G.-G. and D.R.-A.; writing—original draft preparation, S.B.-F. and J.C.-V.; writing—review and editing, S.B.-F., M.G. and D.R.-A.; supervision, S.B.-F. and M.G.; funding acquisition, S.B.-F. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was partially funded by the Occupational Therapy Association of Extremadura (Colegio Profesional de Terapeutas Ocupacionales de Extremadura), grant number Ayudas a proyectos de investigación en Terapia Ocupacional/convocatoria 2019.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Committee on Biomedical Ethics of the University of Extremadura (198/2019).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The datasets used during the current study are available from the corresponding author on reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


## *Article* **A Complementary Sensory Tool for Children with Autism Spectrum Disorders**

**Sabina Barrios-Fernández 1,\* , Margarita Gozalo <sup>2</sup> , Beatriz Díaz-González <sup>3</sup> and Andrés García-Gómez <sup>4</sup>**


Received: 30 September 2020; Accepted: 18 November 2020; Published: 20 November 2020

**Abstract:** Background: Sensory integration (SI) issues are widely described in people with autism spectrum disorder (ASD), impacting in their daily life and occupations. To improve their quality of life and occupational performance, we need to improve clinical and educational evaluation and intervention processes. We aim to develop a tool for measuring SI issues for Spanish children and adolescents with ASD diagnosis, to be used as a complementary tool to complete the Rivière's Autism Spectrum Inventory, a widely used instrument in Spanish speaking places to describe the severity of ASD symptoms, recently updated with a new sensory scale with three dimensions. Methods: 458 Spanish participants complemented the new questionnaire, initially formed by 73 items with a 1–5 Likert scale. Results: The instrument finally was composed of 41 items grouped in three factors: modulation disorders (13 items), discrimination disorders (13 items), and sensory-based motor disorders (15 items). The goodness-of-fit indices from factor analyses, reliability, and the analysis of the questionnaire's classification capability offered good values. Conclusions: The new questionnaire shows good psychometric properties and seems to be a good complementary tool to complete new the sensory scale in the Rivière's Autism Spectrum Inventory.

**Keywords:** sensory processing; emotional regulation; assessment; autism spectrum disorders

### **1. Introduction**

### *1.1. Sensory Integration Process*

Sensory integration (SI) is "the neurological process that organizes sensations from one's own body (internal) and the environment (external) and makes it possible to use the body effectively within the environment" [1] (p. 11). An adequate organization of sensory information is necessary for producing adaptive responses in daily life, which includes different end products: motor, cognitive, behavioral, emotional, or learning outcomes [2]. SI is considered a prerequisite so that more complex functions, as perceptual-motor and cognitive ones, can be appropriately developed [3].

The SI process runs through a series of stages. Firstly, the sensory organs capture fragments of sensory information, which can have either an internal or an external origin. Later, that information is integrated in the central nervous system (CNS) to become a meaningful whole [4]. The SI process takes place in different brain structures in a coordinated way, classifying, and organizing the sensory flow through a series of stages. Firstly, in registration, the CNS detects the sensory sensations from our

sensory receptors and we become aware of those sensations [5]. Next, in modulation, the CNS regulates and processes the sensory stimuli [6]. Then, during discrimination the CNS distinguishes between different sensory stimuli, perceiving their specific qualities and becoming meaningful [6–8]. Finally, we elicit a response, intended to adaptive, which can include attention, organization, self-esteem, self-confidence, movement, reasoning, and learning outcomes [1,7,9]. Within that end products, and in the group of adaptive motor-based responses, we must refer to praxis. Praxis is the ability to conceptualize, plan, and execute unusual motor actions. Thus, it allows us to organize and manage a purposeful interaction with the physical world, thus involving both motor and cognitive skills [8,10].

Although traditionally we have focused in five senses (vision, hearing, smell, taste, and touch), there are three more sensory systems essential to be successful in daily life: proprioception, vestibular system, and interoception. Proprioceptive sense reports on sensations from muscles, ligaments, and joints, providing information about the compression and stretching of muscles and joints. Proprioception and touch together form the somatosensory pathway, considered essential for praxis and movement [11,12]. The vestibular system provides information on movement, gravity and balance, so it is crucial for the building of spatial and temporal relationships [13]. It also provides information about the speed and the direction of the head movement and our position with relation to gravity [9]. Interoception sense processes sensory stimuli within the body, including body sensations (hunger, thirst, body temperature, heart, breathing rate, etc.) and emotional states (happiness, sadness, shame, anger), being intimately related to self-regulation and well-being [14,15].

### *1.2. Sensory Processing Disorders*

When sensations flow in an organized and integrated way, our brain can use those sensations to form perceptions, behaviors, and learning; when the flow of sensations is disorganized, perception, behavior, and learning are like a traffic jam at a rush hour [16]. Therefore, when SI is not working properly, motor, cognitive, emotional, behavioral, and adaptive issues produce a decrease in daily living functioning and learning [17–20]. This dysfunction can be mild, medium, or severe [21]. Sensory processing disorder (SPD) is a neurological disorder in which the ability to process and interpret sensory stimuli results in abnormal responses, causing a decrease in the quality of life and occupational performance [22,23]. Several models have been developed to understand the SPD [1,6,24], being Miller's model one of the most accepted. According to it, SPD can be classified into three categories with their corresponding subtypes: sensory modulation disorders, sensory discrimination disorders, and sensory-based motor disorders. Sensory modulation disorders happen when the CNS has problems in regulating the sensory information (degree, nature, or intensity) resulting in the following subtypes: sensory over-responsivity (exaggerated response), sensory under-responsivity (lack or insufficient response), or sensory craving (desperate seeking for sensory information). Sensory discrimination disorders happen when there is difficulty interpreting the qualities of the sensory stimuli. As a result, the responses are often slow, and sometimes, wrong. Finally, sensory-based motor disorders cause difficulty with motor planning and movement, resulting in postural disorder or dyspraxia subtypes [6,25].

### *1.3. Autism Spectrum Disorders and SPD Relationships*

Taking the latest version of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) as a reference [26], ASD are included in the neurodevelopmental disorders group, and they are defined by the presence of (a) persistent deficits in social communication and interaction, and (b) restricted, repetitive patterns of behavior, interests, or activities. Within the (b) criterion and, for the first time in the DSM, sensory abnormalities were included as "Hyper- or hyporeactivity to sensory input or unusual interest in sensory aspects of the environment (e.g., apparent indifference to pain/temperature, adverse response to specific sounds or textures, excessive smelling or touching of objects, visual fascination with lights or movement)" (p. 50).

With regards to etiology, and although it is widely recognized that genetic and environmental factors and their interactions contribute to the phenotypes of ASD, the precise causal mechanisms keep still unclear [27]. On a neuroanatomical basis, it is hypothesized that ASD symptoms should be a consequence of brain disconnection since hypomyelination of the brain nerves occurs simultaneously with the main behavioral symptoms [28]. Other studies complement this hypoconnectivity hypothesis by suggesting that in addition to hypoconnectivity in some regions of the cerebral cortex and at an interhemispheric level, a compensatory hyperconnectivity between the thalamus and the cerebral cortex, explaining sensory, and social symptoms [29]. Under this assumption, sensory issues in ASD have as origin atypical connectivity of neuronal structures. Nevertheless, it seems that topography of hypoconnectivity in ASD is unique and different from other conditions, such as SPD. In ASD, areas related to socio-emotional processing are highly affected; whereas, in SPD, there is lower connectivity in the brain's perception and integration pathways, which serve as connections for the auditory, visual, and somatosensory systems involved in SI [30].

SI issues are commonly reported in ASD, compared to their peers [31]. Various studies have tried to explain the most frequent sensory profiles or those issues that cause the biggest issues in children with ASD, as well as the proposals of intervention to improve their occupational performance [19,31–37]. With regards to ASD specific sensory profiles, hyporeactivity/under-responsivity is one of the most consistent issues found [24], although hyperreactivity/over-responsivity and sensory seeking have been also reported [38]. Several studies have found relationships between the core symptoms of ASD and sensory impairments, such as repetitive behaviors [34,39], with social communication and interaction [31,40,41], but also, with movement issues, including coordination, planning, and timing [42,43], impacting in their daily life [44]. With regards to the interventions, some of the studies focused on Ayres's Sensory Integration Therapy [36,45], and others in using specific sensory techniques and environmental modifications, thus the promotion of ecological approaches to improve occupational performance [4,37].

### *1.4. Autism Spectrum Disorders and SPD Assessment*

There are different tools to measure SI functioning, including questionnaires, observational tools, and comprehensive tests administered to the children or adolescents. Some reviews have been performed to resume information about SI tools, noticing that there are a large number of proposals [46,47]. Other reviews have checked for the most used SI tools in ASD, providing information about their characteristics and limitations [48–50]. Some of the most representative instruments are the Sensory Profile (SP) [51] and its second version (SP2) [52], a group of standardized questionnaires for assessing sensory processing including the infant, toddler, child, short, and school companion forms, from birth to 14.11 years. The Sensory Processing Measure (SPM), formed by a set of questionnaires to assess SI in home and the school, in children between 5–12 years. It also includes self-evaluation forms to be completed by the children and adolescents. There is a preschool version from 2–5 years [53]. The Sensory Integration and Praxis Test (SIPT) is a comprehensive test formed by 17 subtests to assess visual, tactile, kinesthetic, and motor tasks in children from 4–8.11 years [16]. There are several emerging SI assessment tools. The Sensory Processing 3-Dimensions (SP3D) is a tool composed of a series of task to elicit typical and atypical behavioral responses in children, covering sensory modulation, discrimination, and sensory-based motor disorders; and by a questionnaire with five subscales: sensory over-responsiveness, sensory under-responsiveness, sensory craving, postural disorder, dyspraxia, and sensory discrimination disorder [25,54]. The Evaluation in Ayres Sensory Integration (EASI) is a comprehensive assessment test for SI which includes measures related to sensory perception, sensory responsiveness, postural, ocular and bilateral integration, and praxis [24,55].

Tools for ASD assessment, including detection [56,57], diagnosis and measuring changes after interventions [58,59] are also available. In any case, the assessment of the severity of ASD should be complete and comprehensive and must include the measure of the SI and its impact in daily life. Within these tools, the Autism Spectrum Disorders Inventory developed by Rivière [60,61], is a widely used tool both in Spain and Latin America. It examines the severity of ASD by establishing four disease groups: relationship disorders, communication disorders, anticipation and flexibility, and symbolization, resulting in 12 dimensions, all of which can be scored from 0 to 8 points. The Rivière's Autism Spectrum Inventory was set up before the importance of the SI was spread so, recently, a new sensory scale has been incorporated [62] updating the tool to the current knowledge of ASD. Now it is formed by five disease groups and 15 dimensions (Figure 1). An advantage of the Rivière's Autism Spectrum Inventory is the fact that, as being designed by severity levels, it can help the clinicians' in their judgment to determine the levels of severity required in DSM-5 [26]. However, and although this instrument explains the four levels of affectation in each dimension, it does not define specific behaviors to observe, so using complementary tools to collect information is strongly recommended. *Children* **2020**, *7*, x FOR PEER REVIEW 4 of 14 Rivière's Autism Spectrum Inventory was set up before the importance of the SI was spread so, recently, a new sensory scale has been incorporated [62] updating the tool to the current knowledge of ASD. Now it is formed by five disease groups and 15 dimensions (Figure 1). An advantage of the Rivière's Autism Spectrum Inventory is the fact that, as being designed by severity levels, it can help the clinicians' in their judgment to determine the levels of severity required in DSM-5 [26]. However, and although this instrument explains the four levels of affectation in each dimension, it does not define specific behaviors to observe, so using complementary tools to collect information is strongly recommended.


**Figure 1.** Summary of Rivière's Autism Spectrum Inventory [60,61]. Disease groups from I–IV with their 12 dimensions correspond to Rivière's original version. The V scale with the dimensions 13–15 was added by García-Gómez [62]. Preferred scores for rating the Inventory are the even ones, while odd scores are used to describe intermediate stages. **Figure 1.** Summary of Rivière's Autism Spectrum Inventory [60,61]. Disease groups from I–IV with their 12 dimensions correspond to Rivière's original version. The V scale with the dimensions 13–15 was added by García-Gómez [62]. Preferred scores for rating the Inventory are the even ones, while odd scores are used to describe intermediate stages.

### *1.5. Aim*

*1.5. Aim*  We aim to create a questionnaire to be used as a support for scoring the new sensory scale in the Rivière's Autism Spectrum Inventory, a widely used tool to assess ASD severity in Spanish speaking places**.** We aim to create a questionnaire to be used as a support for scoring the new sensory scale in the Rivière's Autism Spectrum Inventory, a widely used tool to assess ASD severity in Spanish speaking places.

### **2. Methods**

### **2. Methods**  *2.1. Participants*

*2.1. Participants*  The sample was formed by 458 children and adolescents (308 males, 68.7%, and 144 females, 31.3%) from 4 to 19 years (*x* = 9.6, *dt* = 4.42). Of these, 259 were individuals with typical development (57.2%), 145 presented ASD clinical diagnosis (32%) and 54 had other diagnoses different than ASD The sample was formed by 458 children and adolescents (308 males, 68.7%, and 144 females, 31.3%) from 4 to 19 years (*x* = 9.6, *dt* = 4.42). Of these, 259 were individuals with typical development (57.2%), 145 presented ASD clinical diagnosis (32%) and 54 had other diagnoses different than ASD resulting in intellectual, sensory, and/or motor disabilities (11.95%).

### resulting in intellectual, sensory, and/or motor disabilities (11.95%). *2.2. Procedure*

*2.2. Procedure*  After conducting a literature review, a group of experts in the fields of Occupational Therapy and Psychology, with clinical experience, created a preliminary version of the tool composed of 73 items. Then a pilot study was carried out with 31 ASD families with diagnosed children. The 50 items with the best indicators were selected. Participants were recruited using the snowball technique in the case of typical development children, and through different associations, in the case of diagnosed After conducting a literature review, a group of experts in the fields of Occupational Therapy and Psychology, with clinical experience, created a preliminary version of the tool composed of 73 items. Then a pilot study was carried out with 31 ASD families with diagnosed children. The 50 items with the best indicators were selected. Participants were recruited using the snowball technique in the case of typical development children, and through different associations, in the case of diagnosed children. The data collection was carried out between May and August 2020. This protocol adheres to the

children. The data collection was carried out between May and August 2020. This protocol adheres

updates of the Declaration of Helsinki, and the study was approved by the Committee on Biomedical Ethics of the University of Extremadura (97/2020).

Our instrument, the Behavioral Observation on Sensory Stimuli Questionnaire for Parents (BOSS-P) was administered to the families. They were also asked for socio-demographic data, including age, sex, clinical diagnosis, intellectual capacity, language level, comorbidities, and the need for aids in their daily life. Once the questionnaire was administered to the sample, the items were analyzed by the group of experts, discarding those which did not fit on the theoretical model, being the final version composed of 41 items (Supplementary Table S1). The BOSS-P was administered together with the Sensory Profile 2 (SP2) Short Form [63,64] to 31 participants, to obtain validity indicators.

### *2.3. Instrument*

The BOSS-P, a new instrument to better characterize ASD children and adolescents to fulfil the three new sensory dimensions from Rivière's Autism Spectrum Inventory based on Miller's model, must be completed interviewing with main caregivers, which may answer the 41 items through a Likert scale with five response options, from 1 to 5 (higher scores mean greater SI dysfunction). It takes about 25–30 min to complete the interview.

### *2.4. Statistics*

To perform the validation process of the BOSS-P we have carried out: (1) an exploratory factor analysis (EFA), (2) confirmatory factor analysis (CFA), (3) reliability analysis, (4) the assessment of concurrent validity through the correlations with the SP2, and (5) provide descriptive statistics from the typical development and the ASD subsamples.

Because we are handling ordinal variables from a Likert-type scale with five response categories, the EFA was carried out with the FACTOR software [65–67] using polychoric correlations and robust methods [68]. Items with factorial weights below 0.30 were excluded. The CFA was carried out with the IBM SPSS AMOSTM 24 [69] using the Maximum Likelihood estimation procedure, suitable for Likert-type scales of five response categories. The CFA supports the factorial solution provided by the EFA and also offers the model of relations between the variables that best fits with the data [70–72].

The evaluation of the model fit was made taking into account the Chi-Square divided by degrees of freedom (CMIN/DF) and the p of Chi-square following Byrne's criteria [73]. The statistical *p* of Chi-square is dependent on the sample size, so it was convenient to use other goodness-of-fit indicators choosing the Tucker–Lewis index (TLI), the comparative fit index (CFI) following Hu and Bentler's criteria [74], the root-mean-square error of approximation (RMSEA), and the root-mean-square residuals (RMSR) [75,76].

Ordinal alpha coefficients were calculated [77,78] to assess reliability, considering values >0.70 acceptable and >0.90, excellent [79]. The analysis of the correlations between our tool and the SP2, the descriptive statistics of the subsamples and the relative operating characteristic (ROC) analysis were carried out to check the instrument's ability to classify between the two subsamples, using the IBM SPSSTM 24 [80] statistical package. Cohen's d statistic [81] was also calculated to check the magnitude of the effect size of the differences between the subsamples scores.

### **3. Results**

### *3.1. Exploratory Factor Analysis*

After administering the experimental version of the questionnaire to the sample, a solution of 41 items grouped into three correlated factors was obtained. Bartlett's (5025.4; *df* = 820; *p* = 0.000) and Kaiser–Meyer–Olkin test (0.912) statistics showed a very good sample suitability [82]. In Table 1, can be found both the rotated factorial matrix and factorial weights of each item. The three factors obtained represent (F1) modulation disorders with 13 items, (F2) discrimination disorders with 13 items, and (F3) sensory-based motor disorders with 15 items.


**Table 1.** Rotated factorial matrix and factorial weights of each item.

(F1) Modulation disorders; (F2) discrimination disorders; and (F3) sensory-based motor disorders. Items translated for readability; no cross-cultural adaptation performed.

With regards to the correlation between factors, moderate relationships were found between F1–F2 (0.38); F1–F3 (0.61), and F2–F3 (0.53) [81], which was to be expected since they are different stages within the same neurobiological process.

### *3.2. Confirmatory Factor Analysis*

The CFA confirms the exploratory factorial solution revealing three latent variables which group the 41 observable variables (items). Figure 2 shows the graphical representation of the analyzed model, being (F1) modulation disorders, (F2) discrimination disorders, and (F3) sensory-based motor disorders. The factorial weights of every item and the covariation relations between the latent variables are shown.

variables are shown.

*3.2. Confirmatory Factor Analysis* 

The CFA confirms the exploratory factorial solution revealing three latent variables which group the 41 observable variables (items). Figure 2 shows the graphical representation of the analyzed model, being (F1) modulation disorders, (F2) discrimination disorders, and (F3) sensory-based motor

**Figure 2.** The Behavioral Observation on Sensory Stimuli Questionnaire for Parents' (BOSS-P) graphical representation after confirmatory factor analysis (CFA). **Figure 2.** The Behavioral Observation on Sensory Stimuli Questionnaire for Parents' (BOSS-P) graphical representation after confirmatory factor analysis (CFA).

In Table 2, are represented the goodness-of-fit indices from the CFA, showing good values. In Table 2, are represented the goodness-of-fit indices from the CFA, showing good values.


**Table 2.** BOSS-P goodness-of-fit indices from the confirmatory factor analysis (CFA). **Table 2.** BOSS-P goodness-of-fit indices from the confirmatory factor analysis (CFA).

*p* (χ2): chi-squared probability; CFI: comparative fit index; NNFI: non-normed fit index, RMSEA: root mean square error of approximation; RMSR: root mean square of residuals. *p* (χ 2 ): chi-squared probability; CFI: comparative fit index; NNFI: non-normed fit index, RMSEA: root mean square error of approximation; RMSR: root mean square of residuals.

### *3.3. Reliability*

To analyze the concurrent validity, we compared the BOSS-P with the SP2, a tool for SI assessment validated for Spanish children and adolescents. Both questionnaires were administered to 31 participants with ASD to study their correlations. As shown in Table 3, the modulation disorders factor (F1) from the BOSS-P was the only with significant and moderate correlations with the factors analyzed in the SP2.


**Table 3.** Correlation matrix between the Behavioral Observation on Sensory Stimuli Questionnaire for Parents (BOSS-P) and the Short Sensory Profile 2 (SP2).

\*. Correlation is significant at the 0.05 level (2-tailed). \*\*. Correlation is significant at the 0.01 level (2-tailed) F1 = Modulation Disorders factor; F2 = Discrimination Disorders factor; F3 = Sensory-Based Motor Disorders factor; Sensory = Sensory Processing; Behavioral = Behavioral Responses associated with Sensory Processing.

### *3.4. Questionnaire's Capability to Classify between ASD and Typical Development*

Descriptive statistics of participants with ASD (*n* = 145) and with typical development (*n* = 259) subsamples are shown in Table 4. It can be checked that both, mean and standard deviation of every subsample offer different scores.

**Table 4.** Descriptive statistics of ASD and typical development samples.


x: mean; SD: standard deviation.

In Figure 3, graphical representation and statistics from ROC curves are provided. The area under the curve (AUC) shows differences with large effect magnitudes between the three factors, being the BOSS-P total score the most capable dimension to establish a correct classification of subjects according to their reference group. *Children* **2020**, *7*, x FOR PEER REVIEW 9 of 14

**Figure 3.** Graphical representation (**a**) and statistics (**b**) from the receiver operating characteristic (ROC) curves. **Figure 3.** Graphical representation (**a**) and statistics (**b**) from the receiver operating characteristic (ROC) curves.

(**a**) (**b**)

F1 = modulation disorders factor; F2 = discrimination disorders factor; F3 = sensory-based motor

We aimed to create a questionnaire to support the new SI scale [62] added to the Rivière's Autism Spectrum Inventory [60,61]. The Rivière's Inventory is a widely used instrument in Spain and Latin America, which allows us to establish the level of the ASD severity, in line with the levels proposed in the DSM-5 [26]. The Rivière's Inventory is useful both during the diagnosis and intervention

The BOSS-P is a screening instrument, administered through an interview with parents or carers, which is not intended to replace other comprehensive assessments, existing or emerging, with good psychometric properties on SI. However, our instrument has several advantages: (1) it is a quick test which is administered in 25–30 min; (2) that does not require specific training; (3) is open access; (4) is a complete tool, as it assesses items within the three areas described by Miller [6]: sensory modulation, sensory discrimination, and sensory-based motor disorders; (5) with good psychometric properties in terms of validity, reliability, and discrimination capacity; (6) created in Spain and therefore, adapted to the cultural characteristics of this country; (7) which fills a gap in terms of SI tools in Spanish-speaking population; and (8) which complements a psychological test widely used

We have also provided an attempt to the combined use of the BOSS-P and the Rivière's Inventory, by linking the Rivière's level of severity and the BOSS-P quartile scores. Our instrument

in the Spanish-speaking world, the Rivière's Inventory for people with ASD.

**F1 F2 F3** 

1 (8 points) >40 >36 >50 2 (6 points) 34–40 30–36 40–50 3 (4 points) 27–34 27–30 31.5–40 4 (2 points) <27 <27 <31.5

**Table 5.** Combination of the level of affectation in the Rivière's Inventory and the Behavioral

Observation on Sensory Stimuli Questionnaire for Parents' (BOSS-P) interquartile scores.

Rivière's inventory levels of severity

disorders factor.

*4.1. About the BOSS-P Questionnaire* 

**4. Discussion** 

processes.

Considering the Rivière's Autism Spectrum Inventory scoring system, an approximation to the level of SI severity using the level of affectation in the Rivière's inventory and the BOSS-P interquartile scores was obtained in the ASD sample (see Table 5).


**Table 5.** Combination of the level of affectation in the Rivière's Inventory and the Behavioral Observation on Sensory Stimuli Questionnaire for Parents' (BOSS-P) interquartile scores.

F1 = modulation disorders factor; F2 = discrimination disorders factor; F3 = sensory-based motor disorders factor.

### **4. Discussion**

### *4.1. About the BOSS-P Questionnaire*

We aimed to create a questionnaire to support the new SI scale [62] added to the Rivière's Autism Spectrum Inventory [60,61]. The Rivière's Inventory is a widely used instrument in Spain and Latin America, which allows us to establish the level of the ASD severity, in line with the levels proposed in the DSM-5 [26]. The Rivière's Inventory is useful both during the diagnosis and intervention processes.

The BOSS-P is a screening instrument, administered through an interview with parents or carers, which is not intended to replace other comprehensive assessments, existing or emerging, with good psychometric properties on SI. However, our instrument has several advantages: (1) it is a quick test which is administered in 25–30 min; (2) that does not require specific training; (3) is open access; (4) is a complete tool, as it assesses items within the three areas described by Miller [6]: sensory modulation, sensory discrimination, and sensory-based motor disorders; (5) with good psychometric properties in terms of validity, reliability, and discrimination capacity; (6) created in Spain and therefore, adapted to the cultural characteristics of this country; (7) which fills a gap in terms of SI tools in Spanish-speaking population; and (8) which complements a psychological test widely used in the Spanish-speaking world, the Rivière's Inventory for people with ASD.

We have also provided an attempt to the combined use of the BOSS-P and the Rivière's Inventory, by linking the Rivière's level of severity and the BOSS-P quartile scores. Our instrument showed good psychometric values, offering a factorial structure formed by the three groups proposed by Miller [6]. These data are interesting because let us verify that Miller's model is consistent in different cultures and because. The BOSS-P's ability to classify between participants with ASD and typical development children seems adequate (AUC = 0.938), corresponding to a large effect size between the scores of both subsamples (*d* = 2.176) [83].

### *4.2. The BOSS-P and Other Instruments*

Some reviews have found that psychometric properties of some of the SI tools are from poor to moderate, so the professionals must use the obtained data with caution [48,50], selecting appropriate SI assessments depending on the detected SI needs [47]. However, as aforementioned, there are few available instruments for Spanish children and adolescents. According to our best knowledge, neither the SIPT—considered as a Gold Standard for SI assessment [84]—nor the SPM are available for Spanish population, while the SP3D and the EASI are not yet published, being the SP2 the only tool of choice in Spain. The SP2 Spanish version covers a little shorter age range than the original, from 3 to 14.11 years. The BOSS-P covers from 4 to 19 years, a wider range including the full adolescent stage. The correlations between the BOSS-P and the SP2 only find relationships in modulation disorders,

which could lead us to consider the necessity of using different tools to obtain information about SI if using the SP2.

Concerning its psychometric properties, the BOSS-P items present excellent internal consistency (alpha > 0.87), similar or superior other questionnaires used in the international context [46]. The ability of the questionnaire to discriminate between sub-samples offers a large effect size (*d* = 2.176), which is slightly higher than the size effect of the difference reported in other instruments [85].

### *4.3. Limitations and Future Lines*

This research has some limitations. The information was completed through parents, and although instruments completed by families are considered to be valid [86], we must be careful because some parents should overestimate or underestimate the development of their children [87]. The sample was one of convenience. Another limitation was that we could not perform a test–retest. As future lines, we will try to improve the psychometric properties of the questionnaire, as well as to perform studies for its use in other Spanish-speaking countries different than Spain.

### **5. Conclusions**

The preliminary study of the psychometric properties of Behavioral Observation on Sensory Stimuli Questionnaire for Parents (BOSS-P) shows good values for its use in Spanish children and adolescents diagnosed with ASD between 4 and 19 years. This tool was designed to help clinicians and educational professionals to establish the level of severity in children and adolescents with ASD diagnosis through the new SI scale in the Rivière's Autism Spectrum Inventory.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/2227-9067/7/11/244/s1, Table S1: Behavioral Observation on Sensory Stimuli Questionnaire for Parents (BOSS-P) Versión original española: Cuestionario de Observación de la Conducta ante Estímulos Sensoriales para Padres de niños/as y adolescentes con Autismo/TEA (OCS-P).

**Author Contributions:** Conceptualization, S.B.-F., M.G., and B.D.-G.; Methodology, A.G.-G.; Software, A.G.-G.; Formal analysis, A.G.-G.; Writing—original draft preparation, S.B.-F., M.G.; Writing—review and editing, S.B.-F. and M.G.; Supervision, A.G.-G. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Acknowledgments:** We want to thank all the participants who kindly dedicated their time to help us in this research.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **Play in Children with Neurodevelopmental Disorders: Psychometric Properties of a Parent Report Measure 'My Child's Play'**

**Dulce Romero-Ayuso 1,\* , María Ruiz-Salcedo <sup>1</sup> , Sabina Barrios-Fernández <sup>2</sup> , José Matías Triviño-Juárez <sup>3</sup> , Donald Maciver <sup>4</sup> , Janet Richmond <sup>5</sup> and Miguel A. Muñoz <sup>6</sup>**


**Abstract:** Play is essential in childhood, allowing for a positive trend in development and learning. Health professionals need useful tools to assess it, especially in the case of children with neurodevelopmental disorders. The aim of this study was to validate and cross-culturally adapt the My Child's Play questionnaire and to find out if this instrument allows us to differentiate the play of children with neurodevelopmental disorders from the play of children with neurotypical development. A total of 594 parents completed the questionnaire. A confirmatory factor analysis was conducted, which showed a similar structure to the English version: (1) executive functions; (2) environmental context; (3) play characteristics; and (4) play preferences and interpersonal interactions. The reliability of the analysis was high, both for the whole questionnaire and for the factors it comprises. The results provide evidence of the potential usefulness of the My Child's Play questionnaire for determining play needs and difficulties of children; moreover, this tool can also be used to plan intervention programs according to the needs of each child and family.

**Keywords:** play; assessment; executive functions; neurodevelopmental disorders; autism spectrum disorder; specific language disorder

### **1. Introduction**

Play is considered a natural learning mechanism through which children explore and learn about themselves and the world around them [1]. Children spend considerable time playing, providing fun, learning, and activity [2]. Children's play has been studied by many disciplines because of its considerable influence on global development and well-being, being essential for achievement of motor, cognitive, emotional, and social milestones [3]. In fact, participation in play contributes to peer inclusion, improvement of self-concept and self-esteem, promotion of creativity and flexibility, emotional regulation, language development, and frustration tolerance during academic and daily life activities [1]. Play can be defined as a non-serious, spontaneous, or organized activity that provides enjoyment, entertainment, amusement, or diversion and its characteristics include intrinsic motivation, emphasis on process rather than product, pleasure, reward, and voluntary engagement [2,4].

**Citation:** Romero-Ayuso, D.; Ruiz-Salcedo, M.; Barrios-Fernández, S.; Triviño-Juárez, J.M.; Maciver, D.; Richmond, J.; Muñoz, M.A. Play in Children with Neurodevelopmental Disorders: Psychometric Properties of a Parent Report Measure 'My Child's Play'. *Children* **2021**, *8*, 25. https:// doi.org/10.3390/children8010025

Received: 5 December 2020 Accepted: 2 January 2021 Published: 6 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

In children with neurodevelopmental disorders (NDD), especially those with autism spectrum disorder (ASD), developmental delay (DD), and specific language disorder (SLD), differences are observed in the way they participate in play [5,6]. In ASD diagnosed children, deficits in joint attention, imagination, imitation, and communicative intention can affect their play development [7]. Usually, their play interests are oriented toward sensory and physical aspects [8], and together with their repetitive and stereotypical behaviors [9] produce issues while interacting with their peers and achieving a proper play engagement [10,11]. Children with DD spend more time in passive activities rather than playing and with adults more than with their peers [12]. Although children with ASD and SLD diagnosis face different challenges around play, there are few studies that address them, and the instruments to assess them are limited [1,13,14].

Educational and health professionals may use different tools for play assessment [6], such as the Test of Playfulness, Test of Environmental Supportiveness, Revised Know Preschool Play Scale, The McDonald Play Inventory, The Play Assessment for Group Settings, and My Children's Play. Test of Playfulness [15] is designed to measure play in children between 6 months and 18 years, observing their individual free play both indoors and outdoors and its extension. The Test of Environmental Supportiveness assesses how the environment influences play. The Revised Knox Preschool Play Scale [16] uses observation to assess participation in play, taking into consideration the use of space and materials, sensory and motor processing, social behavior, communication, and symbolic play. The McDonald Play Inventory [17] is a two-part self-report instrument that provides information about play activities and play style. For the pretend play evaluation, the Play Assessment for Group Settings [18] and the Child-Initiated Pretend Play Assessment [19,20] are available, having this second one an extension for measuring social aspects of play called the Indigenous Play Partner Scale, both using professional observation.

On the other hand, as parents are often the primary caregivers of their children, it is important to know the way in which they interact during play situations [21]. It is also important to consider parents' beliefs about their children's play, influencing how they organize their contexts, activities, and interactions [1,22,23]. Although parents are able to adapt their play to their child's play level, parents with ASD children can use fewer symbolic solicitations, not giving the opportunity to make their play more complex [24]. For all the above, parents must be considered as important agents for a proper play assessment and intervention.

The My Children's Play (MCP) questionnaire [25] is a measure that provides information about parent's perception about their child's play performance, where they indicate the response that best describes their child's play behavior on a 5-point Likert scale. This instrument has a total of 43 items divided in four factors: executive functions, interpersonal relationships, play choices, and preferences and opportunities in the environment, which makes it a tool with a broad view on a phenomenon as complex as children's play.

The availability of valid, reliable, and appropriate tools to assess play using parental perceptions and beliefs is important for a complete understanding of children's play. The first aim of this study was to conduct a cross-cultural adaptation of the My Child's Play questionnaire for the Spanish-speaking population. Secondly, we studied whether the My Child's Play questionnaire would allow us to know the characteristics of the play of children with NDD, such as ASD, DD, and SLD, and to differentiate them from children with neurotypical development.

### **2. Materials and Methods**

### *2.1. Participants*

The participants were recruited through different educational centers and associations of children with functional diversity: CEIP Parque de la Infantas de Granada, CEIP Maruja Mallo de Málaga, Asociación Serranía de Churriana, Federación Española de Autismo (FESPAU), Autismo España, and Asperger España. Two researchers with experience in child therapy contacted the different entities (M.R.-S. and D.R.-A.) A researcher with

clinical experience in children with ASD and SLD recruited the parents of children with NDD (S.B.-F.) Authorization was requested from those responsible for each entity. Once authorization was obtained, the interested parents were contacted, and the questionnaire was provided in digital format. The participation of all parents was voluntary. Before starting to fill in the questionnaire, the objective of the study was explained to them, and their written informed consent was requested. The researchers resolved the doubts that arose to the parents while completing the questionnaire. Similar to the original questionnaire in the English version, the responses were based on parents' perceptions of their children's play. The same procedure was followed in all centers.

The inclusion criteria in the neurotypical group were to be the main caregiver of a child aged between 3 and 9 years old without any neurological or psychiatric illness, without learning disorders, Spanish nationality, and consent to participate in the study. The criteria in neurodevelopmental disorder group in addition to age were to have a clinical diagnosis of NDD (ASD, DD, or SLD).

### *2.2. Instruments*

The My Child's Play (MCP) is the original English version of the questionnaire that has been translated and validated into Spanish in this study, and it is a tool that has strong psychometric qualities, with good validity and reliability of internal consistency (Cronbach's α = 0.86) [26]. The original tool consisted of 50 items, the last version reduced the number of items to 43, being divided into four dimensions: interpersonal relationships and social participation, executive functions, choices and preferences in play, and opportunities in the environment. Parents should indicate in each item the response that best reflects their child's behavior during play using a 5-point Likert scale ranging from 1 = never to 5 = always. It also includes the option of not applicable to assess an item as not relevant in that specific case. The total score obtained in the questionnaire is interpreted taking into account that higher scores reflect better performance.

The participants answered a series of questions about sociodemographic aspects that included age, gender, place of residence, educational level, and general questions about their children such as premature or full-term birth. Likewise, the parents were asked if the children had any type of learning difficulty and if they did other activities regularly in addition to school, such as sports, playing an instrument, or another hobby. Subsequently, all parents completed the translated version of the MCP questionnaire.

### *2.3. Design and Procedure*

For the development of this study, the methodology of the validation of health questionnaires was followed [27]. Before conducting the study, the authors of the original questionnaire were contacted to inform them about our study and request their authorization to carry out the translation, adaptation, and validation of the questionnaire with the Spanish population. This study was approved by the Ethics and Research Committee of the University of Granada (code 1426/CEIH/2020).

The first phase included the direct and inverse translation of the Spanish version and the cultural adaptation that was generated. In the second phase, a pilot study of the Spanish version of the questionnaire was carried out with participants recruited by the research team. The selection of the sample was by non-probabilistic method, and data collection was carried out over six months. Finally, in the third phase, the analysis of the psychometric properties was carried out to check the validity and reliability of the questionnaire in its Spanish version and the confirmatory factor analysis of the structure of the original questionnaire. The original English version of the MCP questionnaire was translated into Spanish by two people independently, a bilingual translator and one of the researchers in the study who has an adequate knowledge of the original language. Both translations were compared simultaneously by the research team to identify and discuss the discrepancies between the two versions until a consensus was reached, generating a first version of the questionnaire in Spanish. The first Spanish version of the questionnaire generated in the previous step was translated back into English by a translator. An expert panel made up of members of the research team was set up to review and compare the translation of each item with the original version and the Spanish version. This allowed us to check whether the translation generated relevant conceptual differences between the version translated into Spanish and the original. A total of five items were modified to improve their understanding, corresponding to numbers 5, 14, 15, 31, and 50 and considering the rest of the questions as correct (Table 1). These changes gave rise to the second version in Spanish of the questionnaire with which the pilot study was carried out.



In a second phase, a pilot test was performed. The Spanish version of MCP was used to carry out a pilot test with 30 parents of children with neurotypical development. An intentional sampling was used. None of the participants reported having problems understanding the instructions of the questionnaire or any of its items. However, one of the participants suggested a possible change toward a more inclusive terminology, modifying the terms "fathers" and "mothers" by parents to also contemplate gay or single parent families. No other changes were made to the wording of the items or instructions of the translated version of the questionnaire.

Once the final questionnaire was obtained, it was administered to parents of children with neurotypical development and NDD. A total of 591 responses were obtained for total sample, of which 17 did not meet any of the established inclusion criteria, leaving 574 valid responses from parents living in Spain. To complete the questionnaire, parents were asked to indicate the answer that best described their children's play on a 5-point Likert scale from 1 = never to 5 = always. The same instructions were followed as in the original questionnaire. Parents completed the questionnaire at home. In case any questions arose while completing the questionnaire, they were provided with the email of the main researcher and the telephone number of the contact researcher. All parents who agreed to participate in the study completed the questionnaire. Once the questionnaire was completed, it was reviewed by the group of researchers, and in case of doubts or observations made by the parents, they were contacted to resolve possible doubts or suggestions.

### *2.4. Statistical Analysis*

Construct validity allows knowing the degree to which the items of the instrument measure the theoretical construct that they intend to measure. The construct validity of the questionnaire was determined through factor analysis, which is a statistical technique that allows structuring a dataset into factors or components. AMOS extension (version 18.0) is structural equation modeling software that was used to conduct confirmatory factor analysis (CFA). The maximum likelihood method was used for the estimation of the goodness of fit parameters. Once it was verified that the goodness of fit parameters did not allow confirming the model with Pearson's correlations and neither did they improve after eliminating with less weight in each factor, a new CFA was carried out with the FACTOR software (version 10.10.03) [28], since the answer options of the questionnaire are through a Likert scale with 5 answer options. Polychoric correlations with robust analysis and unweighted least squares (ULS) were used, and the method for factor extraction was normalized varimax [29,30]. CFA was performed to verify that the dimensions identified by the authors of the original tool were valid in the translated version of the questionnaire. For this, the measures of the quality of the fit of the model were evaluated through the absolute fit measures: the Chi-square divided by degrees of freedom (CMIN/DF), the Root Mean Square Error of Approximation (RMSEA), incremental adjustment measures such as the comparative fit index (CFI), Tucker–Lewis index (TLI), and other adjusted goodness fit statistics: goodness of fit index (GFI), adjusted goodness of fit index (AGFI).

Reliability of internal consistency of the MCP questionnaire was calculated using Cronbach's alpha. Reliability for each factor was carried out by calculating this coefficient for the items with respect to the global score and the other coefficient for the items of each domain with respect to its value. Cronbach's alpha values >0.70 were considered acceptable to guarantee the internal consistency of the questionnaire [31].

Furthermore, relative operating characteristic (ROC) analysis was performed to know the MCP's ability to classify between children with NDD and neurotypical development. The characteristics of the participants were analyzed using simple descriptive statistics. These statistical analyses were performed using IBM SPSS Statistics for Windows software (version 26.0, IBM Corp., Armonk, NY, USA), for others statistical analysis of the data. Statistical significance was set at *p* < 0.05 (bilateral).

### **3. Results**

The sample in this study was 574 families of children (326 boys, 56.8%, and 248 girls, 43.2%) between 3 and 9 years old. Of these, 469 (81.7%) were children with neurotypical development and 105 (18.3%) were children with NDD: DD (*n* = 47; 8.2%), SLD (*n* = 23; 4%), and ASD (*n* = 34; 5.9%). The mean age of the children was 5.55 years (SD = 1.92), and the mean age of the parents was 39.41 (SD = 5.36) and 39.38 (5.23) respectively. Regarding the characteristics of the children, birth was premature in 59 cases (10.3%) and at term in 515 (89.7%).

### *3.1. Construct Validity*

The construct validity was verified using a CFA with the factor model proposed by the authors of the original questionnaire conforming to the data that we have obtained in the Spanish population (Figure 1). The CFA with FACTOR Software confirmed the four-factor structure for the MCP (Table 2).


**Table 2.** My Children's Play goodness-of-fit index from the confirmatory factor analysis.

p (χ2): Chi-squared probability; RMSEA: root mean square error of approximation; CFI: comparative fit index; NNFI: non-normed fit index, RMSR: root mean square of residuals.

**Figure 1.** Model for neurotypical, specific language disorder (SLD), and autism spectrum disorder (ASD) children. χ2 = 817.584, gl = 737.89 *p* = 0.020 **Figure 1.** Model for neurotypical, specific language disorder (SLD), and autism spectrum disorder (ASD) children. χ2 = 817.584, gl = 737.89 *p* = 0.020.

> **Table 2.** My Children's Play goodness-of-fit index from the confirmatory factor analysis.  **Index Cut-Off Original Model Value Alternative Model Value**  Model Fit 817.584 The composition of the four factors and their factor loading are shown in Table 3. The four MCP factors explain 42.98% of the variance. The first factor, flexibility and executive attention, explained 25.40% of the variance. The second factor, the environmental context, explained 7.11% of the variance. The third factor, play characteristics, explained 5.69%, and the last factor, play preferences and interpersonal relationships, explained 4.77%. Items 27, 15, and 35 were not included in any factor.

> > *p* = 0.020




**Table 3.** *Cont.*

The final questionnaire in the Spanish version consisted of 40 items instead of 43 in the original version (Table 4 & Appendix A).


**Table 4.** Distribution of items for each factor in original and for Spanish version of My Children's Play.

### *3.2. Reliability*

The reliability analysis showed a Cronbach's alpha for total score (0.695) that is acceptable. Despite being slightly below the level of acceptability that was set, we can assume that it is practically 0.7, and therefore, it is an acceptable value, indicating that the internal consistency is adequate (Table 3).

### *3.3. Interpretability*

Table 5 shows the mean scores obtained by neurotypical children and by those with NDD in MCP and in each of its four factors. There were significant differences between neurotypical children and children with NDD with lower scores in the first group (Table 5). Likewise, Figure 2 shows the ROC curve for the predictive level of MCP in the children with NDD. The area under the curve was 0.876 (CI 95%, 0.840–0.912).

**Table 5.** Mean scores in neurotypical and neurodevelopmental children.


**Figure 2.** Relative operating characteristic (ROC) curve used to determine the predictive value of **Figure 2.** Relative operating characteristic (ROC) curve used to determine the predictive value of MCP in the diagnosis of children with neurodevelopmental disorder.

MCP in the diagnosis of children with neurodevelopmental disorder. **4. Discussion**  The objective of this study was to translate and culturally adapt the MCP question-Accordingly, the optimal cut-off score to differentiate children from the neurotypical group versus children with NDD was 142 points. Thus, the score <142 in the MCP is indicative of NDD according to the MCP.

#### naire to the Spanish population, in addition to studying its reliability and validity, provid-**4. Discussion**

ing a new assessment resource for pediatric health and educational professionals that allows them to learn about participation in play in children with neurotypical development and children with NDD. Play is of great importance and very significant in childhood, the assessment of which has been hampered by the complexity for defining the concept and because it is a behavior that is difficult to standardize and quantify. For professionals and those interested in assessment, the lack of time and resources and having only clinical environments where the behavior during play is different from that manifested in the family environment means that many professionals do not use the existing standardized tools [32,33]. These problems can be addressed with questionnaires such as My Child's Play. Scores on the MCP questionnaire represent the parental perceptions on daily living. Although MCP does not include observations or tests of the child's play performance, it could also be considered a strength, since it is relevant from the point of view of familycentred practice, allowing to know the functioning of the child in everyday life and showing good ecological validity. MCP has good psychometric properties [25]. The MCP allows knowing different factors underlying the play that are relevant in the assessment of children with neurodevelopmental disorders: (1) cognitive such as executive functioning; (2) socio-emotional, such as social interaction and participation; (3) the behavior during play; and (4) characteristic of play and child's play preference. Finally, the MCP is a short questionnaire, easily understandable by parents, and easy to complete. *4.1. Reliability*  The objective of this study was to translate and culturally adapt the MCP questionnaire to the Spanish population, in addition to studying its reliability and validity, providing a new assessment resource for pediatric health and educational professionals that allows them to learn about participation in play in children with neurotypical development and children with NDD. Play is of great importance and very significant in childhood, the assessment of which has been hampered by the complexity for defining the concept and because it is a behavior that is difficult to standardize and quantify. For professionals and those interested in assessment, the lack of time and resources and having only clinical environments where the behavior during play is different from that manifested in the family environment means that many professionals do not use the existing standardized tools [32,33]. These problems can be addressed with questionnaires such as My Child's Play. Scores on the MCP questionnaire represent the parental perceptions on daily living. Although MCP does not include observations or tests of the child's play performance, it could also be considered a strength, since it is relevant from the point of view of familycentred practice, allowing to know the functioning of the child in everyday life and showing good ecological validity. MCP has good psychometric properties [25]. The MCP allows knowing different factors underlying the play that are relevant in the assessment of children with neurodevelopmental disorders: (1) cognitive such as executive functioning; (2) socioemotional, such as social interaction and participation; (3) the behavior during play; and (4) characteristic of play and child's play preference. Finally, the MCP is a short questionnaire, easily understandable by parents, and easy to complete.

The questionnaire shows good reliability. Cronbach's alpha coefficient indicates that the internal consistency is acceptable (α = 0.695 for the total score, while in the original, it

### *4.1. Reliability*

The questionnaire shows good reliability. Cronbach's alpha coefficient indicates that the internal consistency is acceptable (α = 0.695 for the total score, while in the original, it is 0.86). In addition, in the four dimensions Cronbach's alpha were 0.861, 0.639, 0.838, and 0.821, respectively. These values were similar to those of the original questionnaire, which were 0.80, 0.81, 0.67, and 0.63. As with the original questionnaire, the factor referring to the environmental context was the one that presented lower reliability.

### *4.2. Construct Validity*

Regarding the construct validity, the results obtained in the CFA of the Spanish translation of the questionnaire do not allow us to support the same original factorial solution of the English version. However, both versions have four factors, although the weight of each of them is different for the two populations. The factor analysis of the original questionnaire revealed the existence of four factors (executive functions, interpersonal relationships and social participation, preferences and choices in the game, and opportunities in the environment) that explained 30% of the total variance and supported the original concept of the authors of the elaboration of a questionnaire based on the person– occupation–environment relationship. The CFA of the version translated into Spanish determines that the construct validity is not completely adequate, as indicated by the values produced by the adjustment variables (RMSEA = 0.058, CFI = 0.792, NNFI = 0.780, and CIM/DF = 2.937); therefore, the model proposed in the original questionnaire did not fit the data obtained in the Spanish population. However, the new factorial solution obtained also supports the person–environment–participation relationship proposed by the original authors. It is important to note that both populations can identify the same factors, although in a different order, and that therefore, the overall structure of the questionnaire is similar in both versions. In the Spanish version, the first factor was executive functions, the second factor was the environmental context, the third factor referred to the characteristics of the play, and the fourth factor can be understood as referring to preferences in the play and interpersonal relationships.

The differences between the original model and the Spanish version may be due to different factors. Probably the most important is that the sample used in our study included children not only with neurotypical development but also with NDD according to the prevalence in the Spanish population [34]. This allows us to differentiate between children with neurotypical development and NDD through the MCP, as indicated by the results of the ROC curve.

This explains the fact that the first factor, referring to executive functions, refers fundamentally to cognitive flexibility and self-regulation of attention, which can be a weak point in children with ASD and SLD. In addition, these functions keep changing between 3 and 9 years [35,36], so it is relevant that it emerges as an important factor in the child's play. Executive functions develop during childhood and are complex mental activities that allow the child to plan, make decisions, show a flexible behavior, to change from one task to another, or have inhibitory control during play. In addition, the play contributes to the self-controlled development of executive functions, obtaining greater benefits when it is slightly structured [37]. Furthermore, the development of executive functions has been related to parenting patterns. In this way, it is possible that these differences are showing different customs and traditions than in the way in which parents understand the play, the type of context in which it is played, the rules, and demands of the context. These aspects are coherent if one takes into account that the play, as a human occupation, has a social and anthropological component and not merely a cognitive one [38].

Another possible explanation for the differences found is that, in the original research, only mothers were included, compared to the inclusion of both parents in our study. The relationship of mothers and fathers with their children is different, and many authors have agreed that today, there is still a greater involvement and presence of women in parenting, that leads to a greater appearance of positive interactions compared to fathers [39]. Furthermore, the sociocultural context has a fundamental role in explaining the play [40].

The Spanish version consisted of 40 items. Removed items from the updated version of the MCP were the following: 27 (My child persists at play only with toys/games that he likes and finds interesting), 15 (My child is able to cope with frustrating situations that arise during play), and 35 (My child prefers to play for long periods with toys and materials that enable him to touch different textures). The first and second items can be related to both executive functions and self-regulation of attention and can be related to items 7, 22, and even item 10. On the other hand, the elimination of item 35 may be due to this type of play in Spanish culture being considered suitable between 18 and 36 months [41,42].

### *4.3. Interpretability*

This study provides preliminary evidence of the discriminant validity of the MCP among children with neurotypical development and NDD, as shown by the scores for factors 1, 3, and 4. In addition, the total MCP score allows for the consistent differentiation of children with disorders of NDD with neurotypical development, with the established cut-off point of 142.

### *4.4. Implications in the Practice*

This study is especially relevant since it is the first one to provide a version in Spanish for the assessment of the play. It provides a simple and quick tool for the evaluation of the play by health professionals and educators, allowing the detection not only whether there are differences between the play of children with NDD or neurotypical development, but it also allows us to know the dimensions in which there are differences, such as executive functions, play preferences, and interpersonal interactions, environmental context, and/or play characteristics. The MCP has a series of advantages for the assessment of play in childhood, allowing guiding treatment on the underlying factors that affect successful participation in the play.

In addition, it is a short, simple, and easy questionnaire for parents to fill out that allows for relatively easy screening. This questionnaire can facilitate professionals working in the field of childhood, to establish intervention programs and treatment plans in children between 3 and 9 years, and with NDD, especially in those with ASD, SLD, and DD.

### *4.5. Limitations and Future Research*

The present study has some limitations. First, we took a non-random sample. Therefore, the study should be replicated in a representative random sample of parents of neurotypical children. Second, it would be advisable to expand the sample size further and be able to establish levels according to different age groups. As a future line of research, it would be of interest to compare different clinical populations with other NDD, such as attention deficit hyperactivity disorders or motor coordination disorders and to establish different profiles in the play, which allow orienting the targeted intervention for these specific groups. Likewise, it would be of interest to be able to develop instruments for younger children and babies not only based on parental perceptions but also on the child's performance in the play.

### **5. Conclusions**

The MCP provides a unique understanding of the processes underlying the play; especially, it allows knowing how some executive functions influence during it, such as cognitive flexibility and attentional control. The MCP allows us to assess the play from a broad perspective. In this sense, it helps us knowing which elements of the child's participation are appropriate to the context, as well as whether the conditions of possibility allow the child to participate positively in the play. It also gives the opportunity of knowing the characteristics of the child's play, their preferences, and social interaction during it.

The results of cross-cultural validation and psychometric analysis confirm its internal consistency, as well as the construct validity and discriminant validity in the Spanish population and with children with ASD, SLD, and delayed development.

**Author Contributions:** Conceptualization, D.R.-A. and M.A.M.; methodology, D.R.-A., M.A.M. and M.R.-S.; validation, M.R.-S. and S.B.-F.; formal analysis, D.R.-A. investigation,; writing—original draft preparation, D.R.-A., S.B.-F., M.R.-S., D.M., J.R. and M.A.M.; writing—review and editing, D.R.-A., S.B.-F., M.R.-S., D.M., J.R., J.M.T.-J. and M.A.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics and Research Committee of the University of Granada (code 1426/CEIH/2020; date of approval: 04/24/2020).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy reasons.

**Acknowledgments:** The authors would like to acknowledge the support received from the original MCP authors, Eleanor Schneider and Sara Rosenblum, in developing this study.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **Appendix A. Spanish Version of My Child's Play (MCP) of © Schneider & Rosenblum 2014**

El niño juego con los juguetes según el uso previsto.

El niño varía el tipo de juego con los juguetes.

El niño pierde el interés por los juguetes.

El niño utiliza ambas manos para jugar.

El niño tiene dificultad para concentrarse con ruido de fondo.

El niño se choca contra los objetos o los deja caer durante el juego.

El niño persiste en el juego incluso cuando tiene dificultades.

El niño intenta resolver los problemas por sí mismo durante el juego.

El niño no puede organizarse para jugar sin la ayuda de un adulto.

El niño necesita ayuda de un adulto para concentrarse en el juego.

El niño se adapta fácilmente a los cambios en las condiciones de juego.

El niño se adapta fácilmente a la intervención en el juego de nuevos adultos o niños.

El niño se relaciona con otros niños durante el juego.

El niño juega con los demás niños de acuerdo con las normas del juego.

El niño puede iniciar el juego.

El niño asume el papel de líder del grupo durante el juego.

El niño está dispuesto a compartir juguetes con otros.

El niño adapta su comportamiento durante el juego al contexto.

El niño controla sus impulsos durante el juego con los demás.

El niño necesita ayuda de un adulto para unirse al juego de un grupo de niños.

El niño prefiere jugar solo con juguetes familiares.

El niño encuentra la oportunidad para jugar en cualquier lugar.

El niño evita el juego que requiere movimiento.

El niño disfruta del juego imaginativo.

El niño necesita muchos descansos para mantenerse atento.

El niño pierde el interés cuando juega solo.

El niño prefiere jugar con adultos en vez de niños.

El niño no juega a juegos que tienen reglas.

El niño se choca contra objetos y superficies a propósito.

Hay espacio accesible para el juego fuera de casa.

Hay espacio accesible dentro de casa para jugar. El niño tiene dificultades cuando juega con demasiados estímulos visuales. El niño cuenta con suficientes juguetes para disfrutar de un juego variado. Los juguetes están organizados en casa para que sean fácilmente accesibles. Tengo en cuenta las preferencias de juego de mi hijo. Le ofrezco ayuda después de que el niño haya intentado jugar solo. Adapto el juego a las capacidades del niño. Defino claramente las reglas para que el niño pueda divertirse. Mi rutina diaria incluye tiempo para jugar con el niño. Estoy satisfecho en la forma que mi hijo juega.

### **References**


## *Article* **A New Instrument to Assess Children's Understanding of Death: Psychometrical Properties of the EsCoMu Scale in a Sample of Spanish Children**

**Manuel Fernández-Alcántara <sup>1</sup> , Macarena de los Santos-Roig <sup>2</sup> , María Nieves Pérez-Marfil 2,\* , Francisco Cruz-Quintana <sup>2</sup> , Juan Manuel Vázquez-Sánchez <sup>3</sup> and Rafael Montoya-Juárez 2,4**


**Abstract:** The acquisition of the death concept in children may influence how these children cope with the losses that they will confront throughout their lives. At the present time, there is a lack of psychometric instruments in Spanish-speaking countries in order to evaluate the components of the death concept in children. The aim of this study was to create and validate a scale (EsCoMu-Escala sobre el Concepto de Muerte) in order to provide insight about the concept of death in children. The sample was formed by 358 children from ages 6 to 13 years. The final EsCoMu version has 27 items which serve to evaluate universality, irreversibility, non-functionality and causality. The results of the confirmatory factor analysis show an adequate fit index for the four dimensions model, reliability (α = 83) and validity evidence, specifically based on the children's age. In conclusion, EsCoMu is an instrument that shows adequate reliability and validity indices in order to assess the concept of death and its four components among children. Due to its simplicity, this instrument can be very useful if applied to the field of neurodevelopmental disorders.

**Keywords:** death concept; school; causality; irreversibility; universality; non-functionality; children; scale development; neurodevelopment; grief

### **1. Introduction**

In Western societies, death is often considered a taboo subject. In the case of children, in addition to the social reticence to talk about death, the way that the concepts of death and dying are learned is extremely important [1] and may influence how they cope with grief and loss. Previous research seems to indicate that children grief manifestations are directly associated to the knowledge they have about death [2]. Also, the possibility to talk about death and understand its meaning may help children to overcome the mistaken ideas or the appearance of unnecessary fears that can have an impact on children's emotional life even in adulthood, interfering in the normal elaboration of bereavement processes that they will have to deal with in the future [3]. Recent reviews also consider the concept of death as a core aspect when communicating bad news to children and adolescents [4]. The concept of death is very complex, as it is influenced by variables such as social beliefs, cultural norms, emotions, biological development, cognitions or previous experiences with death [5,6].

Several studies highlight four core components related to the concept of death: universality, irreversibility, non-functionality and causality [7]. Universality implies that death is conceptualized as a natural phenomenon that applies to all living beings. Irreversibility is linked to the understanding that the dead cannot come back to life. Non-functionality

**Citation:** Fernández-Alcántara, M.; de los Santos-Roig, M.; Pérez-Marfil, M.N.; Cruz-Quintana, F.; Vázquez-Sánchez, J.M.; Montoya-Juárez, R. A New Instrument to Assess Children's Understanding of Death: Psychometrical Properties of the EsCoMu Scale in a Sample of Spanish Children. *Children* **2021**, *8*, 125. https://doi.org/10.3390/ children8020125

Academic Editor: Douglas L. Hill Received: 15 December 2020 Accepted: 6 February 2021 Published: 9 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

includes the acknowledgement that, once a person has died, their bodily functions cease, as well as their internal and external actions. Lastly, causality implies the understanding of the possible internal or external factors which can cause the end of life. Other authors have proposed other dimensions, such as inevitability, personal mortality or unpredictability [8], but most of them agree on the model proposed by Speece and Brent [7].

There are multiple factors that can influence the understanding of the death concept, even if the scientific evidence of previous investigations is contradictory. Age is one of the most consistent variables, as the death concept is more defined the older the child is. However, each component (universality, irreversibility, non-functionality and causality) seems to follow different patterns [9–12]. The child's previous experience with death or illness has also been positively linked to the understanding of the components of the death concept [9]. However, other studies did not find any differences [3,10,13]. The cognitive ability of the children has also been considered an important aspect, especially in the background of the cognitive development model by Jean Piaget. During the preoperational stage, there is a predominance of magical thinking and egocentricity, so it is harder for the child to understand the different aspects of the concept of death. However, De la Herrán et al. [14,15] noted that, as early as 3 years old, children are able to distinguish between life and absence of it. Moreover, between 3 and 5 years old, children start to be curious about the signs of devitalization and the causes of death [16,17]. In subsequent stages, such as in the concrete-operational (7–11 years), the child begin to understand the logical operations and reversibility of thought [18], so they can develop a more mature understanding of death by including components such as irreversibility, non-functionality and causality [19]. Different studies have also indicated the existence of two complementary approaches to death in children that include the biological aspect and the meta-psychological, afterlife or religious conception of death [20]. Both conceptions of death seem to be influenced by culture [21,22].

Keeping in mind the construct complexity, it is necessary to have valid and reliable assessment instruments in order to evaluate the concept of death in children. Previous investigations have used open qualitative interviews, as well as other art-related approaches, such as drawings, storytelling or play in order to investigate the understanding of the death concept [3,9,23]. However, there is a lack of quantitative instruments showing adequate reliability and validity indices in order to assess the concept of death, as, to our knowledge, a Spanish adapted quantitative scale that meets such conditions has yet to be created.

One of the classic resources is the Death Concept Questionnaire [24], which is formed by two groups of 13 questions about death in people and animals. The factor analysis displayed four main factors: irreversibility, non-functionality, causality and inevitability of death and old age [9]. The rating of each item depends on the correctness of the reply (ranging from 0 to 3) and the sophistication of the child's explanation, finding an overall Cronbach's alpha of 0.77 [24]. The main limitations of the Death Concept Questionnaire are that each factor is composed of a small number of items and that the responses of the child need to be categorized.

In Spanish, we can find two qualitative instruments. First, we can find the interview developed by Viñas et al. [25,26], named "Entrevista Estructurada del Concepto de Muerte—ECM" ("Structured Interview on Death Concept"), which evaluates the universality, irreversibility and non-functionality through 11 closed questions with dichotomous answers. Another series of 17 questions are included in order to assess beliefs related to afterlife and the child's personal experience with death, as well as three open questions where the child is asked to specify three causes of death (animal, person and own) and a final question about the definition of suicide [26]. In Mexico, Gutiérrez et al. [27] have recently developed a qualitative interview in Spanish, where 14 items assessing universality, finality, non-functionality and causality where used. The main limitation of both instruments is that they don't report evidence involving instrument validity, reliability or factorial structure.

In accordance with the above, it is essential to develop a new instrument in the Spanish language which allows us to assess the components of the death concept in children while having proper psychometric features. The main benefit of having an instrument of this kind is that it can be easily completed and quickly distributed among a large number of participants, without having to invest too much time codifying or correcting the results (as in interviews or other qualitative approaches). Also, the scores from the scale can be easily compared between studies and populations. Finally, we need to establish a reliable measure of the death concept among primary school children which will serve as a starting point to assess other populations of children, such as those with intellectual disabilities or other neurodevelopmental disorders.

Due to this, the main aim of this study is to develop and present the psychometric properties (factorial structure, reliability and validity) of a scale which is able to assess the acquisition of the components of the concept of death in primary school children (6–13 years). The factorial structure of the scale will be tested through confirmatory factor analysis, and reliability will be calculated through internal consistency analysis. The main validity evidence was assessed through the comparison of the scale between different age groups. We hypothesized that younger children will have lower scores in the four dimension of the death concept, in comparison with older children. In addition, we wanted to explore the differences in the following variables: sex, existence of a previous loss and school setting.

### **2. Materials and Methods**

The study was formed by 358 primary school students (6–13 years) coming from five schools of the Spanish provinces of Granada, Jaén and Cádiz (see Table 1).


**Table 1.** Schools in which the sample is taken.

The mean age of children was 9.92 years (SD = 1.57). Different schools of the provinces mentioned before were contacted to select the participant sample. The inclusion criteria were: school's willingness to participate, informed consent signed by children's parents or legal guardians, and children who were between 6 and 13 years old. Data were collected about each student's sex, age and their school setting. Given the foreseeable differences between children of different ages, they were grouped into four levels (see Table 2).

**Table 2.** Sociodemographic data sample divided by age group.


### *2.1. Instruments*

Ad-hoc demographic data questionnaire:

Age, sex and school setting (rural/urban/semi-urban) were considered. When the school setting was not fully rural or urban (e.g., towns located in a range of less than 10 km from the province capital), it was coded as semi-urban. A question asking whether they have suffered a recent loss (yes/no) was also added.

Scale to assess the concept of death—EsCoMu (Escala sobre el Concepto de Muerte):

This dichotomous scale was developed by four of the authors of this study who are professionals and researchers specialized on the field of bereavement and end of life processes. The dichotomous rating (yes/no) was based on previous instruments and interviews used in the field [25,28]. The items were evaluated by a panel of experts, who established items relevancy, adequacy and belonging to each of the four dimensions or theoretical components: irreversibility, universality, non-functionality and causality [29]. Each item was rated on a likert scale ranging from 0 to 100 assessing its relevancy (if the item was significantly relevant for the dimension assessed) and adequacy (if the item was appropriate for the proposed dimension of the concept of death). Experts could also include qualitative commentaries about the items. The initial version of the scale consisted on 38 items, of which 10 were eliminated as they showed mean values less of than 70 in any of the dimensions assessed (relevancy and/or adequacy), as well as those that experts identified as difficult to understand for children (in the qualitative part of the survey). An additional item was removed because of their lower factor loadings in the exploratory analysis.

The definitive EsCoMu version is formed by a group of 27 dichotomous items (yes/no answer), grouped into four dimensions: 6 for universality, 7 for irreversibility, 7 for nonfunctionality and 7 for causality (see Supplementary Material for the Spanish version of the scale). Each item has a score of 1 (if the answer is correct) or 0 (if the answer is incorrect), with some items being reversed. The total score is calculated by adding all items from each component. The scale global Cronbach's alpha of this investigation was α = 0.83.

### *2.2. Procedure*

Two informative meetings were held in order to inform the management of each school participating in the study. After obtaining the school's permission, with the help of Parent Associations (AMPA—Asociación de Madres y Padres de Alumnos), an informative meeting with parents was held in order to explain the aim of the study. Secondly, each student was given an informative sheet and an informed consent in order to be signed by their parents. Subsequently, this consent was returned to the school teacher along with each parent's or legal guardian's signature. The students whose parents did not fill in the informed consent performed a different activity unrelated to the study.

The assessment was performed in groups, in the students' regular classroom. In one session, they filled in all the sociodemographic data as well as the scale EsCoMu. The evaluation lasted around 20 min, and all students received similar guidelines. The evaluation was performed by a team of experts experienced in end-of-life processes.

At the beginning of the evaluation, all students were given the option of not participating in the activity if they did not want to, regardless of the parent's consent. Each participant's emotional state was evaluated during and after the assessment in order to provide them with emotional support if needed, but none of the participants had reported issues in this regard.

### *2.3. Ethical Considerations*

Prior to the data collection and the inclusion of participants in the study, both school management and parents were informed about the aim, purpose and confidentiality of the study. In every case, the evaluation was performed after obtaining both the school authorization and after collecting parent-signed informed consent for their children's participation. The present study was approved by the University of Granada's Ethics Committee on Human Research (Ref. 1056/CEIH/2020).

### *2.4. Data Analysis*

To perform the descriptive analysis, the frequency of each answer was calculated for every item. To verify the factor structure, a confirmatory factor analysis (CFA) was performed. Given the dichotomous condition of data, the WLSMV (Variance-Adjusted Weighted Least Squares) estimation method was used. The following indices were used in order to calculate items' fit with the proposed model: RMSEA (Root Mean Square Error of Approximation), TLI (Tucker–Lewis Index), CFI (Comparative Fit Index) and WRMR (Weighted Root Mean Square Residual).

To gather validity evidence about possible relations between EsCoMu's ratings and other external variables, multivariate analysis of variance (MANOVA) was applied, with variables including age, sex, having suffered a loss (yes/no) and school setting (urban, semiurban and rural) for each subcomponent of the scale. For post-hoc contrasts, Bonferroni correction and partial eta-squared effect size were used. Statistical software SPSS 22 (IBM, 2013) was used. For gathering structural validity evidence, MPLUS 6.11 [30] was used.

### **3. Results**

### *3.1. Descriptive Analysis of EsCoMu's Response*

In Table 3, the percentage of correct answers for each age group is shown in each EsCoMu scale item, based on their dimension.


**Table 3.** Descriptive analysis of the EsCoMu scale items for each age group.

### **Table 3.** *Cont.*


\* Reverse items, U = Universality, IR = Irreversibility, NF = Non-Functionality, C = Causality.

### *3.2. EsCoMu Factor Structure*

Two one-order models (One-Factor and Four-Factor) were tested. However, fit indices for the one-factor model (see Table 4) were not adequate. As correlation values were not appropriate for causality-universality dimensions on the Four-Factor model (with a coefficient bigger than 1), a second-order model was tested (Second order with Four Factor model). CFA results showed adequate fit indices in this model (see Table 4 and Figure 1). The dimensions showed medium-high values of intercorrelation (see Table 5).

**Table 4.** Fit indices for EsCoMu factor models.


Note. \*\* = *p* < 0.001, X<sup>2</sup> = Chi Square goodness of fit statistic, *df* = degrees of freedom, CFI = Comparative Fit Index, TLI = Tucker–Lewis Index, RMSEA = Root Mean Square Error of Approximation, WRMR = Weighted Root Mean Square.

### *3.3. Evidence of Validity*

As observed in Table 6, the MANOVA results indicate a significative medium-low effect size with age in all EsCoMu dimensions as well as in the total score. Post-hoc analysis (Bonferroni) indicate lower scores in Universality, Irreversibility, and Non-functionality in the 6–7 years age group if compared with the 8–9 years age group (*p* < 0.01) and with the 10–11 years age group (*p* < 0.01). There were no statistically significant differences among the youngest and oldest age groups. However, both in the Causality dimension and in the total score, the youngest children showed statistically significant differences (both *p* < 0.01) if compared with other groups, where such differences do not occur.

As observed in Tables 7 and 8, there were no differences between any of the instrument dimensions involving sex, nor between the participants who have suffered a recent loss and those who did not.

**Figure 1.** Second-order four factors EsCoMu model. Latent variables are represented by ellipses and measured variables are represented by rectangles. Values are standardized estimated correlations between factors.

**Table 5.** Intercorrelations between EsCoMu dimensions.


Note. \*\* = *p* < 0.01.


**Table 6.** MANOVA results for school setting with EsCoMu dimensions and total scores.

Note. 1. = Universality, 2. = Irreversibility, 3. = Non-functionality, 4. = Causality, SD = Standard Deviation, df = degree of freedom, η <sup>2</sup> = partial eta-squared effect size, \*\* = *p* < 0.01.


**Table 7.** MANOVA results for gender with EsCoMu dimensions and total scores.

Note. 1. = Universality, 2. = Irreversibility, 3. = Non-functionality, 4. = Causality, SD = Standard Deviation, df = degree of freedom, η <sup>2</sup> = partial eta-squared effect size

Finally, when analyzing the children's school setting, MANOVA showed lower results in rural areas (Table 9). The effects appear in the four dimensions as well as in the instrument's total score, with medium-to-low effect sizes. The post-hoc analysis results (Bonferroni) indicate differences only between rural and semi-urban settings (*p* < 0.01) in terms of Universality, Irreversibility and Non-functionality. Again, in this case, Causality and the EsCoMu total score are the dimensions with more differences between rural settings and the other two groups (*p* < 0.05).


**Table 8.** MANOVA results for recent loss with EsCoMu dimensions and total scores.

Note. 1. = Universality, 2. = Irreversibility, 3. = Non-functionality, 4. = Causality, SD = Standard Deviation, df = degree of freedom, η <sup>2</sup> = partial eta-squared effect size

**Table 9.** MANOVA results for school setting with EsCoMu dimensions scores.


Note. 1. = Universality, 2. = Irreversibility, 3. = Non-functionality, 4. = Causality, SD = Standard Deviation, df = degree of freedom, η <sup>2</sup> = partial eta-squared effect size, \* = *p* < 0.05, \*\* = *p* < 0.01.

### **4. Discussion**

The aim of this study was to develop and present the psychometric properties (factor structure, reliability and validity) of a scale which was able to assess the acquisition of the components of the concept of death in primary school children (6–13 years). Results prove that the scale has an adequate reliability and factor structure, showing promising validity evidences.

The scale global alpha shows acceptable values, in the same direction as the value reported to this date by the Death Concept Questionnaire (α = 0.77 in the original study and α = 0.81 in the study by Bonoti et al. [9]). Test–retest measures could not be included, so future research should investigate whether the EsCoMu scale maintains its reliability over time, as well as whether it is sensitive to death education-based interventions [2,14].

The four scale factors showed positive and moderate correlations between each other. Furthermore, the CFA model showed that the four components of the concept are closely related to each other, suggesting that they are part of an underlying construct, in this case the concept of death. This is supported by the results of the second-order CFA, where the concept of death is explained by the four factors, which are also explained by their

respective items. Following this rationale, previous research has highlighted the relevance of these four components to explain the concept of death [6,10].

Age is one of the factors that seems to systematically influence the acquisition of the concept of death. In the present study, age-dependent differences were found in the four evaluated dimensions, in line with previous investigations [10,31]. In the dimensions of universality, irreversibility and non-functionality, differences were identified between younger and older children, whereas, in causality, statistically significant differences were found between 6–7-year-old children and the rest of the age groups. This seems to indicate that not all dimensions are acquired by following the same pattern. However, we do observe that, from the age of 8–9 years, all four subcomponents are well acquired in children [7,9]. In their sample, Gutiérrez et al. [27] found that not all the components of the death concept differed according to age, finding no significant differences in universality and causality based of this variable, but rather between finality and non-functionality. However, results for these variables were very close to significance (*p* < 0.08 in both cases).

Regarding the school setting, lower scores have been found in rural schools as compared to these semi-urban and urban schools. Previous studies have shown different results, pointing to greater acquisition in children living in rural environments. Panagiotaki et al. [31] found significant differences in the irreversibility component when comparing three children groups (British living in London, British Muslims living in London, and Muslims living in rural areas of Pakistan), being higher in the latter group. However, in this study, since the groups were not equivalent due to a series of key cultural variables, no conclusive evidence can be drawn regarding the area of origin. Lastly, other studies did not find significative differences regarding the death concept in urban and rural settings [32].

In the present study, we also did not find significant differences in any of the components of the death concept based on the child's sex or in the event of having a recent loss. These findings are consistent with one prior study [3], but differ from another study that found differences on this variable [9]. Future studies must investigate the influence of the loss type and the appearance of specific symptoms in terms of bereavement, which sheds light on to what extent it is the experience of loss itself, or the intensity of the child's experience that will affect the acquisition of the concept of death [33].

The development of a scale that serves to evaluate the concept of death has important clinical implications. On one hand, death is not a common topic in school subjects or academic curricula. On the other hand, adults and families are often hesitant about how to respond to the questions that children raise about the concept of death and the dying process. Moreover, in spite of children's curiosity, some adults are unsure about the appropriateness of discussing death with their children [34]. Therefore, it is a common situation that children are not able to find the adequate space to clarify their doubts regarding what death means. This may prevent them from developing more adaptive coping responses, which can lead to emotional issues. Therefore, it is essential to have valid and reliable instruments which allow us to evaluate the conceptualization of death in different ages and contexts, as well as to work in education, as many teachers are currently demanding [35,36].

The EsCoMu scale, due to its fast and easy application, can be widely used in populations with neurodevelopmental disorders or problems. Previous qualitative studies have shown that, in cases involving diagnosed intellectual disabilities or neurodevelopmental conditions, the acquisition of the death concept seems to follow a different pattern. Markell and Hoover [37] highlight how even learning problems or physical and emotional issues in children can affect the bereavement process and understanding of death. Children diagnosed with intellectual disability (ID) have shown confusion and difficulty understanding the concepts of non-functionality, irreversibility (associating death with the illness) and universality [37]. Finally, recent studies in adults diagnosed with ID show that they do acquire the components of the death concept, but in a different and, in many cases, partial way [38,39], showing greater difficulty understanding concepts such as causality and universality [40]. Future studies that use the EsCoMu scale may explore the concept of death in these populations, as well as the effectiveness of interventions related to death education and supporting methods for bereavement and end-of-life processes in this population [41].

However, this research has a series of limitations: Firstly, the child age groups were not equivalent, the 10–11-year-old group being the biggest one. Moreover, it is necessary for future research to examine the usefulness of EsCoMu among populations under 6 years of age as well as to include test–retest measures to evaluate the temporal reliability of the scale. We did not perform a pilot study or cognitive interviews prior to the initial assessment, so it may be useful to perform pilot testing when applying this scale to children less than 6 years old. However, the assessed population did not have any problem understanding any of the items. No measures of anxiety or depression were taken in the children who completed the scale, so it would be necessary in the future to control such variables and check their effect and connection to the EsCoMu score. Future studies should verify if the acquisition of the components of the death concept correlates with the most common themes regarding death, such as biological, psychological or metaphysical death [3,9]. In the present study, age was considered as the main evidence of validity, but there are other variables associated with the concept of death, such as religious, cognitive or socioeconomic aspects, that should be included in future studies to have measures of convergent validity of the EsCoMu scale. Finally, the use of mixed methods design to explore the relationship between the acquisition of the components of the concept of death and the subjective experience of the child [27] will give additional information about the validity of the EsCoMu scale.

### **5. Conclusions**

In conclusion, the EsCoMu scale is an instrument with adequate factor structure that shows adequate reliability and validity indices in order to assess the concept of death and its four components (universality, irreversibility, non-functionality and causality) among children.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/2227-906 7/8/2/125/s1, Table S1: Escala sobre el Concepto de Muerte (EsCoMu).

**Author Contributions:** Conceptualization, M.F.-A., M.N.P.-M., F.C.-Q. and R.M.-J.; Data curation, M.d.l.S.-R., F.C.-Q., J.M.V.-S. and R.M.-J.; Formal analysis, M.F.-A., M.d.l.S.-R. and J.M.V.-S.; Investigation, M.N.P.-M. and R.M.-J.; Methodology, M.F.-A., M.d.l.S.-R., M.N.P.-M. and R.M.-J.; Software, M.d.l.S.-R.; Supervision, M.N.P.-M. and F.C.-Q.; Writing—original draft, M.F.-A.; Writing—review and editing, M.d.l.S.-R., M.N.P.-M., F.C.-Q., J.M.V.-S. and R.M.-J. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the University of Granada's Ethics Committee on Human Research (Ref. 1056/CEIH/2020).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Data will be available upon request to the corresponding author.

**Acknowledgments:** We thank the Alma y Vida Association in Chiclana (Spain) for their help in the data collection.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


## *Article* **An Autoencoder-Based Deep Learning Classifier for E**ffi**cient Diagnosis of Autism**

### **Harshini Sewani and Rasha Kashef \***

Department of Electrical, Computer, and Biomedical Engineering, Ryerson University, Toronto, ON M5B 2K3, Canada; harshini.sewani@ryerson.ca

**\*** Correspondence: rkashef@ryerson.ca

Received: 25 September 2020; Accepted: 8 October 2020; Published: 14 October 2020

**Abstract:** Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by a lack of social communication and social interaction. Autism is a mental disorder investigated by social and computational intelligence scientists utilizing advanced technologies such as machine learning models to enhance clinicians' ability to provide robust diagnosis and prognosis of autism. However, with dynamic changes in autism behaviour patterns, these models' quality and accuracy have become a great challenge for clinical practitioners. We applied a deep neural network learning on a large brain image dataset obtained from ABIDE (autism brain imaging data exchange) to provide an efficient diagnosis of ASD, especially for children. Our deep learning model combines unsupervised neural network learning, an autoencoder, and supervised deep learning using convolutional neural networks. Our proposed algorithm outperforms individual-based classifiers measured by various validations and assessment measures. Experimental results indicate that the autoencoder combined with the convolution neural networks provides the best performance by achieving 84.05% accuracy and Area under the Curve (AUC) value of 0.78.

**Keywords:** autism; diagnosis; autoencoder; convolution neural network; machine learning

### **1. Introduction**

Autism spectrum disorder (ASD) is one of the brain development disorders. According to the World Health Organization (WHO), 1 in 60 children has an autism spectrum disorder [1]. Difficulties in social communication and interaction characterize this disorder; individuals on the spectrum also tend to have restricted interest and repetitive behaviour. ASD is an intellectual disability; many of those on the autism spectrum have extraordinary abilities and skills. Roughly 40% are intellectually above average and have a unique ability to see the world with pride from a different perspective. According to the National Autism Spectrum Disorder Surveillance System (NASS), the most up-to-date Canadian prevalence rate is: 1 in 66 Canadian children and youth (ages 5–17) were diagnosed with autism spectrum disorder [2]. According to the National Institute of Mental Health (NIHM) [3], scientists do not know how and what causes autism, but some research suggests that genes and environmental factors cause autism. Some of the potential risk factors associated with ASD include having a sibling with ASD, older parents, pre-existing genetic conditions like Down syndrome, fragile X syndrome, and Rett syndrome, and finally, a low birth weight. Additionally, it is worth mentioning that researchers found differences in the brains of babies born before 27 weeks [4], i.e., babies born very prematurely are at higher risk for developing ASD. Early diagnosis in the first few years of life significantly improves results for people on the autism spectrum, but there are often delays in recognizing and diagnosing ASD. If health systems were better capable of identifying children at high risk for ASD and bringing them in earlier for a comprehensive evaluation, more children could benefit from early intervention. Many machine learning and neural network methods have recently shown an improvement in autism

diagnosis [4–7]. The classification accuracy and the required computational time for building an automatic diagnosis system are significant challenges for many classifiers. In the paper, we focus on classifying individuals who have ASD from typically developing controls subjects using functional magnetic resonance imaging (fMRI) images provided by the autism brain imaging data exchange (ABIDE) [8] to study brain activities. There are 17 different brain imaging centers from where these images are collected. The dataset contains 539 individuals who have ASD and 573 typical controls (TC). ABIDE is preprocessed by four different pipelines like the connectome computation system (CCS), the configurable pipeline for the analysis of connectomes (CPAC), the data processing assistant for resting-state fMRI (DPARSF), and the neuroimaging analysis kit. We used the CPAC pipeline with preprocessing steps, including slice time correction, motion correction, nuisance signal remover, low-frequency drift, and voxel intensity normalization. We propose a hybrid classifier that combines an autoencoder with some well-known supervised machine learning algorithms and deep learning methods. For the deep learning methods, we focused on the convolution neural network (CNN). The autoencoder, combined with CNN, has shown a maximum accuracy of 83.39%. This paper is structured as follows: In Section 2, a literature survey is provided. Section 3 discusses the algorithms used from the machine learning and the neural network literature and essential parameters in each algorithm. In Section 4, we introduced the autoencoder-based hybrid diagnosis model. Section 5 provides details on the performance of the individual and hybrid algorithms using rs-fMRI experimental datasets. In Section 6, we concluded the paper along with some future directions.

### **2. Literature Review**

Various machine learning and deep neural network methods distinguish between ASD and non-ASD. These methods are categorized into three main categories as image-based, questionnaire-based, or behavioural-based.

### *2.1. Image-Based Classification*

Several studies have employed deep neural network learning on large brain image datasets obtained from the autism brain imaging data exchange [8] to identify ASD individuals from the typical control (TC). In [4], the authors used the deep learning model to perform binary classification of ASD and the typical control based on their neural patterns using rs-fMRI data. There were 505 ASD participants and 530 typically developing controls from 17 different imaging sites. They opted for using two stacked autoencoders to extracted lower-dimensional features. The model achieved 70% accuracy, a sensitivity of 74%, and a specificity of 63%, which is better than support vector machines (SVMs) and random forest (RF) models used in a previous study; these models used 10-fold cross-validation. Due to the high probability of noise in the image dataset obtained from several sites, the model does not achieve promising results despite using a dedicated Graphical Processing Unit (GPU) to speed up the training time. The entire model took about 33 h for training. Another model that diagnosed ADHD and autism from 3-D structural MRI and 4-D (fMRI) is presented in [9]. There are three learners in their work, including texture-based filters obtained using the sparse encoders that extracts features from MRI datasets. The fMRI scans are used to compute spatial non-stationary independent components, which decompose the subject's scans into the sequence order in time from the obtained component. The multimodal features obtained from the learner serves as input to the SVM model. The accuracy of the ADHD-200 dataset is 67.3%, and the accuracy of the ABIDE dataset is 64.3%. These learning models have found a "Signal" in the data to identify differences between case and control. The current results are not yet clinically applicable. In [10], authors have used 6 personal characteristic data (PCD) of 851 subjects (421 ASD and 430 non-ASD) from the ABIDE database to predict autism. The authors have evaluated the performance of nine machine learning models. Comparing to the other eight models, the neural network-based stacked sparse encoder performance is better with an area under the curve of 0.646. The advantage of their work includes understanding the predictive power of PCD for ASD classifications. The limitation of their work was

that the dataset is from 17 different clinical and research sites; this fact leads to heterogeneity in the data and might underfit the models' accuracy; the database's size is another issue. In [5], a hybrid approach is introduced to detect autism using fMRI data [5]. In their work, they have proposed the ASD-DiagNet framework. Their experimental dataset is preprocessed using the C-PAC pipeline and parcellated into 200 functionally regions. They have used data augmentation techniques using the synthetic minority over-sampling technique (SMOTE). SMOTE uses the nearest neighbour method to generate augmented data. The process is performed using two phases; the first phase evaluates the model's performance on the whole dataset to find functional connectivity between regions, which is acquired using Pearson's correlation. The results achieved an accuracy of 69.4% on the original dataset, whereas data generated using augmentation shows some improvement, i.e., 70.3% accuracy. The second phase of their work includes 5-fold cross-validation on every 17 sites separately. The average result of all sites with the ASD-DiagNet model without augmented data is 60.7% accuracy, and 63.8% with augmented data. Another experiment was performed on automated anatomical labelling (AAL) and Talairach and Tournoux (TT). For AAL, 67.5% is the highest accuracy obtained (using augmented data), whereas TT achieved an accuracy of 65.3% (using augmented data). The convolutional neural network has been applied to detect the autism spectrum disorder using the ABIDE dataset [11] and has achieved 70.22% accuracy using fewer parameters. The authors pointed out four essential regions for ASD classification: C115, C188, C247, and C326 for the CC400 functional parcellation. The performance of SVM, K-nearest neighbour (KNN), and the random forest was evaluated, and achieved accuracy 69%, 62%, and 60%, respectively, after hyperparameter tuning. Their work has used fewer parameters, which helped in the reduction of computational cost.

### *2.2. Questionnaire-Based Classification Methods*

In [12], a research study had attempted to detect ASD and ADHD using a crowdsource recruitment procedure. The survey database contains 248 (ASD) and ADHD (174) subjects, aged 2–17. The second dataset collected from the previous study is called archival, including 2775 ASD and 150 ADHD. Five machine learning models, ENet, Lasso, SVM [13], linear discriminant analysis (LDA) [14], and Ridge, were used. These models are applied to three independent datasets and combinations of the archival and survey datasets. All five models achieved Area under the Curve (AUC) that exceeds 0.90 when the archival dataset was used as training. When considering the survey sample as training, the ENet and LDA models work well. By using the mixture of two datasets, they have achieved an AUC equal to 0.89. The disadvantages are that the data obtained in archival ADHD is composed of siblings of children with autism. This data is biased, and therefore, results are compromised. In [15], authors have used two algorithms to train a structure parent-reported questionnaires and significant behaviours from short videos of children [15]. The dataset is obtained from multiple repositories of Autism Diagnostic Observation Schedule (ADOS) and Autism Diagnostic Interview Revised (ADI-R) score-sheets of children between 18 and 84 months. The random forest (RF) is trained over the ADI-R instrument data on a parent questionnaire (2299 with ASD, 585 with TC, and 364 with other conditions). The video of the subject (i.e., a 1-min home video taken by parents) was required to evaluate the target label's presence. The responses of all the questionnaires and short clips collaborated using L2-regularized logistic regression. The Receiver Operating Characteristics (ROC) of the combined data shows a boost in the performance of the clinical study samples. The advantage is that the ROC curve outperforms when compared with tools like Modified Checklist for Autism in Toddlers (M-CHAT) and Child Behaviour Checklist (CBCL). Additionally, allowing some subjects with lower certainty output from the algorithms to be classified as inconclusive. Their work utilizes experts who have used ADI-R and ADOS tools. These tools consumed hours to evaluate the results since parents were conducting it without experts and not more than a minute to complete the test, which caused significant data degradation and adding bias with an expected loss of screening accuracy. The authors have utilized machine learning algorithms for detecting autism in the below-explained study [16]. They have used three ASD datasets available from the University of California Irvine (UCI) repository

(https://archive.ics.uci.edu). The dataset has 20 attributes, of which 1–10 are screening questions, and the remaining 10 attributes are personal information. The dataset has missing values, which were handled during the preprocessing steps before applying machine learning models. They have evaluated SVM, random forest, and KNN by splitting the dataset into five different sets. The results show that the random forest has performed very well for the classification compared to SVM and KNN. The authors recommend in their study to use a large data set and fewer missing values. Authors have worked on the UCI dataset [17] to predict autism using machine learning and neural network models [18]. The accuracy of the CNN and SVM models on the adult dataset exceeds 98%, while CNN's accuracy on the youth data set exceeds 96%. The CNN, Artificial Neural Networks (ANN), logistic regression, and SVM achieve accuracy above 98% on the children dataset.

### *2.3. Behavioural-Based Classification Methods*

In [19], the authors identified a few behavioural measures that were enough to differentiate ASD from ADHD. The data set is from the Boston Autism Association and Autism Genetic Resources Exchange of 2775 autistic individuals and 150 ADHD subjects. Using Social Responsiveness Scale (SRS) items containing responses of a child's behaviour, six machine learning models like SVM, LDA, categorical Lasso, and logistic regression, random forest, and decision tree was trained and tested. Minimal redundancy, maximal relevance, undersampling, forward feature selection, and 10-fold cross-validation applied to the dataset before training. The results show out of six algorithms, four algorithms, including SVM, LDA, categorical Lasso, and logistic regression, achieved an accuracy ranging from 96.2% to 96.5%. Each model used only 5 of the 65 behaviour indicators. The models mentioned were able to do classification tasks optimally, not only due to less error but probabilistic qualities. The disadvantages of their work include the massive imbalance between subjects of ASD and ADHD. This subject imbalance was overcome with undersampling but prevented authors from devoting a part of data exclusively for validation because of the constrained sample data. In [20], authors have designed a normalization layer and activation layer into a single tensor to tensor computation graph, which forms high sparse and large search space. The normalization layer and activation function are essential components of deep learning that stabilize optimization and improved generalization. The authors have shown an experiment of Evo Norms (normalization-activation layer) on image classification models such as Res Nets, Mobile Nets, and Efficient Nets. Evo Norms consist of two series: (1) the B-series, which is batch dependent, and (2) the S-series, which works on independent samples introduced by rejecting any batch dependent operation. Their method discovered novel layers with structures that achieved strong generalization across many architectures and tasks.

### **3. Machine Learning and Deep Learning Classifiers**

In our proposed hybrid model, we combined an autoencoder for dimensionality reduction with one of the well-known classifiers, including support vector machines (SVMs), random forest (RF), K-nearest neighbours (KNNs), and convolutional neural network (CNN). This section introduces each of the adopted methods in our hybrid model.

### *3.1. Autoencoder*

The autoencoder is a type of neural network that does not require the labeling of data, and therefore it is an unsupervised learning algorithm. The aim is to learn an input function to reconstruct the input to an output of fewer dimensions. It approximates the identity function to get the outcome of a neural network similar to the input. In order words, it tries to copy the input to its output. Mathematically, if *x* is the input (also called an encoder), *x*' is the network (also called a decoder). The architecture of autoencoders reduces dimensionality using non-linear optimization. In Equation (1), *h* is the hidden

layer, which can be calculated by multiplying the vector *x* with weights and adding the bias and passing it to the activation function. The decoder *x*' is calculated in Equation (2).

$$h = \sigma(\mathcal{W} \times \mathfrak{x} + b) \tag{1}$$

$$\mathbf{x}' = (\mathbf{W} \times \mathbf{h} + \mathbf{b}) \tag{2}$$

There are many autoencoder variants, such as the undercomplete autoencoder, denoising autoencoder, sparse autoencoder, and adversarial autoencoder. A research study in [21] has demonstrated that when the input has a relation or a basic structure, the input reconstruction will be painless. They have described several applications that involve the use of the autoencoder algorithm, one of which is brain disorders. Whenever clinical neuroimaging studies are considered, they always opt for an autoencoder before feeding data to any other classification model because of the high dimensionality of genetic and neuroimaging data. The difficulty lies in constructing and learning output from a high dimensional input. In [22], the authors have evaluated the performance by comparing an autoencoder with other models that can do a similar task of reduction, for example, models like principal component analysis, linear discriminant analysis (linear models), locally linear embedding, and Isomap (non-linear models). They have applied their model on various datasets and concluded that the number of hidden layer nodes affects the autoencoder's performance. When the hidden nodes adjusted around the dataset's inherent dimensionality, the Modified National Institute of Standards and Technology (MNIST) data set performs well. Other experiments in their paper involved a synthetic dataset and Olivetti dataset. Experimental analysis on the synthetic dataset showed stable performance by principal component analysis (PCA), linear discriminant analysis (LDA), and Isomap. However, an autoencoder and locally linear embedding (LLE) were not stable. The unstable models retain the original data's geometric characteristics, so the result is still acceptable. Experiments on the MNIST dataset showed that the autoencoder performed better than PCA. The autoencoder projected points to represent the digit "1". An autoencoder tends to project images of the same class to edges and corners. The results showed that the autoencoder not only helps in dimensionality reduction but also in detecting the recurrent structures. The lower-dimensional data generated during the encoding process contains useful patterns from the original input, is provided to a convolutional neural network or a machine learning algorithm [23–25].

### *3.2. Support Vector Machines (SVMs)*

Support vector machines (SVMs) are a supervised machine learning model applied in regression and classification tasks. The SVM aims to find a hyperplane in an N-dimensional space that classifies the data points. Let us consider an example of two classes. Figure 1 shows multiple hyperplanes (Y1, Y2, and Y3) that divide these data points into their respective classes. We had to choose a hyperplane that assures data points are on the right side of the graph. From Figure 1, we could conclude that Y2 segregated data points more efficiently. Two points were closest to the two lines (Y1 and Y3) and had less margin. These points (pink filled shapes) were a little further from hyper-plane Y2. Hence, we set Y2 as the right hyperplane with a high margin.

In this paper, we perform binary classification such that the SVM predicts the binary outcome on brain image data. The knowledge of the SVM is incomplete without understanding kernels. Kernel functions play a vital role in SVM. They transform inputs space into feature space in any required form. Kernels provide shortcuts to avoid heavy calculations. The significant fact about the kernel is that we can go to higher dimensions and perform effortless calculations. With the kernels, we can go up to an infinite number of dimensions using kernels. There are various types of kernels; some of these kernels are discussed as follows: [26–29].

*3.2. Support Vector Machines (SVMs)*

set Y2 as the right hyperplane with a high margin.

input, is provided to a convolutional neural network or a machine learning algorithm [23–25].

Support vector machines (SVMs) are a supervised machine learning model applied in regression and classification tasks. The SVM aims to find a hyperplane in an N-dimensional space that classifies the data points. Let us consider an example of two classes. Figure 1 shows multiple hyperplanes (Y1, Y2, and Y3) that divide these data points into their respective classes. We had to choose a hyperplane that assures data points are on the right side of the graph. From Figure 1, we could conclude that Y2 segregated data points more efficiently. Two points were closest to the two lines (Y1 and Y3) and had

There are many autoencoder variants, such as the undercomplete autoencoder, denoising autoencoder, sparse autoencoder, and adversarial autoencoder. A research study in [21] has demonstrated that when the input has a relation or a basic structure, the input reconstruction will be painless. They have described several applications that involve the use of the autoencoder algorithm, one of which is brain disorders. Whenever clinical neuroimaging studies are considered, they always opt for an autoencoder before feeding data to any other classification model because of the high dimensionality of genetic and neuroimaging data. The difficulty lies in constructing and learning output from a high dimensional input. In [22], the authors have evaluated the performance by comparing an autoencoder with other models that can do a similar task of reduction, for example, models like principal component analysis, linear discriminant analysis (linear models), locally linear embedding, and Isomap (non-linear models). They have applied their model on various datasets and concluded that the number of hidden layer nodes affects the autoencoder's performance. When the hidden nodes adjusted around the dataset's inherent dimensionality, the Modified National Institute of Standards and Technology (MNIST) data set performs well. Other experiments in their paper involved a synthetic dataset and Olivetti dataset. Experimental analysis on the synthetic dataset showed stable performance by principal component analysis (PCA), linear discriminant analysis (LDA), and Isomap. However, an autoencoder and locally linear embedding (LLE) were not stable. The unstable models retain the original data's geometric characteristics, so the result is still acceptable. Experiments on the MNIST dataset showed that the autoencoder performed better than PCA. The autoencoder projected points to represent the digit "1". An autoencoder tends to project images of the same class to edges and corners. The results showed that the autoencoder not only helps in dimensionality reduction but also in detecting the recurrent structures. The lowerdimensional data generated during the encoding process contains useful patterns from the original

′ = ( × ℎ + ) (2)

**Figure 1.** Hyperplane of the support vector machine (SVM). **Figure 1.** Hyperplane of the support vector machine (SVM). bipolar sigmoid function is used as an activation function for the neurons.

perceptron kernel. The sigmoid kernel is obtained from the neural network field, where the

In this paper, we perform binary classification such that the SVM predicts the binary outcome on brain image data. The knowledge of the SVM is incomplete without understanding kernels. Kernel • Linear: This form of kernel function is very simple, straightforward. It is given by the inner product of (*x*,*y*) plus an optional constant bias, as shown in Equation (3): (, ) =∝ × tanh( + ) (4) Radial basis function (RBF): is used when we have no prior knowledge of data.

$$k(\mathbf{x}, \mathbf{y}) = \mathbf{x}^T \mathbf{y} + \text{bias} \tag{3}$$

• Sigmoid: The sigmoid kernel is also called hyperbolic tangent kernel and as a multilayer perceptron kernel. The sigmoid kernel is obtained from the neural network field, where the bipolar sigmoid function is used as an activation function for the neurons. The SVM is an effective classifier when the number of dimensions is greater than the number of data samples. SVM has high-efficiency memory functions and has various forms of kernels [13].

2 2

$$k(\mathbf{x}, y) \; = \text{ or } \times \tan \mathbf{h} (\mathbf{x}^T y + \text{bias}) \tag{4}$$

• Radial basis function (RBF): is used when we have no prior knowledge of data. trees is that they are not flexible enough to classify new sets of data points. Random forest combines the simplicity of a decision tree with flexible results and improves performance metrics. The

Step 2: Consider a random subset of variables at each step.

$$k(x, y) = \exp\left(\frac{\left\|x - y\right\|^2}{2\sigma^2}\right) \tag{5}$$

The SVM is an effective classifier when the number of dimensions is greater than the number of data samples. SVM has high-efficiency memory functions and has various forms of kernels [13]. Step 3: Perform a vote for a new dataset by sending it to all the trees. Step 4: Select the prediction result with the highest votes as the final prediction.

#### *3.3. Random Forest* The bootstrap dataset does not select all columns to determine the root of the tree. Step 1 mentioned above is performed again until varieties of trees are built. The idea of creating multiple

*3.3. Random Forest*

An ensemble of decision trees forms the random forest (RF). The disadvantage of using decision trees is that they are not flexible enough to classify new sets of data points. Random forest combines the simplicity of a decision tree with flexible results and improves performance metrics. The following steps are required to build a random forest tree, as in Figure 2. trees leads to an efficient performance of the random forest. Assume a new input dataset (also called the out of bag dataset) is added to all the bootstrap datasets, the decision from each bootstrap is recorded. The aggregated decision is the decision for the new input. This type of decision-making using a bootstrap dataset is called bagging [28,29].

**Figure 2.** The random forest algorithm. accurate and robust method because of the number of decision trees constructed to provide a **Figure 2.** The random forest algorithm.

The algorithm is used for binary classification and regression problems. RF is considered an


The bootstrap dataset does not select all columns to determine the root of the tree. Step 1 mentioned above is performed again until varieties of trees are built. The idea of creating multiple trees leads to an efficient performance of the random forest. Assume a new input dataset (also called the out of bag dataset) is added to all the bootstrap datasets, the decision from each bootstrap is recorded. The aggregated decision is the decision for the new input. This type of decision-making using a bootstrap dataset is called bagging [28,29].

The algorithm is used for binary classification and regression problems. RF is considered an accurate and robust method because of the number of decision trees constructed to provide a prediction for the new dataset. RF does not suffer from the overfitting problem. The random forest is a time-consuming algorithm because decisions are made from many trees [30–32]. *Children* **2020**, *7*, x FOR PEER REVIEW 7 of 17 prediction for the new dataset. RF does not suffer from the overfitting problem. The random forest is a time-consuming algorithm because decisions are made from many trees [30–32].

#### *3.4. K-Nearest Neighbours 3.4. K-Nearest Neighbours*

The K-nearest neighbours (KNN) is a simple supervised, non-parametric algorithm used for classification and regression problems. For a data point x to be classified, its K-nearest neighbours are retrieved, as a neighbourhood of x. Voting among the neighbourhood data is usually used to decide the classification for the point x. The critical point is to select the value for the *k*. If the value of *k* is not appropriate, there is a low chance of obtaining promising results for any dataset. The algorithm becomes slow when we increase the number of k values [30,31]. The K-nearest neighbours (KNN) is a simple supervised, non-parametric algorithm used for classification and regression problems. For a data point x to be classified, its K-nearest neighbours are retrieved, as a neighbourhood of x. Voting among the neighbourhood data is usually used to decide the classification for the point x. The critical point is to select the value for the *k*. If the value of *k* is not appropriate, there is a low chance of obtaining promising results for any dataset. The algorithm becomes slow when we increase the number of k values [30,31].

#### *3.5. Convolutional Neural Network (CNN) 3.5. Convolutional Neural Network (CNN)*

CNN is a deep learning algorithm that is very helpful for image classification. The CNN model on image datasets generally takes an image as the input, and the output is the likelihood of the class to which it belongs. The critical layers in the CNN are the convolution layer and the max pooling layers. The input image is given to the convolution layer, and it applies different kernels/filters that extract the low-level features. The following are the fundamental building blocks on CNN [33]. CNN is a deep learning algorithm that is very helpful for image classification. The CNN model on image datasets generally takes an image as the input, and the output is the likelihood of the class to which it belongs. The critical layers in the CNN are the convolution layer and the max pooling layers. The input image is given to the convolution layer, and it applies different kernels/filters that extract the low-level features. The following are the fundamental building blocks on CNN [33].

#### 3.5.1. Stride 3.5.1. Stride

of size 2 × 2.

3.5.2. Padding

It is the number of picture elements the kernel/filter shifts over the input at a time. Figure 3 shows an example of how the stride process works. In the example, the filter convolved around the input by shifting two units at a time. To calculate the first block of the resulting image after a stride operation with stride = 2 was performed. We applied a cross product between the image pixel and the filter pixel as (0 × 0) + (1 × 2) + (2 × 1) + (3 × 2) = 10; such that the first number (in orange color) was the image pixel and second number (in blue color) was the filter pixel. It is the number of picture elements the kernel/filter shifts over the input at a time. Figure 3 shows an example of how the stride process works. In the example, the filter convolved around the input by shifting two units at a time. To calculate the first block of the resulting image after a stride operation with stride = 2 was performed. We applied a cross product between the image pixel and the filter pixel as (0 × 0) + (1 × 2) + (2 × 1) + (3 × 2) = 10; such that the first number (in orange color) was the image pixel and second number (in blue color) was the filter pixel.

**Figure 3.** A Stride Example**. Figure 3.** A Stride Example.

The image was of size 5 × 5 with a filter of size 2 × 2, and a stride value equal to 2. The size of the reduced image was calculated using Equation (6): where *nin* (in our case, 5) is the number of input

 + 2 − 

The convolution layer without padding does not preserve the spatial size of the input image. When the input is given to the convolution layer with padding, we add zeros to the border of input, which helps in extracting the features from the corner of the image. The input image dimensions in Figure 4 are 6 × 6 × 1, where 6 × 6 is the image size, and the number of channels is 1, which means the


= |

The image was of size 5 × 5 with a filter of size 2 × 2, and a stride value equal to 2. The size of the reduced image was calculated using Equation (6): where *nin* (in our case, 5) is the number of input features, *nout* is the number of output features, *k* is kernel size (in our case, 2), *p* is the convolution padding size (in our case, 0), and *s* is convolution stride size (in our case, 2). The resulting image was of size 2 × 2.

$$n\_{out} = \left| \frac{n\_{in} + 2p - k}{s} \right| + 1 \tag{6}$$

3.5.2. Padding

The convolution layer without padding does not preserve the spatial size of the input image. When the input is given to the convolution layer with padding, we add zeros to the border of input, which helps in extracting the features from the corner of the image. The input image dimensions in Figure 4 are 6 × 6 × 1, where 6 × 6 is the image size, and the number of channels is 1, which means the image has only one channel. If we set the padding size = 2, we get the image size of image 8 × 8 × 1. Figure 4 shows the original image and the matrix after applying the padding process. *Children* **2020**, *7*, x FOR PEER REVIEW 8 of 17 image has only one channel. If we set the padding size = 2, we get the image size of image 8 × 8 × 1. Figure 4 shows the original image and the matrix after applying the padding process. *Children* **2020**, *7*, x FOR PEER REVIEW 8 of 17 image has only one channel. If we set the padding size = 2, we get the image size of image 8 × 8 × 1. Figure 4 shows the original image and the matrix after applying the padding process.

**Figure 4.** The Padding Example. **Figure 4.**The Padding Example.

#### 3.5.3. Max Pooling 3.5.3. Max Pooling 3.5.3. Max Pooling The max pooling reduces the size of the feature maps. The advantage of using this layer is to

3.5.4. Activation Function

3.5.4. Activation Function

block.

The max pooling reduces the size of the feature maps. The advantage of using this layer is to reduce the computational power. Although the size of the image is reduced, the vital information is still maintained. There are different types of pooling, which are as follows max pooling and average pooling. Max pooling chooses the highest/topmost value from the image covered by the kernel. In contrast, average pooling calculates the midpoint/mean value from the portion of an image covered by the filter. An example is shown in Figure 5, where the input image is 4 × 4; if we apply stride = 2 and perform max pooling, the results are obtained in each block as the max value out of the selected The max pooling reduces the size of the feature maps. The advantage of using this layer is to reduce the computational power. Although the size of the image is reduced, the vital information is still maintained. There are different types of pooling, which are as follows max pooling and average pooling. Max pooling chooses the highest/topmost value from the image covered by the kernel. In contrast, average pooling calculates the midpoint/mean value from the portion of an image covered by the filter. An example is shown in Figure 5, where the input image is 4 × 4; if we apply stride = 2 and perform max pooling, the results are obtained in each block as the max value out of the selected block. reduce the computational power. Although the size of the image is reduced, the vital information is still maintained. There are different types of pooling, which are as follows max pooling and average pooling. Max pooling chooses the highest/topmost value from the image covered by the kernel. In contrast, average pooling calculates the midpoint/mean value from the portion of an image covered by the filter. An example is shown in Figure 5, where the input image is 4 × 4; if we apply stride = 2 and perform max pooling, the results are obtained in each block as the max value out of the selected block.

**Figure 5.** Max pooling. **Figure 5.** Max pooling. **Figure 5.** Max pooling.

layer, activation, and max pooling layer) and the output layer. The activation function choice will help us perform classification or regression tasks when throwing the model's output. After the hidden layer, the activation function is invoked to learn a non-linear form of linear mapping before making any predictions. Some activation functions are Rectilinear, sigmoid, SoftMax, and tanh [28]. Sigmoid function: The sigmoid function exists between 0 and 1, and its shape looks like an S shape. Sigmoid is the correct choice when we have to predict the likelihood of a model. Equation (7) illustrates the sigmoid function. Since the sigmoid function is differentiable, the sigmoid

help us perform classification or regression tasks when throwing the model's output. After the hidden layer, the activation function is invoked to learn a non-linear form of linear mapping before making any predictions. Some activation functions are Rectilinear, sigmoid, SoftMax, and tanh [28]. Sigmoid function: The sigmoid function exists between 0 and 1, and its shape looks like an S shape. Sigmoid is the correct choice when we have to predict the likelihood of a model. Equation (7) illustrates the sigmoid function. Since the sigmoid function is differentiable, the sigmoid

function's derivative is shown in Equation (8) to calculate the slope of the sigmoid curve.

function's derivative is shown in Equation (8) to calculate the slope of the sigmoid curve.

The activation function is also called a transfer function, computes a weighted sum of the input

The activation function is also called a transfer function, computes a weighted sum of the input and biases. The activation function decides if the weights and bias' values will activate/fire the

### 3.5.4. Activation Function

The activation function is also called a transfer function, computes a weighted sum of the input and biases. The activation function decides if the weights and bias' values will activate/fire the neuron. The activation function aims to convert a linear input signal of the model into non-linear output signals. The simple CNN model consists of an input layer hidden layer (having a convolution layer, activation, and max pooling layer) and the output layer. The activation function choice will help us perform classification or regression tasks when throwing the model's output. After the hidden layer, the activation function is invoked to learn a non-linear form of linear mapping before making any predictions. Some activation functions are Rectilinear, sigmoid, SoftMax, and tanh [28].

• Sigmoid function: The sigmoid function exists between 0 and 1, and its shape looks like an S shape. Sigmoid is the correct choice when we have to predict the likelihood of a model. Equation (7) illustrates the sigmoid function. Since the sigmoid function is differentiable, the sigmoid function's derivative is shown in Equation (8) to calculate the slope of the sigmoid curve.

$$\sigma(\mathbf{x}) = \frac{1}{1 + e^{-z}} \tag{7}$$

$$
\sigma' = \sigma(\mathfrak{x})(1 - \sigma(\mathfrak{x})) \tag{8}
$$

• Rectilinear function: The Rectilinear function, also called ReLU. It has values between 0 and infinity, and it provides better performance than the sigmoid function. Equation (9) shows the derivative of the ReLU function.

$$R(z) = \begin{cases} 1 & \text{if } z > 0\\ 0 & \text{if } z < 0 \end{cases} \tag{9}$$

In one of the proposed models, we used the sigmoid activation function since our problem evolved to predict whether the candidate has autism or not, making it a binary classification.

### **4. The Proposed Hybrid Autoencoder-Based Classifier**

Autoencoders play a vital role in extracting low dimensional features, and these features can be given to machine learning models or deep learning models to perform classification tasks. The architecture of autoencoders reduces dimensionality using non-linear optimization. Our proposed method focused on using the undercomplete autoencoder to extract useful information from the input layer by having fewer neurons in the hidden layer than the input. The architecture of an undercomplete autoencoder is shown in Figure 6. It is the simplest form of constructing an autoencoder by limiting the amount of information that can flow through the network. This can be achieved by reducing the number of neurons in the hidden layer. This helps to obtain essential features from the data. By penalizing the structure according to the reconstruction error, our architecture learns the most important input data attributes and show how to reconstruct the original input from an "encoded" state.

The input fMRI was fed to the Pearson's correlation function to compute the pairwise correlation. This data was provided as input to an autoencoder, which will help extract lower-dimensional features (in our case, 4975 features from 9950) to send it to the classification models. In the paper, we focused on using four main classifiers, including KNN, SVM, RF, and CNN. Figure 7 shows the flowchart of combing the autoencoder and the adopted machine learning classifiers. In the autoencoder–KNN model, for calculating the distance between *k* points, the value of *p* was set to 2, which was for the Euclidean distance and 100 neighbours. For the autoencoder–SVM model, in the parameter list, we chose C = 2.0, which is a regularization parameter. For the autoencoder–RF model, we assigned the estimators value to 1050 and the depth value to 90.

In the proposed autoencoder–CNN, the CNN consisted of two 1D convolution layer followed by batch normalization, an activation function, and a max pooling layer. After flattening the CNN

CNN.

random forest.

**5. Experiment Results**

**5. Experiment Results**

derivative of the ReLU function.

**4. The Proposed Hybrid Autoencoder-Based Classifier**

layer, it worked as a fully connected neural network, and we had two linear layers with an activation function. Since the output predicts whether an individual is diagnosed with autism or not, we applied the sigmoid activation function. Figure 8 shows the workflow of combining an autoencoder with CNN. achieved by reducing the number of neurons in the hidden layer. This helps to obtain essential features from the data. By penalizing the structure according to the reconstruction error, our architecture learns the most important input data attributes and show how to reconstruct the original input from an "encoded" state.

autoencoder by limiting the amount of information that can flow through the network. This can be

*Children* **2020**, *7*, x FOR PEER REVIEW 9 of 17

 Rectilinear function: The Rectilinear function, also called ReLU. It has values between 0 and infinity, and it provides better performance than the sigmoid function. Equation (9) shows the

1 1 + −

1 > 0 0 < 0

In one of the proposed models, we used the sigmoid activation function since our problem

Autoencoders play a vital role in extracting low dimensional features, and these features can be given to machine learning models or deep learning models to perform classification tasks. The architecture of autoencoders reduces dimensionality using non-linear optimization. Our proposed method focused on using the undercomplete autoencoder to extract useful information from the input layer by having fewer neurons in the hidden layer than the input. The architecture of an

′ = ()(1 − ()) (8)

} (9)

(7)

() =

() = {

evolved to predict whether the candidate has autism or not, making it a binary classification.

**Figure 6.** The autoencoder architecture. **Figure 6.** The autoencoder architecture. *Children* **2020**, *7*, x FOR PEER REVIEW 10 of 17

**Figure 7.** The hybrid autoencoder-based machine learning classifier. KNN: K-nearest neighbour; RF: random forest. **Figure 7.** The hybrid autoencoder-based machine learning classifier. KNN: K-nearest neighbour;RF: random forest. function. Since the output predicts whether an individual is diagnosed with autism or not, we applied the sigmoid activation function. Figure 8 shows the workflow of combining an autoencoder with


**Figure 8.** The hybrid model of an autoencoder and convolutional neural network (CNN). **Figure 8.** The hybrid model of an autoencoder and convolutional neural network (CNN).

**Figure 8.** The hybrid model of an autoencoder and convolutional neural network (CNN).

information. This data is slice time corrected, motion-corrected, and normalized. In our study, all rsfMRI data were from the CPAC preprocessing pipeline and band-pass filtered (0.01–0.1 Hz). From these 1112 subjects, 1035 subjects were considered for our study since only these subjects had completed phenotypic information. To limit the variance between outputs to just preprocessing, statistical derivatives for each pipeline and strategy were calculated using the CPAC software [8]. The rs-fMRI (shown in Figure 9) stands for resting-state fMRI, a type of functional magnetic resonance imaging (fMRI) used in brain mapping to evaluate regional interactions in a resting or task-negative state when an explicit task is not performed. The resting-state process helps explore the

The Google Colaboratory (Colab) was used to perform experiments, a free online cloud-based Jupyter notebook that allowed us to train our machine learning and deep learning models on CPUs, GPUs, and TPUs. The ABIDE-I dataset has rs-fMRI data for 1112 candidates, along with phenotypic information. This data is slice time corrected, motion-corrected, and normalized. In our study, all rsfMRI data were from the CPAC preprocessing pipeline and band-pass filtered (0.01–0.1 Hz). From these 1112 subjects, 1035 subjects were considered for our study since only these subjects had completed phenotypic information. To limit the variance between outputs to just preprocessing, statistical derivatives for each pipeline and strategy were calculated using the CPAC software [8]. The rs-fMRI (shown in Figure 9) stands for resting-state fMRI, a type of functional magnetic resonance imaging (fMRI) used in brain mapping to evaluate regional interactions in a resting or task-negative state when an explicit task is not performed. The resting-state process helps explore the

### **5. Experiment Results**

The Google Colaboratory (Colab) was used to perform experiments, a free online cloud-based Jupyter notebook that allowed us to train our machine learning and deep learning models on CPUs, GPUs, and TPUs. The ABIDE-I dataset has rs-fMRI data for 1112 candidates, along with phenotypic information. This data is slice time corrected, motion-corrected, and normalized. In our study, all rs-fMRI data were from the CPAC preprocessing pipeline and band-pass filtered (0.01–0.1 Hz). From these 1112 subjects, 1035 subjects were considered for our study since only these subjects had completed phenotypic information. To limit the variance between outputs to just preprocessing, statistical derivatives for each pipeline and strategy were calculated using the CPAC software [8]. The rs-fMRI (shown in Figure 9) stands for resting-state fMRI, a type of functional magnetic resonance imaging (fMRI) used in brain mapping to evaluate regional interactions in a resting or task-negative state when an explicit task is not performed. The resting-state process helps explore the brain's functional organization and examine if altered in neurological or mental disorders. Figure 9 shows the brain structure of a child having ASD [34]. *Children* **2020**, *7*, x FOR PEER REVIEW 11 of 17 brain's functional organization and examine if altered in neurological or mental disorders. Figure 9 shows the brain structure of a child having ASD [34].

**Figure 9.** The brain structure variations in an autistic child**. Figure 9.** The brain structure variations in an autistic child.

### *Experimental Analysis*

the prediction.

*Experimental Analysis* To evaluate the performance, we used k-fold cross-validation. We used three machine learning algorithms and two forms of neural networks to detect ASD. Table 1 compares the accuracy, sensitivity, and specificity of the four proposed models with individual-based classifiers. The accuracy refers to the number of correct predictions made by the predictive model over the rest of the predictions. Sensitivity refers to how sensitive the classifier is in detecting positive instances, and To evaluate the performance, we used k-fold cross-validation. We used three machine learning algorithms and two forms of neural networks to detect ASD. Table 1 compares the accuracy, sensitivity, and specificity of the four proposed models with individual-based classifiers. The accuracy refers to the number of correct predictions made by the predictive model over the rest of the predictions. Sensitivity refers to how sensitive the classifier is in detecting positive instances, and specificity is the proportion of the true negatives correctly identified by a diagnostic test. The results show that an autoencoder, combined with CNN, had achieved 84.05% accuracy, which outperformed other methods. The percentage of improvement in each measure is illustrated in Table 2.

specificity is the proportion of the true negatives correctly identified by a diagnostic test. The results show that an autoencoder, combined with CNN, had achieved 84.05% accuracy, which outperformed other methods. The percentage of improvement in each measure is illustrated in Table 2. **Table 1.** Sensitivity and specificity. **Model Accuracy % Sensitivity % Specificity %** Figures 10–13 compare the performance of the proposed hybrid models against individual classifiers. We could observe that the hybrid autoencoder-based models outperformed the individual methods measured by the increased value of the accuracy, sensitivity, and the low values of the specificity measures. The Receiver Operating Characteristic (ROC) curve for each hybrid model, compared to the individual classifier, is shown in Figures 14–17. We could observe that the ROC curve for autoencoder–CNN was better than any other model. ROC is a probability curve, and AUC represents the degree or measure of separability. The higher the AUC, the better the model was at the prediction.

> Autoencoder–CNN 84.05 80 75.3 Evo Norm CNN 74 71.33 65.2

> Autoencoder–SVM 69.1 66.5 71.69 Random Forest (RF) 61.5 53.8 68.8

Autoencoder–KNN 60.1 35 84

CNN: convolution neural network; SVM: support vector machine; KNN: K-nearest neighbour.

**Table 2.** Percentage of improvement in accuracy, sensitivity, and specificity.

Autoencoder–CNN 10.05% 8.67% 10.1% Autoencoder–SVM 8.9% 31.4% −12.41% Autoencoder–RF 3.8% −0.5% 3.3% Autoencoder–KNN 2% −33.2% 28.5%

**Model Accuracy % Sensitivity % Specificity %**

Figures 10–13 compare the performance of the proposed hybrid models against individual

classifiers. We could observe that the hybrid autoencoder-based models outperformed the individual methods measured by the increased value of the accuracy, sensitivity, and the low values of the specificity measures. The Receiver Operating Characteristic (ROC) curve for each hybrid model, compared to the individual classifier, is shown in Figures 14–17. We could observe that the ROC curve for autoencoder–CNN was better than any other model. ROC is a probability curve, and AUC represents the degree or measure of separability. The higher the AUC, the better the model was at

SVM 60.2 35.1 84.1


**Table 1.** Sensitivity and specificity.

CNN: convolution neural network; SVM: support vector machine; KNN: K-nearest neighbour.

**Table 2.** Percentage of improvement in accuracy, sensitivity, and specificity.


*Children* **2020**, *7*, x FOR PEER REVIEW 12 of 17

*Children* **2020**, *7*, x FOR PEER REVIEW 12 of 17

**Figure 10.** Autoencoder-CNN vs. CNN. CNN: convolution neural network. **Figure 10.** Autoencoder-CNN vs. CNN. CNN: convolution neural network. **Figure 10.** Autoencoder-CNN vs. CNN. CNN: convolution neural network.

**Figure 11.** Autoencoder-SVM vs. SVM. **Figure 11.** Autoencoder-SVM vs. SVM. **Figure 11.** Autoencoder-SVM vs. SVM.

**Figure 12.** Autoencoder-RF vs. RF.

**Figure 12.** Autoencoder-RF vs. RF.

**Figure 13.** Autoencoder-KNN vs. KNN.

**Figure 13.** Autoencoder-KNN vs. KNN.

**Figure 11.** Autoencoder-SVM vs. SVM.

**Figure 11.** Autoencoder-SVM vs. SVM.

**Figure 10.** Autoencoder-CNN vs. CNN. CNN: convolution neural network.

**Figure 10.** Autoencoder-CNN vs. CNN. CNN: convolution neural network.

*Children* **2020**, *7*, x FOR PEER REVIEW 12 of 17

**Figure 13.** Autoencoder-KNN vs. KNN. **Figure 13. Figure 13.** Autoencoder Autoencoder-KNN vs. KNN. -KNN vs. KNN. AUC: Area under the Curve.

**Figure 14.** ROC curve and AUC values for autoencoder–CNN (**Left**) and CNN (**Right**). ROC: Receiver Operating Characteristics; AUC: Area under the Curve. **Figure 14.** ROC curve and AUC values for autoencoder–CNN (**Left**) and CNN (**Right**). ROC: Receiver Operating Characteristics; AUC: Area under the Curve.

Table 3 shows the values of accuracy, sensitivity, specificity, and AUC [35]. The first value indicates the average of all the 10 k-folds, and the second value is the standard deviation. The standard deviation helps to calculate the amount of variation in a set of values [36]. A low standard deviation indicates

**Figure 15.** ROC curve and AUC values for autoencoder–SVM (**Left**) and SVM (**Right**).

that the values tend to be close to the mean, while a high standard deviation indicates that the values are dispersed over a broader range [37,38].

*Children* **2020**, *7*, x FOR PEER REVIEW 13 of 17

**Table 3.** Average and standard deviation in accuracy, sensitivity, specificity, and AUC.

**Model Accuracy % Sensitivity % Specificity % AUC** KNN 0.582 (+/−) 0.02 0.682 (+/−) 0.07 0.555 (+/−) 0.02 0.58 (+/−) 0.02

Autoencoder–KNN 0.601 (+/−) 0.035 0.35 (+/−) 0.09 0.84 (+/−) 0.08 0.595 (+/−) 0.03 Evo Norm CNN 0.743 (+/−) 0.05 0.713 (+/−) 0.07 0.652 (+/−) 0.09 0.68 (+/−) 0.05 Autoencoder–CNN 0.84 (+/−) 0.07 0.8 (+/−) 0.19 0.753 (+/−) 0.22 0.78 (+/−) 0.11

Autoencoder–RF 0.653 (+/−) 0.02 0.583 (+/−) 0.06 0.721 (+/−) 0.05 0.651 (+/−) 0.03 SVM 0.603 (+/−) 0.03 0.351 (+/−) 0.07 0.841 (+/−) 0.04 0.6 (+/−) 0.03

Autoencoder–SVM 0.691 (+/−) 0.03 0.665 (+/−) 0.06 0.716 (+/−) 0.06 0.69 (+/−) 0.03 AUC: Area under the Curve.

RF 0.615 (+/−) 0.01 0.583 (+/−) 0.06 0.688 (+/−) 0.04 0.612 (+/−) 0.01

indicates that the values are dispersed over a broader range [37,38].

Table 3 shows the values of accuracy, sensitivity, specificity, and AUC [35]. The first value indicates the average of all the 10 k-folds, and the second value is the standard deviation. The standard deviation helps to calculate the amount of variation in a set of values [36]. A low standard deviation indicates that the values tend to be close to the mean, while a high standard deviation

The F1 score was also calculated for all the proposed models, as shown in Figure 18. CNN achieved better results compared to other models. The F1 score can be interpreted as a weighted average of precision and recall. The relative contribution of precision and recall to the F1 score are equal. The F1 score [39,40] is calculated as:

$$\mathbf{F1} = \mathbf{2} \times \left( precision \times recall \right) / \left( precision + recall \right) \tag{10}$$

**Figure 15.** ROC curve and AUC values for autoencoder–SVM (**Left**) and SVM (**Right**). **Figure 15.** ROC curve and AUC values for autoencoder–SVM (**Left**) and SVM (**Right**). *Children* **2020**, *7*, x FOR PEER REVIEW 14 of 17

**Figure 16.** ROC curve and AUC values for autoencoder–RF (**Left**) and RF (**Right**). **Figure 16.** ROC curve and AUC values for autoencoder–RF (**Left**) and RF (**Right**).

**Figure 17.** ROC curve and AUC values for autoencoder–KNN (**Left**) and KNN (**Right**).

**Figure 18.** The F1 score.

equal. The F1 score [39,40] is calculated as:

The F1 score was also calculated for all the proposed models, as shown in Figure 18. CNN achieved better results compared to other models. The F1 score can be interpreted as a weighted average of precision and recall. The relative contribution of precision and recall to the F1 score are

F1 = 2 × ( × )/( + ) (10)

**Figure 16.** ROC curve and AUC values for autoencoder–RF (**Left**) and RF (**Right**).

*Children* **2020**, *7*, x FOR PEER REVIEW 14 of 17

**Figure 17.** ROC curve and AUC values for autoencoder–KNN (**Left**) and KNN (**Right**). **Figure 17.** ROC curve and AUC values for autoencoder–KNN (**Left**) and KNN (**Right**).

The F1 score was also calculated for all the proposed models, as shown in Figure 18. CNN **Table 3.** Average and standard deviation in accuracy, sensitivity, specificity, and AUC.


AUC: Area under the Curve. F1 = 2 × ( × )/( + ) (10)

The experimental results in Figure 18 show that the autoencoder with CNN was the best performing hybrid model for our problem. The computational time taken by each model is illustrated in Figure 19. The autoencoder combined with CNN has the highest computational time for training, while the autoencoder–KNN had the lowest computational time.

while the autoencoder–KNN had the lowest computational time.

The experimental results in Figure 18 show that the autoencoder with CNN was the best performing hybrid model for our problem. The computational time taken by each model is illustrated

**Figure 19.** Computational time (s). **Figure 19.**Computational time (s).

#### **6. Conclusions and Future Direction 6. Conclusions and Future Direction**

The purpose of this paper was to provide an automatic diagnosis algorithm that classifies individuals with ASD and non-ASD. Experimental results show that the hybrid autoencoder with convolution networks gave a better performance by achieving 84.05% accuracy and an AUC value of 0.78. An autoencoder with CNN consumed approximately 55 min to train the model. The autoencoder performed feature selection, and those features can be given as input to any classification model. The AUC score for an autoencoder combined with SVM, random forest, and KNN was 0.69, 0.65, and 0.59, respectively. The main drawback of our experiment was the size of the dataset. We had a small dataset with 1112 rs-fMRI images. In the future, personal characteristics data like birth weight, mother's age, and family history could give us more promising results. Survey data related to the child's daily behaviour or a short clip of the participant could help detect autism, which can add more weightage to the number of features. Complex models like ResNet-50 can be applied to the data of fMRI through the approach of transfer learning. Future directions also include a combative analysis to compare the performance of other dimensionality reduction methods such as the principal component analysis, linear discriminant analysis (linear models), locally linear embedding, and Isomap (non-linear models) when combined with machine learning or deep learning classifiers. The results of KNN can be improved by using the Grid Search CV method, and it helps to loop through predefined hyperparameters and fit your estimator (model) on your training set. So, in the end, we could select the best parameters from the listed hyperparameters. In summary, we developed and compared an autoencoder's performance combined with other models to the performance of the individual models to understand how much the emphasis of encoded features The purpose of this paper was to provide an automatic diagnosis algorithm that classifies individuals with ASD and non-ASD. Experimental results show that the hybrid autoencoder with convolution networks gave a better performance by achieving 84.05% accuracy and an AUC value of 0.78. An autoencoder with CNN consumed approximately 55 min to train the model. The autoencoder performed feature selection, and those features can be given as input to any classification model. The AUC score for an autoencoder combined with SVM, random forest, and KNN was 0.69, 0.65, and 0.59, respectively. The main drawback of our experiment was the size of the dataset. We had a small dataset with 1112 rs-fMRI images. In the future, personal characteristics data like birth weight, mother's age, and family history could give us more promising results. Survey data related to the child's daily behaviour or a short clip of the participant could help detect autism, which can add more weightage to the number of features. Complex models like ResNet-50 can be applied to the data of fMRI through the approach of transfer learning. Future directions also include a combative analysis to compare the performance of other dimensionality reduction methods such as the principal component analysis, linear discriminant analysis (linear models), locally linear embedding, and Isomap (non-linear models) when combined with machine learning or deep learning classifiers. The results of KNN can be improved by using the Grid Search CV method, and it helps to loop through predefined hyperparameters and fit your estimator (model) on your training set. So, in the end, we could select the best parameters from the listed hyperparameters. In summary, we developed and compared an autoencoder's performance combined with other models to the performance of the individual models to understand how much the emphasis of encoded features helps classify ASD candidates from fMRI images.

helps classify ASD candidates from fMRI images. **Author Contributions:** Software, H.S.; Supervision, R.K.; Visualization, H.S.; Writing – original draft, H.S. and R.K.; Writing – review & editing, H.S. and R.K.; Validation, R.K.; Source, H.S. and R.K. All authors have read **Author Contributions:** Software, H.S.; Supervision, R.K.; Visualization, H.S.; Writing—original draft, H.S. and R.K.; Writing—review & editing, H.S. and R.K.; Validation, R.K.; Source, H.S. and R.K. All authors have read and agreed to the published version of the manuscript.

and agreed to the published version of the manuscript. **Funding:** This research received no external funding.

**Funding:** This research received no external funding. **Conflicts of Interest:** The authors declare no conflict of interest.

#### **Conflicts of Interest:** The authors declare no conflict of interest. **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## **Using Stakeholder Involvement, Expert Knowledge and Naturalistic Implementation to Co-Design a Complex Intervention to Support Children's Inclusion and Participation in Schools: The CIRCLE Framework**

**Donald Maciver 1,\* , Cathleen Hunter <sup>1</sup> , Lorna Johnston 1,2 and Kirsty Forsyth <sup>1</sup>**


**Abstract:** Whist inclusion is recommended for most children most of the time it remains difficult to implement. In this paper, we present the process undertaken to review and redesign a preexisting complex intervention (The CIRCLE Framework) which was designed to enhance teachers confidence and competence in provision of universal first level supports for 5–12 year old children with additional support needs. The approach presented draws on the Medical Research Council guidance for the development of complex interventions. A series of ten co-design workshops with 70 stakeholders was completed, applying interactive and participatory methods. Analysing outputs of each workshop revealed recurring design ideas that became the main aspects of the new framework and associated manuals. Intervention content, theoretical frameworks, manuals to support use in practice and implementation strategies were developed. On completion, the updated intervention was extended up to 18 years of age and redistributed to all teachers in the participating local authority. We present the main conclusions and interpretations around the design and naturalistic implementation of the framework, and reflections on use in practice, including a detailed list of recommendations for implementation across schools and staff.

**Keywords:** inclusive education; complex interventions development; teachers; health professionals; qualitative; co-production; co-design

### **1. Introduction**

Increasingly, children with additional support needs are educated alongside peers in inclusive schools [1–5]. There are numerous studies and theoretical perspectives associated with what makes an effective inclusive school [6]. Core principles that should underpin comprehensive school reform to facilitate inclusion are widely accessible. However, teachers still have difficulty operationalising these concepts, and including learners with additional needs. There are issues with attitudes to inclusion and disability [7], understanding of specific needs, for example autism and other neurodevelopmental disorders [8], and level of knowledge of staff [9], leading to ongoing calls for more training, support, and resources [10]. Attempting to reduce unequal outcomes for all children is the primary goal [11]. Within this context, we developed a novel intervention to improve the confidence and competence of teachers around inclusion and participation in the school setting [12]. This paper presents an analysis of the steps taken to improve and extend this intervention.

Modern methods of intervention development, for example the Medical Research Council complex interventions development framework [13], are systematic and theorydriven approaches to developing novel interventions. Developing the evidence base for

**Citation:** Maciver, D.; Hunter, C.; Johnston, L.; Forsyth, K. Using Stakeholder Involvement, Expert Knowledge and Naturalistic Implementation to Co-Design a Complex Intervention to Support Children's Inclusion and Participation in Schools: The CIRCLE Framework. *Children* **2021**, *8*, 217. https://doi.org/10.3390/children 8030217

Academic Editor: Sari A. Acra

Received: 10 December 2020 Accepted: 9 March 2021 Published: 11 March 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

new interventions may include a range of methods. Taking an approach based on quantitative statistical techniques, such as meta-analysis or controlled trial, whilst desirable, is not possible when robust quantitative data does not exist for the intervention or population [14]. Qualitative research is a useful alternative [15,16]. Qualitative methodologies can be used to develop new understanding about how to deliver interventions, mechanisms and outcomes of interventions, and key features of interventions believed to be important [17]. Extensive stakeholder involvement is of fundamental importance, helping to identify priorities, understand problems and find solutions that will work in the real world, ensuring that interventions are realistic and effective for their context [18].

This paper presents an intervention development process, drawing on techniques of qualitative research and stakeholder involvement. The process covers the review and redevelopment of the CIRCLE Framework, an intervention that was designed to support teachers to be inclusive practitioners and to work effectively with children with additional support needs in schools.

### *1.1. Background and Context*

In Scotland, where this research was carried out, "additional support needs" is the term used to describe children who require support over and above that which is typically required. Direct comparison across countries are complicated by differing systems and definitions, however, analogous terms internationally are Special Educational Needs (SEN) and/or Special Educational Needs and Disabilities (SEND). In line with international trends, numbers of children with additional support needs in the school system have been rising significantly in Scotland. In 2012, 16.9% of 5–12 year old children (62,572 children) were recorded as having one or more additional support need, rising to 27% in 2019 (107,635 children) [19]. For secondary schools (12–18 year old learners), 16.5% of learners were recorded as having one or more additional support need in 2012 (48,486 learners) compared to 35% in 2019 (101,130 learners) [19]. Additional support needs are commonly identified for autistic children, children with social and emotional needs, children with learning difficulties and children with speech or language disorders [19] [for more information, including a breakdown of neurodevelopmental disorders, see Supplemental Materials A1–A4]. Segregated specialist schools are available, but this is for a very small minority. In 2019 there were 7132 children registered for special education in Scotland (across primary and secondary education) [19]. Therefore, it is clear that the focus on inclusion has led to the majority of children being educated in general education, leading to a subsequent interest in tools and methods to improve inclusion for this group [20].

Placement is no longer the key determinant, i.e., special vs. other schools [21]. From a rights-based perspective, inclusion is an ethical priority [22]. Contemporary practice focusses on ensuring children's participation and inclusion through changes to the school, with particular emphasis on staff behaviours, environments, routines and structures. Most commentators broadly support the case for inclusion, indicating that there is a strong argument, and some empirical evidence, that inclusive educational settings confer benefits [23,24]. However, the strength of the evidence is variable [25] and hampered by lack of clear definitions and theoretical frameworks [26]. A recent review has identified that the field has a large preponderance of theory and policy; however, research that actually develops, applies, and adds evidence on how support should be provided is less common [27]. A key issue is an overt focus on individual conditions leading to a "program for every problem" [28]. An overabundance of programs makes getting it right for children with additional needs difficult. There are also rights-based issues around making sure children are not reduced to their limitations [21]. Research indicates that teachers often require new skills and knowledge to work with diverse learners, leading to recommendations that additional training around inclusive education is necessary within initial teacher training and continuing professional development [29].

Considering the above, a cross-discipline partnership was developed between education staff, health professionals and academics to address these issues. The title of the

partnership was the Child Inclusion: Research into the Curriculum, Learning and Education (CIRCLE) Collaboration. This group undertook the development of the CIRCLE framework [12]. This framework was in response to a requirement for a classroom teacher focussed intervention for use in mainstream schools to improve teachers' practices.

### *1.2. Initial CIRCLE Development and Evaluation*

The first version of the CIRCLE framework was developed using a literature review and a 2-year period of qualitative research [12]. The qualitative studies involved in-depth interviews and observation of professionals from education (working in mainstream and specialist settings), health staff and parents/carers of children aged 5–12 years [12]. The CIRCLE development team included Head (Principle) Teachers, Educational Psychologists, a range of specialists in inclusive practice and additional support for learning, and senior paediatric specialists from occupational therapy, speech and language therapy, and physiotherapy, in addition to academics and clinical researchers. The development team helped to synthesise key practices identified from analysis of the preceding research. Through a cyclical process of review and refinement, this led to the development of the CIRCLE framework and a manual that was considered reflective of best practice.

The CIRCLE framework and associated manual were designed to provide a universal first level framework for teachers and other related staff. Reflecting the social model of disability, the CIRCLE framework emphasises embracing difference and the importance of considering and adapting environmental factors. As such, the framework and manual include strategies based on this set of ideas to support adaptations to structures and routines, modifications to the physical environment and teacher approaches for individual children. A key underpinning concept is the premise that supporting children of all abilities is not the remit of specialists, but the duty of everybody; and that all teachers within the school can and should provide support for all children.

Initial research found that the CIRCLE framework was received well [12]. Teachers and other school staff reported it was a useful resource [12]. Teachers reported that the framework and associated manual were feasible and acceptable, and supported them to be systematic in their approach to meeting the needs of children with additional needs [12]. Users reported that the framework and manual were useful in supporting joint working by providing a common language for collaboration [12]. Following the initial research, CIRCLE was disseminated across the local education authority, which was an urbanised area encompassing c85 primary schools, c1400 teachers, c30,000 primary aged children.

In view of its perceived utility, the local authority commissioned further development of the CIRCLE framework and expanded the remit to include support for older children (12–18 years). This process is covered in this paper.

### **2. Materials and Methods**

### *2.1. Ethical Approval*

Ethical approval was provided by University Research Ethics committee (Queen Margaret University "CIRCLE Project" 01062013) and permission to work with teachers and schools was granted by the Local Authority Research Access Service. Written informed consent was obtained from workshop participants and anyone who provided qualitative data. All participants were volunteers and given the opportunity to withdraw at any time without giving a reason.

### *2.2. Process in Development of the Updated CIRCLE Framework*

Following the Medical Research Council complex interventions development framework [13] several steps were completed, encompassing co-design workshops, stakeholder involvement, theory development, and naturalistic implementation. See Figure 1. This paper presents each of these stages in turn, covering methods and outputs sequentially.

*2.2. Process in Development of the Updated CIRCLE Framework* 

Following the Medical Research Council complex interventions development framework [13] several steps were completed, encompassing co-design workshops, stakeholder involvement, theory development, and naturalistic implementation. See Figure 1. This paper presents each of these stages in turn, covering methods and outputs sequentially.

**Figure 1.** Process for development of CIRCLE framework. **Figure 1.** Process for development of CIRCLE framework.

#### **3. Results 3. Results**

#### *3.1. Co-Design Workshops A: Initial Review and Feedback 3.1. Co-Design Workshops A: Initial Review and Feedback*

#### 3.1.1. Procedure and Analysis 3.1.1. Procedure and Analysis

The next iteration of the CIRCLE framework was targeted at 5–12 and 12–18 year age ranges. A qualitative study including 125 teachers and other staff, focussing on best practices for use with older children with additional needs, was completed. Details of which are published elsewhere [30]. This study provided the detail for interventions, supports and strategies that would underpin development for older learners' content. The next iteration of the CIRCLE framework was targeted at 5–12 and 12–18 year age ranges. A qualitative study including 125 teachers and other staff, focussing on best practices for use with older children with additional needs, was completed. Details of which are published elsewhere [30]. This study provided the detail for interventions, supports and strategies that would underpin development for older learners' content.

Next, a series of ten co-design workshops applying interactive and participatory methods [31,32] were completed over a 2-year period. Seventy stakeholders participated. Participants in the workshops included the research team, specialist additional support for learning teachers, classroom teachers, specialist therapists, managers of education and health services, psychologists, medical doctors, and parents/carers. The research team included combined expertise in complex intervention development, education, rehabilitation, autism, occupational therapy, physiotherapy, and speech and language therapy. Next, a series of ten co-design workshops applying interactive and participatory methods [31,32] were completed over a 2-year period. Seventy stakeholders participated. Participants in the workshops included the research team, specialist additional support for learning teachers, classroom teachers, specialist therapists, managers of education and health services, psychologists, medical doctors, and parents/carers. The research team included combined expertise in complex intervention development, education, rehabilitation, autism, occupational therapy, physiotherapy, and speech and language therapy.

A core group of stakeholders attended most of the workshops and acted as an Advisory Group with additional responsibilities for consultation, review and leadership. This core group comprised a senior local authority education manager, a senior additional support for learning manager, a senior educational psychologist, senior members of the additional support for learning service, a head teacher, class teachers and senior therapists from speech and language therapy and occupational therapy. A core group of stakeholders attended most of the workshops and acted as an Advisory Group with additional responsibilities for consultation, review and leadership. This core group comprised a senior local authority education manager, a senior additional support for learning manager, a senior educational psychologist, senior members of the additional support for learning service, a head teacher, class teachers and senior therapists from speech and language therapy and occupational therapy.

In the workshops, which included open discussion, small group work and brainstorming, participants were presented with updates and ideas for development of the CIRCLE resources. Facilitators led the workshops, and recorded notes. The aim of the workshops was to review the existing CIRCLE manual, gather feedback on current use, In the workshops, which included open discussion, small group work and brainstorming, participants were presented with updates and ideas for development of the CIRCLE resources. Facilitators led the workshops, and recorded notes. The aim of the workshops was to review the existing CIRCLE manual, gather feedback on current use, determine the validity of development ideas and develop new ideas. Detailed notes capturing each group's discussion, how the participants tackled each activity, and feedback on the intervention ideas were produced and analysed. The analytic procedure drew on qualitative approaches of thematic and framework analysis involving familiarisation, identification of a thematic framework, coding of the data according to the framework, charting the themes, and mapping and interpreting the data [33–36]. To safeguard trustworthiness and transparency, peer checking was undertaken with colleagues about emergent ideas. The facilitators prepared detailed presentations to reflect group discussions. These presentations

were given to the next workshop in the sequence. In this way, the analysis and outputs from each workshop were verified and built upon at the next workshop in sequence, leading to the design ideas (i.e., broad goals for the CIRCLE update) which became the new aspects of the framework and associated materials.

### 3.1.2. Outputs

Overall utility of CIRCLE. Feedback from participants in the early workshops emphasised that the utility of CIRCLE was perceived to be high, indicating that the existing materials had a positive impact on staff and children in terms of improving the practices of teachers. Use of the resources were perceived to increase understanding of children's needs and related supports. As a reference point for busy teachers, the current manuals were perceived as easy to use, clearly laid out, comprehensive and comprehensible. School leaders and experienced teachers reported that CIRCLE was a useful part of the strategy to increase the amount of support provided directly by teachers in schools, and to effectively deal with issues in schools via school pathways (prior to requiring external support). It was reported that using the resource had helped staff to shift towards meeting the needs of all children along the lines of a more inclusive classroom, rather than a "pull out" or "expert" model. CIRCLE resources were perceived to raise awareness of the inclusion "agenda."

Uses in practice. As well as use by classroom teachers, specialist staff had been using the resource to share information between colleagues (e.g., at times of transition), or between the teacher and the additional support for learning services, or for meetings. The CIRCLE manual had been used as a resource by teachers to photocopy and tick strategies to try with individual pupils. The resources were also perceived to give teachers ideas as to what to do next, help to guide staff through the referral process for extra support, and in some cases prevent unnecessary referrals. It was also reported that there was a function for clear and transparent documentation about which strategies had been used in school.

Requirement for pencil and paper tools and other issues. Participants reported that they were frustrated that there was no formal method presented in the manuals to record information on children's progress, and no formal way to communicate information using the existing CIRCLE framework. How to use the manuals to facilitate recording of information, and the associated tools required for this, was a key area of discussion. Participants reported that it would be helpful if the manual contained specific checklists and other pages, which could be photocopied and shared between teachers and related services personnel. Concerns around duplication of content across the resource were also reported. Descriptions and acronyms used in the resource were felt to be confusing, and some of the sections (particularly the inclusive classroom section) was perceived to be too short and lacking in content. Additionally navigation of the main document was described as problematic, and there were reported difficulties in finding the relevant sections to support a specific child.

Theoretical aspects. Presentation of a more coherent theoretical framework was requested. It was considered necessary to move the focus away from the child's perceived deficits towards wider aspects of inclusion. It was felt that there should be more content related to inclusive environments. The overall presentation of strategies was to be reviewed, particularly to highlight the importance of environmental adaptations. The participants in the early workshops unanimously agreed the documents should be carefully written to reduce emphasis on child deficits. Throughout, it was felt terms should be replaced by positive headings wherever possible. Although the resources were largely felt to be clear, it was requested that a systematic process should be developed to facilitate use of the materials.

### *3.2. Co-Design Workshops B: Theoretical Framework*

### 3.2.1. Procedure and Analysis

The Medical Research Council guidance on development of complex interventions includes conceptual framework development as a necessary step [13]. The main ideas

considered in the development of the CIRCLE framework focussed on person-environment interactions [37] and models of participation [38]. The primary theoretical framework used was the Model of Human Occupation (MOHO) [39]. The MOHO is a rehabilitation model that aims to enhance people's participation, and includes concepts related to how people engage in everyday life focusing on values, attitudes, habits, routines, skills and the environment [39]. The relevance of the MOHO to understanding psychosocial and environmental factors in community, home and school situations for children is well-established [40–44]. The MOHO uses a holistic understanding of people, their daily life activities, interests and needs, and relationship with their environment to develop interventions [39]. People are conceptualised as being participation driven, with satisfactory engagement in personally and socially valued activities and roles seen as the fundamental outcome of interventions [39]. MOHO theory encourages interventions at multiple levels, and provides a structure for assessment and application of supports and strategies. It was agreed, due to its known utility for understanding the needs of children, and person-environment approach, to use the MOHO to guide the development of the CIRCLE framework. Using this model as the broad basis for CIRCLE meant several aspects had to be considered in tandem. The MOHO examines personal (i.e., motivation, habits and skills) and environmental (e.g., physical and social environment) influences [39]. This means understanding that "impairment" and "disability" are not person focussed, but linked to, and oftentimes driven by the environment. Adopting such a model is in response to critique around the labelling issues, stigma and disadvantage associated with a deficit or "medical" model approach [45]. A model that focuses on participation provides a beneficial alternative structure, meeting calls for a "social" or "biopsychosocial" approach [38,39].

Using MOHO theory as a guiding structure, common themes and features that related to the core intervention concepts were developed, discussed, modified, debated and eventually ratified in the co-design workshops. Across workshops, the focus was towards developing a framework that could be used to organise and present important ideas to the users, and how to "translate" MOHO concepts, some of which were unfamiliar to teachers, into a useable format and language. To our knowledge, this was the first application of these ideas for teachers. Using the above review, discussion and analysis led to the development of a new CIRCLE theoretical framework.

### 3.2.2. Outputs

The updated CIRCLE Framework, which encompasses a streamlined and education focussed application of MOHO concepts, presents children's inclusion and participation in terms of four main areas (see Figure 2 and Table 1):


We presented the framework as a jigsaw puzzle to highlight the interconnectedness of the component parts. Workshop participants had perceived that there was a tendency by some teachers and others to focus on a child's physical, sensory or behavioural deficits. They agreed that it was important to shift focus towards different areas (including environment and attitudes) rather than on deficits. Within this framework, it was important to stress some key ideas. Firstly, that participation and inclusion are a function of the child and environment together [38,39] with particular reference to the social model of disability [22]. We also took from MOHO the idea that people and environments are a complex dynamic system, and that children's needs are best understood within a framework that considers motivation, structures and routines, and skills together. A key aspect identified was that the support for children should continue to take place in typical classrooms—and that this was the main and best place for most children. A key focus was therefore the environment. The idea of "environment first" was consistently highlighted as important by the workshop participants. Therefore, environmental adaptation and review of the environment should

Structures and routines

Skills

be the first step. Rather than thinking about what a learner can and cannot do, or only thinking about their underlying ability, the framework encourages teachers to think more widely, particularly paying attention to the disabling aspects of the environment. Using MOHO theory, the environment is understood to contain physical and social components, each of which requires attention [39]. *Children* **2021**, *8*, x FOR PEER REVIEW 7 of 18

**Figure 2.** Updated CIRCLE framework. Reproduced from CIRCLE manuals [46,47]. **Figure 2.** Updated CIRCLE framework. Reproduced from CIRCLE manuals [46,47].

**Table 1.** Overview of CIRCLE framework components \*. **Framework Component Description**  Physical environment The physical environment refers to the physical layout of the classroom and the resources used within it. Social environment The social environment is concerned with the attitudes, expectations and actions of those within the class and how these can affect children either positively or negatively. Structures and routines are events that happen in the same way with regularity. The start, middle and end of the routine becomes predictable through repetition. Structures and routines were also included as an explicit aspect of the model [39]. Routines (i.e., daily structures of the school) support children to be able to anticipate transitions and required actions. Order and consistency are clearly helpful for children, and for some, school is the place where this is most apparent and helpful. For example, children may benefit from strategies including explicit structuring of the day/week or provision of visual supports (i.e., visual timetables) to help them follow routines or understand what comes next.

Motivation Motivation gives children incentive, enthusiasm and interest when engaging with activities and the people around them. Children are motivated by their own feelings, desires, self-esteem and confidence. Skills refer to a learner's skills in the key areas: attention and concentration; organisation and planning; posture and mobility; dexterity and manipulation; socialising, emotions and relationships; verbal and non-verbal communication. \* reproduced from CIRCLE manuals [46,47]. We presented the framework as a jigsaw puzzle to highlight the interconnectedness of the component parts. Workshop participants had perceived that there was a tendency by some teachers and others to focus on a child's physical, sensory or behavioural deficits. They agreed that it was important to shift focus towards different areas (including environment and attitudes) rather than on deficits. Within this framework, it was important to stress some key ideas. Firstly, that participation and inclusion are a function of the child Motivation was included as an explicit aspect of the model, again drawing on MOHO theory. This focused across three overarching themes: "interests," "values," and "abilities." Children's "interests" is related to motivation and activities that are engrossing, enjoyable or satisfying to them. Supportive practices include utilising learners' own specific ideas, hobbies, or cultural background, and ensuring individualisation and options. "Values" refers to what children find important and meaningful to them. Supportive practices include listening to and valuing children's views, jointly setting goals, and self-assessment. Lastly, "abilities" refers to how children perceive themselves in terms of their ability and capacity. Supportive practices associated with this idea include differentiating work, setting achievable goals whilst ensuring challenge, giving positive feedback, and affirming interests, languages and cultures.


**Table 1.** Overview of CIRCLE framework components \*.

\* reproduced from CIRCLE manuals [46,47].

The presentation of supports and strategies was split into areas that were felt to be "neutral" labels (rather than condition specific or deficit focused). For younger learners these were: Attention and Concentration Skills; Organisational and Planning Skills; Posture and Mobility (Gross Motor) Skills; Dexterity and Manipulation (Fine Motor) Skills; Social, Emotional and Relationship Skills; and Verbal and Non-verbal Communication Skills. For older learners these were Attention and Concentration Skills; Organisation and Planning Skills; Motor Skills; Social, Emotional and Relationship Skills; Verbal and Non-Verbal Communication Skills. These categories and labels were agreed by the co-design workshop participants, who identified them as the main underlying areas of challenge experienced by children in schools. Each area was described and suggestions of strategies were developed using positive language, with culturally diverse examples. In line with the theoretical framework, throughout the manuals the strategies were split into "modifications to the learning environment" first, followed by "establishing structures and routines" second and "approaches to enhance motivation" third [for examples, see Supplementary Materials A5 and A6].

### *3.3. Co-Design Workshops C: Developing Pencil and Paper Tools*

### 3.3.1. Procedure and Analysis

To reflect the conceptual framework, and at the request of the stakeholders in the co-design workshops, pencil and paper tools to support information gathering and information sharing were developed. Two tools were developed: the CIRCLE Participation Scale (CPS) and the CIRCLE Inclusive Classroom Scale (CICS). These tools were designed to form the assessment portion of the CIRCLE intervention, to facilitate teachers' engagement with the manual, and as a method for identifying where to target supports and strategies for children.

Workshop participants had indicated that teachers valued and tended to engage with tools that were quick and easy to use. They also reported that teachers were looking for tools that could be used to record input/change in order to support communication with colleagues, partner services and agencies, parents, and with the children themselves. Furthermore, a key output of the co-design workshops was to have tools that would shift teacher focus from children's impairments towards one where they considered environmental factors first. It was also important to develop easy to use tools that would support

teachers to consider their role in the inclusion of children, rather than considering this a role devolved to specialists.

The CPS and CICS were developed as first level tools to be used sequentially (environment tool first, followed by the participation tool). They were designed to be concise and easy to use by all education and related services personnel irrespective of training or level. To develop these tools, factors considered to have a potential influence on children's participation were identified drawing on our previous research [12,30] and using the Model of Human Occupation (MOHO) [39] as a theoretical guide. To ensure content validity, face validity and utility, over the course of three meetings the co-design workshop Advisory Group reviewed and provided commentary. Items were identified for inclusion and exclusion at this point. In the next phase, a longstanding independent group (N = 9) of senior professionals was asked for further comment. This group included managers, educational psychologists and researchers. This group met four times. Based on these activities, pilot versions were designed. A pilot version of the CPS was field tested in three schools by teachers, who provided detailed feedback. Classroom teachers, head teachers and representatives from a parent organisation also provided feedback and commentary on the CICS. Comments helped the research team to clarify and revise the tools, including wording of items and instructions for use, before finalisation.

### 3.3.2. Outputs

Circle Inclusive Classroom Scale (CICS): The CICS is a pencil and paper rating tool which identifies environmental barriers/supports to inclusion and participation. The tool comprises 3 domains developed using MOHO theory: the physical environment, the social environment, and structures and routines within the environment. Sub-domains for consideration include children's participation in decision making, routines, appeal of activities, expectations, activity demands, empowerment, provision of information, relationships, support and facilitation, attitudes, availability of objects, visual supports, sensory space, and accessibility/adequacy of physical spaces. The CICS utilises a 4-point rating scale for all the items within each sub-domain. Using this scale, based on the judgement of the teacher completing the assessment, a "4" rating indicates a domain that strongly supports participation of learners, whereas a "1" rating indicates aspects that strongly interfere with participation of learners where improvement is required. As well as a rating sheet and recording format, the CICS also includes a set of reflective questions that help users when considering the quality of the classroom environment. Scoring is based on observation and a "walk through" of the classroom. The CICS can be completed by one individual, or by colleagues working together [see Supplementary Materials A7–A9 for examples of content].

CIRCLE Participation Scale (CPS): The aim of the CPS is to facilitate identification and measurement of factors impacting on children's participation in school. The CPS was designed for use in general classrooms by teachers and other related staff for children aged 5–18 years with additional support needs, including physical, developmental or learning needs. The CPS consists of 10 short sections that cover the environment, structures and routines, motivation, and skills. The item pool contains items assessing the potential determinants of school participation across these areas. Each section of the CPS contains five items, which are positively framed statements with a 4 point Likert scale in which teachers rate the frequency with which, in their opinion, each item is observed for a given child. A lower score on the CPS indicates presence of barriers or needs for the child in that area, and potential requirement for additional support from staff. Once the CPS is completed, it directs users to relevant sections of the CIRCLE resources containing supports and strategies for teachers to try with children [see Supplementary Materials A7–A9 for examples of content].

Feedback: Feedback received on the tools indicated they were functional and straightforward to use. Teachers highlighted the benefit of tools that could be used for a broad range of children that directed them towards appropriate supports and strategies within

the CIRCLE manuals. Feedback indicated they were a useful way to consider provision of supports for children and a useful way to share information between colleagues, e.g., between teachers at times of transition, or between the teacher and the specialist team, or at meetings. Specialist staff reported that the tools could be useful in supporting communication between classroom teachers and specialist teachers. It was felt that the CPS and CICS encouraged teachers to look at areas of the child (e.g., motivation) or the environment (e.g., peer support) that they might not have initially considered and encouraged teachers to complete assessment of the child and the environment. Although feedback was mainly positive, a few teachers expressed concerns. These included anxieties around the time that it might take to complete the tools and anxieties around the replacement of more comprehensive or specialised (e.g., dyslexia) assessments. Others were concerned that some of the items related to the environment were outside their control. Several suggestions were made on how the tools could be used to support collaborative working.

### *3.4. Design Specification and Building a New Manual*

### 3.4.1. Procedure and Analysis

This stage of the research developed the ideas and outputs from the previous manual and new research into new materials, principally new intervention manuals. Workshop participants were supportive of developing the manuals, as they stressed the usefulness of having useable, physical copies of the CIRCLE Framework (including the tools and supports/strategies) easily available in the classroom setting. To develop the new manuals a tendering process to identify a graphics design company was undertaken. The design brief and specification were included in the tender. Low-fidelity prototype documents were created in Microsoft publisher and sent to the contracted company. The research team worked with the contracted company using the following specification:


Following development of high-fidelity prototypes, a final co-design workshop took place in a university space, including several members of the target audience, i.e., teachers. Copies of the manuals were also sent to the Advisory Group and representatives from paediatric therapy services, head teachers, senior managers within the local authority, parents and experts in inclusive education, with feedback requested. Participants were provided with a draft copy of the manual and asked to "use" the manual with a child in mind with the aim of testing its usability and validity. Feedback, was recorded and transcribed. A list of required revisions was approved.

### 3.4.2. Outputs

A list of revisions was developed. Issues were identified including document structure and signposting to relevant initiatives nationally and locally. Issues were attended to, and comments were fed back to the design company and a final version of the younger (5–12 years) [46] and older children's (12–18 years) manual was published [47].

### *3.5. Naturalistic Implementation*

### 3.5.1. Procedure and Analysis

The newly developed manuals were distributed within the local authority who commissioned the work from 2016/2017 onwards. Across primary (elementary) and secondary (senior) provision this included c85 primary schools (5–12 years), c20 secondary schools (12–18 years), and c3000 teachers.

The roll out of the resources was managed by a senior member of the Additional Support for Learning Service. To facilitate training, a senior educational psychologist

also worked with colleagues to develop a training structure for teaching staff, focused on self-evaluation and using the various tools in the resources effectively. The training and implementation was fully led by the local authority. Designing the implementation in this way ensured that it was absorbed into naturally occurring practices. This aspect was designed to foster informal networks, to leverage existing organisational structures, to ensure that knowledge of the framework and ability to train and educate using the framework was distributed, and to have the change be led by insiders rather than external "experts." This ensured that the implementation was resilient and robust, and met the needs of the target community.

Once the manuals had been made available to schools, and training completed, we collected feedback from individuals representing a convenience sample of 10 schools (staff from 8 primary schools and 2 secondary schools), as well as from a group of 20 senior professionals including senior managers, head teachers, and senior additional support for learning teachers.

### 3.5.2. Outputs

Evaluation of implementation reach across the authority. After approximately 18 months, all schools had been trained and were in possession of CIRCLE manuals. Specialist additional support for learning staff had also received training and were given resources with agreement that they would train additional staff as required. One copy each of the relevant resource had been printed for all teachers in the local authority (primary, secondary and special). Additional arrangements for further training were that leadership and support for learning teachers could attended specialist training, and cascade this training as appropriate. All teaching staff were expected to have an awareness of the resource and use it in their practice. Use in formal local authority processes was instigated, including requests from schools for extra support for children, school self-evaluation and school policies for inclusion. CIRCLE implementation was therefore supported by senior management within the authority and through strategic planning, making CIRCLE a core tool for the authority that teachers were expected to engage with. Several schools reported using CIRCLE as part of formal processes, including writing individual education plans, referrals to specialist providers, and communication with parents. Some schools reported making the use of the CICS a feature of their school calendar, where teachers carried the assessment out at the start of a new school year, ensuring that all started the year engaged with the CIRCLE framework. However, implementation between schools was varied. Schools reported that implementation was more robust where training had been completed as a 'whole school' strategy, rather than led by individuals. Some schools reported that implementation was still in the early stages.

Feedback from users. The CIRCLE resources have been well received in terms of usefulness, structure and ease of use. CIRCLE was reported to be an effective first level universal intervention for new staff, as well as a guide for more experienced staff. Teachers reported that use of the manuals supported them to clearly articulate strategies that they may include as part of their routine practice. This encouraged teachers to focus on their own individual responsibility for pupils within their class and provided a framework for discussions with other teachers and colleagues. The resources were felt to support teachers and related services personnel to look at areas of the child and environment that they might not have initially considered—and encourage teachers to do "detective work" around the child and what could help that child. The parts of the resource most commonly used were the CIRCLE Inclusive Classroom Scale, the CIRCLE Participation Scale and the various Supports and Strategies pages. Most feedback about these sections was positive, but some teachers reported that the resource, whilst helpful, was long, and required time to read and understand.

A few teachers also reported that they found it challenging to complete the tools without discussion with others. Some secondary school teachers highlighted that they did not have their own specific teaching space, so that they perceived that they had little control

over many aspects of the environment. Concern was also expressed that some teachers were reluctant to change some aspects of the environment to suit one or a small group of children if they thought that there was no benefit for the larger group. Additionally it was noted that the process, as recommended in the manual, would be difficult to complete at the start of a new session/school year, as the class would be new at that point, so that the teacher would not know the children. Not unexpectedly, a few teachers reported concerns around the time taken to complete the tools possibly in addition to other assessments; however, others thought they could be completed in a timely manner.

### **4. Discussion**

The development process described in this paper included qualitative research, stakeholder involvement and naturalistic implementation, to develop, refine and implement a complex intervention.

Principles underlying the CIRCLE intervention are that activities and interventions should be collaborative, preventative, high frequency, and based in the classroom. The thinking on this approach is that interventions should be embedded in daily lessons and routines, provided by classroom teachers as part of routine practice, rather than creating an isolated strategy or event, provided by "experts." High frequency refers to use in the daily routines of the class. Given the opportunity to provide this first level support in the natural environment of the classroom, this might reduce the need for specialist services (i.e., the CIRCLE intervention might be preventative). The CIRCLE intervention supports collaborative working with others as required, by providing a common framework and language to support communication and discussion with a range of education and health staff, parents and the child.

Within CIRCLE, the child's difficulties are not the primary concern and a combination of strategies, in particular those focusing on the environment, are required. Rather than thinking about what a learner can and cannot do, the CIRCLE Framework encourages teachers to consider what affects learner's inclusion and participation within the context of the environment, and to take responsibility for "delivering" inclusion within their classroom. Such approaches support professionals to embrace the complexity of the child's needs, create a "working hypothesis" about a child's situation and provide transparency of reasoning. They also support a consistency of service provision and provide a common language for communication and research.

It is likely that a key mechanism underpinning CIRCLE is focused discussions on inclusion. It is possible that other frameworks with similar aims, but developed with different content in different ways, might also achieve similar outcomes. Although this may be the case, it remains true that there is considerable debate in the field on the best methods of achieving inclusion-focused change [23–27]. A framework underpinning inclusion must be clear and unambiguous in order to support understanding, improve practices and influence policies. Within the CIRCLE framework, a key strength is the clarity and combination of components across child and environment.

With reference to other approaches, there are several other inclusion-focused frameworks [48–51]. However, these have tended to focus on measurement of whole school inclusive practice [49], theoretical principles [50], or require a whole system approach [51]. It is arguably the case that previous efforts to develop inclusion-focused frameworks have sometimes involved "ivory-tower" efforts failing to take account of stakeholder perspectives and real-world problems faced by schools. Our research suggests that what teachers require is a practical framework that is easy to understand, with supports and strategies for use in the classroom, which are applicable to the range of needs and abilities that are increasingly common in schools. Reflection on practice, a common language and a structure for communication and information sharing are also important. As noted, there is a very significant literature on ideas around inclusion. Along with the extensive range of different specialised interventions, this makes application for teachers, who have a general responsibility for an age-based cohort with varying support needs, difficult. The

CIRCLE framework is an intervention that any teacher can use. The framework has been designed with significant collaboration from stakeholders, including teachers and school leadership, with excellent ecological and face validity, and proven feasibility for use in the school context. This meets the need for a universal inclusion focused framework that can be applied in schools as a first level support.

Within the CIRCLE framework, the CICS and CPS pencil and paper measurement tools were developed to provide individual teachers, departments or schools a means of assessing the environment and children's participation, and of documenting and sharing this. The CICS was intended to allow the user to formally rate the classroom. Together with strategies oriented for developing an inclusive classroom, this was designed to encourage focus on the environmental aspects of inclusion. The CPS was considered the next step and intended to enable identification of a learner's strengths and also areas requiring development. It was designed to help identify which groups of supports and strategies to try by directing the user to further specific sections within the CIRCLE manual. It is a truism noted within any system, including education, that what is measured will become what is done. Therefore, the CPS and CICS provide an avenue for encouraging a beneficial focus on inclusion across child and environment.

A final issue of interest concerns the role of external specialists, who may be part of the multidisciplinary team involved with schoolchildren. Many countries have multidisciplinary and multiagency teams involved with diagnosis, assessment and intervention. A key challenge is engagement of these teams across services; particularly as inclusion and participation in school is increasingly accepted as an important component of a holistic health intervention [52]. Recent priority setting exercises for children with learning difficulties [53] and neurodisability [54] have emphasised the importance of schools and teachers in terms of training, identifying optimal learning environments and facilitating interagency collaboration. There is therefore inherent value in working with and considering the views of multiple professional groups. External professionals collaborating with teachers can support ecological assessment and intervention, as well as facilitating a focus on school participation within their own practice. CIRCLE could provide a framework and language to support discussions between external staff and education staff around implementation of first level supports for children.

### *4.1. Recommendations*

Since the initial evaluation, the CIRCLE manuals continue to be used within schools in the local authority where the research was originally carried out. The CIRCLE manuals have also been adopted in other areas across Scotland. Feedback indicates positive influence in terms of practice, through improved teacher confidence and competence in delivering early interventions and first level supports. To support dissemination, all CIRCLE manuals are freely available online [55]. To accompany the materials, a detailed list of recommendations for implementation across different school levels and individual roles was developed (Table 2). A key emphasis of CIRCLE is that teachers take responsibility for inclusion, without overt focus on specific diagnoses or needs. However, it is known that children with a range of needs for example autistic children, have particular needs which require greater understanding and more focused interventions. Use of CIRCLE represents the first universal level of intervention that should be in place. This will support all children, as well as providing a basis for support of children with more specific needs. Such first level support comprises core information and key messages required by all staff. With this foundation in place, training in more specific evidence based approaches or ways of working is still recommended. Using CIRCLE will provide staff with a foundation on which to build their knowledge and skills around supporting children with additional needs in schools.


**Table 2.** Recommendations for implementation of CIRCLE across various levels of practice.

### *4.2. Limitations*

This paper focusses on the processes used to develop a complex intervention. The preliminary findings are positive as outlined, providing good data on adequacy of theoretical underpinnings and feasibility and acceptability of CIRCLE in a variety of real-world conditions. However, although there has been extensive stakeholder involvement, feedback represents a convenience sample, which may not have been representative of the views of all teachers in the authority. Additionally, it is beyond the scope of this paper to discuss investigation into the efficacy of the CIRCLE intervention in terms of outcomes for children and young people. Future study will be required to ascertain these wider outcomes.

### **5. Conclusions**

Building application of inclusion-fostering theory and interventions into practice is fundamentally important. In this research, the Medical Research Council guidance on the development of complex interventions has been used to improve and implement a new intervention. The methods and processes undertaken mean that professionals, academics and experts with a wide range of backgrounds and expertise have been involved, including teachers, educational psychologists, a range of specialist teachers, and senior representatives from children's therapy services. The research provides a useful and novel example of developing a complex intervention in schools including co-design and stakeholder involvement.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/2227-9 067/8/3/217/s1, Supplementary Material A1 Pupils in primary mainstream schools, 2007–2019 Scotland. Supplementary Material A2 Pupils in secondary mainstream schools, 2007–2019, Scotland. Supplementary Material A3 Pupils in primary mainstream schools, 2012, 2017, 2019, Local Authority where CIRCLE was implemented. Supplementary Material A4 Pupils in secondary mainstream schools, 2012, 2017, 2019, Local Authority where CIRCLE was implemented. Supplementary Material A5 CIRCLE manual examples ages 12–18 year (4x pages). Supplementary Material A6 CIRCLE manual examples ages 5–12 years (4x pages). Supplementary Material A7 Domains, sub-domains and examples of from CIRCLE Inclusive Classroom Scale (CICS). Supplementary Material A8 Circle Inclusive Classroom Scale (CICS) reflective question examples. Supplementary Material A9 Overview of the CIRCLE Participation Scale (CPS).

**Author Contributions:** Conceptualization, D.M., C.H., L.J., and K.F.; methodology, D.M., C.H., L.J., and K.F.; software, D.M., C.H., L.J., and K.F.; validation, D.M., C.H., L.J., and K.F.; formal analysis, D.M., C.H., L.J., and K.F.; investigation, D.M., C.H., L.J., and K.F.; resources, D.M., C.H., L.J., and K.F.; data curation, D.M., C.H., L.J., and K.F.; writing—original draft preparation, D.M.; writing—review and editing, D.M., C.H., L.J., and K.F.; visualization, D.M., C.H., L.J., and K.F.; supervision, D.M., C.H., L.J., and K.F.; project administration, D.M., C.H., L.J., and K.F.; funding acquisition, K.F. and D.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by the City of Edinburgh Council.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by Institutional Review Board (Queen Margaret University "CIRCLE Project" 01062013).

**Informed Consent Statement:** Written informed consent was obtained from workshop participants and anyone who provided qualitative data. All participants were volunteers and given the opportunity to withdraw at any time without giving a reason.

**Data Availability Statement:** Please contact the corresponding author for study data. Data are not publically available due to confidentiality and ethical requirements. The Scottish Government Pupil Census Supplementary Statistics data are available online: https://www.gov.scot/publications/ pupil-census-supplementary-statistics.

**Acknowledgments:** Thanks are due to the Head Teachers and staff from the participating schools. Thanks to the Additional Support for Learning Service, City of Edinburgh Council. Thanks to the parents and children who have supported the research. We are grateful for a final implementation

support, which has been development of an online module based on CIRCLE that has been developed by a Scottish Government agency with responsibility for professional development in education (Education Scotland) [56].

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

### **References**


*Article*

## **Application of Low-Intensity Modified Constraint-Induced Movement Therapy to Improve the Affected Upper Limb Functionality in Infantile Hemiplegia with Moderate Manual Ability: Case Series**

#### **Rocío Palomo-Carrión 1 , Rita-Pilar Romero-Galisteo 2,\* , Elena Pinero-Pinto 3,\* , Purificación López-Muñoz <sup>1</sup> , Helena Romay-Barrero <sup>1</sup> and Francisco García-Muro San José 4**


Received: 15 July 2020; Accepted: 2 September 2020; Published: 4 September 2020

**Abstract:** Objective: To assess the functionality of the affected upper limb in children diagnosed with hemiplegia aged between 4 and 8 years after applying low-intensity modified Constraint-Induced Movement Therapy (mCIMT). Methods: Prospective case series study. A mCIMT protocol was applied for five weeks, with two hours of containment per day. The study variables were quality of movement of the upper limb, spontaneous use, participation of the affected upper limb in activities of daily living, dynamic joint position, grasp–release action, grasp strength, supination and extension elbow movements. Four measurements were performed, using the quality of upper extremity test (QUEST) scale, the Shriners Hospital for Children Upper Extremity Evaluation (SHUEE) Evaluation, a hand dynamometer and a goniometer. Results: The sample was composed of eight children with moderate manual ability. Statistically significant differences were detected in all the studied variables (*p* < 0.05) between the pre-treatment and post–treatment results (Week 0–Week 5), except for upper limb dressing, putting on splints and buttoning up. In the first week, the changes were statistically significant, except for protective extension, grasp strength, grasp–release and all functional variables (level of functionality and participation of the patient's upper limbs) in the SHUEE Evaluation (*p* > 0.05). The greatest increase occurred in spontaneous use from Assessment 1 to Assessment 4 (*p* = 0.01), reaching 88.87% active participation in bimanual tasks. The quality of movement of the upper limb exhibited a significant value due to the increase in dissociated movements and grasp (*p* = 0.01). Conclusion: A low dose (50 h) of mCIMT increased the functionality of children diagnosed with congenital hemiplegia between 4 and 8 years of age with moderate manual ability.

**Keywords:**family; infantile hemiplegia; modified Constraint-Induced Movement Therapy; physical therapy modalities; upper extremity

### **1. Introduction**

Infantile hemiplegia is a subtype of infantile cerebral palsy, characterized by the affectation of one of the hemibodies as a consequence of brain injury. Its prevalence is 1 case per 1300 live births [1]. There is more affectation of the upper limb than the lower limb due to the alteration of the corticospinal tract. The affected hand has a deficit in proprioception and tactile perception, which hinders fine motor skills, generally those of the fingers and the strength exerted by them [2]. Sensory abnormalities, weak grasp and loss of manual ability (fine movements) may appear, specifically in the fingers, with slower movements, poorer coordination and longer phases associated with mirror movements. This leads to a decrease in the use of the affected hand and often interferes with the manual ability of the healthy upper limb [3].

From early childhood, children with hemiplegia, even the least affected, use their healthy hand as the dominant hand in all tasks. Therefore, they learn "not to use" their affected upper limb, which is known as developmental disregard [4]. This "non-use" of the affected upper limb produces an increase in muscle tone in the affected segment, poor motor control, decreased active and passive range of motion, generalized weakness and delayed musculoskeletal maturation. The non-use affectation is caused by neural dysfunction as a result of brain injury. This neuronal alteration [5] can be improved through the activation of certain brain areas that remained inactive after the brain lesion and also through experience and learning (trial–error).

Thus, in order to improve the affected upper limb "non-use", Constraint-Induced Movement Therapy (CIMT) is used [6,7], which consists of constraining the healthy upper limb with a whole or partial containment (glove), thus promoting the use of the affected upper limb in activities of daily living. The programmed tasks integrate the repetition of the motor action with a variety of exercises. The use of CIMT has spread in recent years among physiotherapists and occupational therapists, due to the large number of studies that support the effectiveness of this intervention compared to traditional interventions that do not restrict the use of the healthy side [8]. However, in its original conception, two premises had to be met: the restriction of the less-affected upper limb and the application of an intensive treatment applied in a structured way to the upper limbs [9]. Different variants of CIMT have emerged over the years, under the term "modified Constraint-Induced Movement Therapy" (mCIMT). Likewise, there are a variety of protocols used for CIMT, in which different containment systems are proposed [10–15] in which it is the dose that varies [6,12,14,16–27]. In this line, one of the modifications widely used in pediatrics is the one based on mCIMT, which constrains the healthy upper limb for less than 3 h [23,28–30]. McConnell et al. [31] found that a less intensive treatment (63 h of treatment over 21 days) produced similar benefits compared to a more intensive approach (126 h of treatment over 21 days). Functional gains may be feasible for some children with a less intense program adjusted to 20 h of therapy in more than two consecutive weeks [31]. According to Schweighofer et al. [32], the existence of a "functional threshold" would be necessary for the maintenance of functionality after therapy, below which the use of the upper limb decreases while the benefits to the individual remain above such threshold. It would be useful to determine the specific doses of therapy in each patient.

McConnell et al. [31] applied mCIMT for 2 weeks with two hours of dose per day in a clinical setting, designed for children aged 8–15 years, and the therapist increased the dose to continue with the use of the affected upper limb for 30 min at home. The children executed 20 h of total dose with functional changes. We proposed to apply 50 h of total dose, with the same distribution per day according to this study, although increasing the dose by 30 h, since the participants in our study were younger than those in McConnell et al. [31] and the therapy was performed at home, thus the children and their families needed more time to obtain significant results. Thus, we decided to assess the children at Week 2 of treatment (20 h) in order to verify whether the changes would be the same at Week 5 (after treatment with a total dose of 50 h). The systematic review [33] included 31 papers, each of which applied different doses per day, total doses, measurement tools, etc. All 31 studies were compared, with the main difficulty being that the children had different manual ability levels, and it would be ideal to know the correct dose for each level. In this systematic review, the manual ability levels were assessed with the Manual Classification System, MACS; some studies show Levels I–III, I–IV or I–V [34]. We could consider a moderate hand ability level for children classified as Level I–II in MACS. These levels reflect that the children are independent in the execution of activities using one or two hands or including compensation strategies (neck, trunk, mouth, etc.) to complete the bimanual task; they also show that the movement restrictions do not impede the complete use of the affected upper limb.

However, most of the published studies include functional activities in their treatment proposals. To our knowledge, a few of them contemplate an ecological vision of human development as initially proposed by Bronfenbrenner [35]. According to this perspective, it is crucial to incorporate the principles of therapy to the environment in which the child develops, which is essential to ensure the long-term persistence of the achieved results [36]. From the ecological point of view, the evolution of the child is understood as a process of progressive differentiation of the activities that he/she carries out, his/her role and his/her interactions with the environment. The interactions and transactions established between the child and the elements of his/her environment are very important, especially with his/her parents.

One of the latest reviews on the use of CIMT [8] showed that the risk of bias of the analyzed clinical trials was between moderate and high; therefore, a new randomized controlled trial is proposed in this work, whose main objective was to analyze the effectiveness of the use of CIMT by modifying the applied dose. Moreover, the ecological perspective of development was considered, introducing functional tasks that children usually carry out in their usual environment. To our knowledge, no study has been published in Spain that combines a low dose of treatment with the performance of functional activities at home with the parents. For this reason, we consider assessing the functionality of the affected upper limb in children diagnosed with congenital hemiplegia with moderate manual ability between 4 and 8 years of age after applying low-intensity modified constraint-induced movement therapy (50 h) at home

### **2. Materials and Methods**

This is a case series, prospective and longitudinal study with non-probability sampling (clinical.gov NCT02178371). The study was approved (060-13) by the ethics committee of the CEU-San Pablo University of Madrid in accordance with the Declaration of Helsinki of the World Medical Association. Before initiating the study, an informed consent form was given to the children's families to participate, which guaranteed the right to withdraw from the study at any time, if required by the participants.

The inclusion criteria were a medical diagnosis of left/right congenital infantile hemiplegia, age between 4 and 8 years, lack of activity of the affected upper limb, ability to exceed 10◦ extension in the metacarpophalangeal and interphalangeal joint, ability to complete a 20◦ extension of the wrist of the affected upper limb, adequate cognitive development to understand the verbal orders given for the execution of tasks and cooperation in their execution. In the same way, the exclusion criteria were visual problems that prevented the individual from carrying out the intervention, suffering from significant balance disturbances that put the child at risk of falling as a consequence of having the healthy upper limb contained, presenting uncontrolled epilepsy and having received botulinum toxin within 6 months prior to the intervention.

All the children were selected according to the inclusion criteria by their rehabilitation doctor of the "Virgen de la Salud" Hospital in Toledo for the execution of the therapy.

### *2.1. Intervention Method*

The study was carried out over a period of 5 weeks of treatment, containing the healthy upper limb for 2 h per day [11,13] (not continuously) from Monday to Friday. McConnell et al. [31] and Al-Oraibi 2011 [21] used mCIMT for 2 h per day, obtaining positive results, and the children wore the hand containment for 96% of the total dose [13]. Thus, in our study, the children were requested to perform the structured activities for two non-consecutive hours, with the aim of increasing the adherence and avoiding great physical effort. The tasks were separated by at least 30 min of rest to allow the children to concentrate on the next activities, perform properly and stay motivated. The families were advised to set aside one hour in the early afternoon and another hour in the late afternoon to ensure that the child was attentive, frustration-free and effort-tolerant. The families were also instructed to run a full hour, repeating the activity and designing a story, in which the child was the protagonist and the activity was an enjoyable game to complete. Each of the proposed hours could be divided into periods of 30 min to improve the child's attention and motivation in case the child was tired during the treatment. We chose this minimum period of time, since it is the smallest dose of mCIMT used to treat babies aged 3 to 8 months with babyCIMT, in order to ensure sustained attention to activity [37]. This minimum dose of 30 min was used also in a systematic review [11]. When dealing with children from 4 to 8 years old, attention is greater and maintained for a longer time, which allows establishing a continuity of 1 h of intervention. The intervention was carried out by the family at home. Previously, an informative meeting was held with the parents, in which all the details of the study were explained to them and a program of unimanual activities was given to them to be executed with the affected upper limb every treatment week (Table 1). The activities were programmed to work different movements that were limited in the affected upper limb: shoulder flexion, elbow extension, forearm supination, wrist extension and grasp. Each activity was repeated for around 15 min to obtain a lesson about a functional strategy to use in their usual activities. Each intervention hour was completed with four activities (Table 1), which were repeated in the second hour to increase the movement learning and the possibility to modify and obtain new functional strategies to perform the activity. The same activities were performed every week (first and second hours from Monday to Friday) to improve the movement (shoulder flexion, forearm supination, elbow extension, wrist extension and grasp–release).


**Table 1.** Example of "modified Constraint-Induced Movement Therapy" (mCIMT) designed tasks.


**Table 1.** *Cont.*

The parents were instructed to correctly carry out the intervention avoiding possible errors during the treatment protocol. The weekly exercises, the containment technique and its correct use were also taught to them. The treatment was only initiated when the families and children were confident about it. The family and therapist met every week to assess the activities and make adjustments if necessary. during the treatment protocol. The weekly exercises, the containment technique and its correct use were also taught to them. The treatment was only initiated when the families and children were confident about it. The family and therapist met every week to assess the activities and make adjustments if necessary.

The containment applied was partial, such as a glove or a bandage [21]. In this way, manipulation with the healthy hand was prevented and the wrist and elbow joints were also free to allow the child to react effectively to an external disturbance (Figure 1). The containment applied was partial, such as a glove or a bandage [21]. In this way, manipulation with the healthy hand was prevented and the wrist and elbow joints were also free to allow the child to react effectively to an external disturbance (Figure 1).

**Figure 1.** Child with left hemiplegia wearing a bandage as a partial containment in the right hand (dominant hand). In this task, the child is working the grasp–release action. **Figure 1.** Child with left hemiplegia wearing a bandage as a partial containment in the right hand (dominant hand). In this task, the child is working the grasp–release action.

Each week, the therapist and the parents of the participants had a meeting to follow-up the tasks and solve problems. The follow-up with the families was conducted online, reviewing all the activities and modifying those that were too difficult for the child, maintaining great therapist– family–child feedback. The activities were designed according to the limitations of the children and with simplicity in order for the latter to be able to perform them. In the follow-up, the activities that Each week, the therapist and the parents of the participants had a meeting to follow-up the tasks and solve problems. The follow-up with the families was conducted online, reviewing all the activities and modifying those that were too difficult for the child, maintaining great therapist–family–child feedback. The activities were designed according to the limitations of the children and with simplicity in order for the latter to be able to perform them. In the follow-up, the activities that were not adequate were valued and changed for others considering the interest of the child.

were not adequate were valued and changed for others considering the interest of the child. The parents were asked to fill out a table with the performed tasks, completing these with photographs and videos of the tasks carried out daily and the execution time to complete 96% of total The parents were asked to fill out a table with the performed tasks, completing these with photographs and videos of the tasks carried out daily and the execution time to complete 96% of total dose [13], in this case, 48 h (total dose: 50 h) (Figure 2).

dose [13], in this case, 48 h (total dose: 50 h) (Figure 2).

**Figure 2.** Weekly follow-up for the families. **Figure 2.** Weekly follow-up for the families. **Figure 2.** Weekly follow-up for the families.

#### *2.2. Data Collection 2.2. Data Collection 2.2. Data Collection*

intervention (Figure 3).

Four assessments (Basal (T0, before therapy), T1 (Assessment 2, after Week 1 of therapy), T2 (Assessment 3, after Week 2 of therapy) and T3 (Assessment 4, after Week 5 of therapy)) were performed to measure the study variables and to compare the results before, during and after the Four assessments (Basal (T0, before therapy), T1 (Assessment 2, after Week 1 of therapy), T2 (Assessment 3, after Week 2 of therapy) and T3 (Assessment 4, after Week 5 of therapy)) were performed to measure the study variables and to compare the results before, during and after the intervention (Figure 3). Four assessments (Basal (T0, before therapy), T1 (Assessment 2, after Week 1 of therapy), T2 (Assessment 3, after Week 2 of therapy) and T3 (Assessment 4, after Week 5 of therapy)) were performed to measure the study variables and to compare the results before, during and after the intervention (Figure 3).

**Figure 3.** Representation of the assessment number in the 5 weeks of treatment (mCIMT). **Figure 3.** Representation of the assessment number in the 5 weeks of treatment (mCIMT).

**Figure 3.** Representation of the assessment number in the 5 weeks of treatment (mCIMT). 2.2.1. Studied Variables and Measuring Instruments 2.2.1. Studied Variables and Measuring Instruments

avoiding trunk compensations to gain greater joint width.

Grasp Strength in the Affected Hand

Grasp Strength in the Affected Hand

2.2.1. Studied Variables and Measuring Instruments Quality of Movement and Bimanual Dexterity of Both Upper Extremities Quality of Movement and Bimanual Dexterity of Both Upper Extremities

Quality of Movement and Bimanual Dexterity of Both Upper Extremities The quality of upper extremity test (QUEST) [38–40] was used, validated for children with neuromotor dysfunction with spasticity from 18 months to 8 years of age. It provides a numerical value that is obtained from the mean of the percentages in 36 items distributed in four categories: The quality of upper extremity test (QUEST) [38–40] was used, validated for children with neuromotor dysfunction with spasticity from 18 months to 8 years of age. It provides a numerical value that is obtained from the mean of the percentages in 36 items distributed in four categories: dissociated movements, grasp, weight bearing and protective extension of both extremities. It takes a value from 0 to 100 and can be expressed in percentages (%). The quality of upper extremity test (QUEST) [38–40] was used, validated for children with neuromotor dysfunction with spasticity from 18 months to 8 years of age. It provides a numerical value that is obtained from the mean of the percentages in 36 items distributed in four categories: dissociated movements, grasp, weight bearing and protective extension of both extremities. It takes a value from 0 to 100 and can be expressed in percentages (%).

dissociated movements, grasp, weight bearing and protective extension of both extremities. It takes Active Extension of the Wrist and Active Supination of the Forearm in the Affected Upper Limb Active Extension of the Wrist and Active Supination of the Forearm in the Affected Upper Limb

a value from 0 to 100 and can be expressed in percentages (%). Active Extension of the Wrist and Active Supination of the Forearm in the Affected Upper Limb Both variables were measured with an arm goniometer [41], making three measurements for each variable and selecting the best result. The measurements were made with the child sitting. Wrist Both variables were measured with an arm goniometer [41], making three measurements for each variable and selecting the best result. The measurements were made with the child sitting. Wrist extension was measured with elbow flexion, with the child leaning on a table to decrease muscle tension and associated reactions. Supination was measured with the forearm close to the body, avoiding trunk compensations to gain greater joint width. Both variables were measured with an arm goniometer [41], making three measurements for each variable and selecting the best result. The measurements were made with the child sitting. Wrist extension was measured with elbow flexion, with the child leaning on a table to decrease muscle tension and associated reactions. Supination was measured with the forearm close to the body, avoiding trunk compensations to gain greater joint width.

extension was measured with elbow flexion, with the child leaning on a table to decrease muscle tension and associated reactions. Supination was measured with the forearm close to the body,

This was measured using a hand dynamometer [18] with a scale between 0 and 150 that expressed grasp strength in PSI (pound per square inch, 1 psi = 0.0703 kg/cm2). The test was

expressed grasp strength in PSI (pound per square inch, 1 psi = 0.0703 kg/cm2). The test was performed with the child sitting on a chair in front of a table with the healthy hand under the table and the affected forearm resting on the table to give stability to the upper limb (Figure 4). The child

### Grasp Strength in the Affected Hand

This was measured using a hand dynamometer [18] with a scale between 0 and 150 that expressed grasp strength in PSI (pound per square inch, 1 psi = 0.0703 kg/cm<sup>2</sup> ). The test was performed with the child sitting on a chair in front of a table with the healthy hand under the table and the affected forearm resting on the table to give stability to the upper limb (Figure 4). The child was asked to press the dynamometer with the affected hand (global grasp) as hard as possible to obtain the best measurement. *Children* **2020**, *7*, x FOR PEER REVIEW 7 of 20 was asked to press the dynamometer with the affected hand (global grasp) as hard as possible to obtain the best measurement.

**Figure 4.** Grasp strength measurement with a hand dynamometer. **Figure 4.** Grasp strength measurement with a hand dynamometer.

Spontaneous Use, Dynamic Positioning of the Affected Upper Limb, Grasping and Releasing Action (Wrist Position in Neutral Flexion–Extension), and Level of Functionality and Integration of Spontaneous Use, Dynamic Positioning of the Affected Upper Limb, Grasping and Releasing Action (Wrist Position in Neutral Flexion–Extension), and Level of Functionality and Integration of the Affected Upper Limb in Various Activities of Daily Living

the Affected Upper Limb in Various Activities of Daily Living The Shriners Hospital for Children Upper Extremity Evaluation (SHUEE) [40,42] was used to obtain the values in the four measurements. This evaluation consists of videotaping the children while they execute a series of tasks to observe the functionality and the joint alignment of the affected upper limb, and it has been validated for use in children with hemiplegia aged between 3 and 18 The Shriners Hospital for Children Upper Extremity Evaluation (SHUEE) [40,42] was used to obtain the values in the four measurements. This evaluation consists of videotaping the children while they execute a series of tasks to observe the functionality and the joint alignment of the affected upper limb, and it has been validated for use in children with hemiplegia aged between 3 and 18 years. The results are expressed in percentages, with 100% being the best result.

years. The results are expressed in percentages, with 100% being the best result. The level of functionality and participation of the patient's upper limbs was determined through the SHUEE assessment as dependent, assisted or independent. Activities of daily living, such as dressing upper limbs, dressing lower limbs, buttoning, putting on socks, putting on shoes, putting The level of functionality and participation of the patient's upper limbs was determined through the SHUEE assessment as dependent, assisted or independent. Activities of daily living, such as dressing upper limbs, dressing lower limbs, buttoning, putting on socks, putting on shoes, putting on splints and personal hygiene were assessed.

#### on splints and personal hygiene were assessed. *2.3. Statistical Analysis*

**Independent** 

and 50% females, diagnosed with congenital hemiplegia.

**3. Results** 

*2.3. Statistical Analysis*  Statistical analyses were performed using [43]. Given the small sample size, we used nonparametric tests. First, a Friedman's test was used to evaluate the existence of statistical significance for the assessments performed at different times in each variable. Subsequently, a Wilcoxon test (paired samples) was performed on those variables that presented statistical significance, in order to observe statistically significant differences between paired assessments. Significance was set at a *p*value of 0.05. The results are shown as medians and interquartile ranges (IQRs) with 95% confidence Statistical analyses were performed using [43]. Given the small sample size, we used non-parametric tests. First, a Friedman's test was used to evaluate the existence of statistical significance for the assessments performed at different times in each variable. Subsequently, a Wilcoxon test (paired samples) was performed on those variables that presented statistical significance, in order to observe statistically significant differences between paired assessments. Significance was set at a *p*-value of 0.05. The results are shown as medians and interquartile ranges (IQRs) with 95% confidence intervals. Those with a *p*-value < 0.05 were considered as significant values. The qualitative variable of "functionality" was turned into a quantitative variable, graduating it in 5 levels, from 0 = worst functionality to 4 = maximum functionality (Table 2).


intervals. Those with a *p*-value < 0.05 were considered as significant values. The qualitative variable of "functionality" was turned into a quantitative variable, graduating it in 5 levels, from 0 = worst **Table 2.** Description of the functionality levels.

Twenty-four children were recruited; 14 of them did not meet the inclusion criteria and two of them declined to participate. The final sample consisted of eight children (Figure 5), with 50% males

Value 2. Uses the healthy upper limb exclusively. Value 3. Uses the upper limb to provide stability. Value 4. Uses both upper limbs to execute the action.

### **3. Results**

Twenty-four children were recruited; 14 of them did not meet the inclusion criteria and two of them declined to participate. The final sample consisted of eight children (Figure 5), with 50% males *Children*  and 50% females, diagnosed with congenital hemiplegia. **2020**, *7*, x FOR PEER REVIEW 8 of 20

**Figure 5.** Flowchart of the sample selection. **Figure 5.** Flowchart of the sample selection.

Of the entire sample, 62.5% had affectation of the left upper limb. The average age was 6 years, with a standard deviation of 1.77 years. After assessing the motor ability of the children, they were classified as Level I in the Gross Motor Function Classification System (GMFCS) [44] and Level II in Of the entire sample, 62.5% had affectation of the left upper limb. The average age was 6 years, with a standard deviation of 1.77 years. After assessing the motor ability of the children, they were classified as Level I in the Gross Motor Function Classification System (GMFCS) [44] and Level II in the manual ability classification system (MACS) [34]. (Table 3)


the manual ability classification system (MACS) [34]. (Table 3) **Table 3.** Baseline characteristics of the participants.

Child 4 5 M Preterm Perinatal stroke II I Left No Sex: F: Female; M: Male. Gest. Week: Gestational Week: preterm (>32 weeks and <36 weeks); term: >36 weeks; MACS: Manual Ability Classification System; GMFCS: Gross Motor Function Classification System. Add. Impair: Added impairment.

Sex: F: Female; M: Male. Gest. Week: Gestational Week: preterm (>32 weeks and <36 weeks); term: >36 weeks; MACS: Manual Ability Classification System; GMFCS: Gross Motor Function Classification

stroke II I Right Speech

stroke II I Left Speech

stroke II I Left No

Child 5 7 F Term Perinatal

Child 7 8 F Preterm Perinatal

Child 8 8 M Term Perinatal

System. Add. Impair: Added impairment.

### *3.1. Quality of Movement of the Upper Limb*

The total score for the quality of movement in the upper limbs exhibited an increase of 94.07%. All the variables that compose it, i.e., dissociated movements, grasp, weight bearing and protective extension, showed statistical significance (*p* ≤ 0.001) in the Friedman test. When the Wilcoxon test was applied, dissociated movements and grasp obtained significant changes for all pairwise comparisons with a *p*-value < 0.03. However, no statistically significant differences were found for weight bearing between the second and third assessments (*p* = 0.14), second and fourth assessments (*p* = 0.10) or third and fourth assessments (*p* = 0.11), nor for protective extension between the baseline and second assessments (*p* = 0.07) or second and third assessments (*p* = 0.07). The results are shown in Table 4.




**Table 4.** *Cont.*

\* Statistically significant when the *p*-value < 0.05; quality of movement measured with quality of upper extremity test (QUEST) scale. Results expressed in percentages (%) as median (IQR).

### *3.2. Grasp Strength*

The Friedman test showed significance in all assessments (*p* < 0.001). All pairwise comparisons between assessments were statistically significant (*p* < 0.05), except between the baseline and second assessments (*p* = 1) (Table 5). The largest increase observed occurred from the third to the fourth measurement, with 1 PSI.


**Table 5.** Results and pairwise comparisons of grasp strength.

\* Statistically significant when *p* value < 0.05. Results expressed in median (IQR) measured in PSI units.

### *3.3. Active Elbow Extension and Forearm Supination*

The Friedman test showed significance in all assessments (*p* < 0.001). Both variables increased their value in each of the assessments carried out, exhibiting an increase of 21◦ for elbow extension between the baseline and fourth measurements (*p* = 0.011), and an increase of 11.50◦ for the supination of the forearm (*p* = 0.011) between the baseline and fourth measurements (Table 6).


**Table 6.** Results and pairwise comparisons of elbow extension and forearm supination.

\* Statistically significant when *p* < 0.05. Results expressed in median (IQR), measured in degrees of movement.

### *3.4. Spontaneous Use, Dynamic Joint Position of the A*ff*ected Upper Limb, Grasp and Release Action (Wrist Position in Neutral Flexion–Extension), and Level of Functionality and Integration of the A*ff*ected Upper Limb in Di*ff*erent Activities of Daily Living*

Spontaneous use increased in all evaluations, reaching 88.87% in the fourth assessment, as did dynamic joint position and grasp–release action with different wrist positions, with 88.20% and 91.67%, respectively. These three variables showed statistical significance in the Friedman test with a *p*-value < 0.001. The pairwise comparison showed that, in spontaneous use, the values of the second and third assessments were not significant (*p* > 0.05) when compared with the values of the fourth measurement. In dynamic joint positioning, there were no significant differences between the second and third assessments (*p* = 0.237). Grasp–release action was only significant between the baseline and fourth assessments (*p* = 0.03) and between the second and fourth assessments (*p* = 0.04). (Table 7).

An increase was observed in all the variables of functionality and integration of the affected upper limb in various activities of daily living (proposed in SHUEE). All the increases were statistically significant (*p* < 0.05) in the Friedman test, except for "buttoning buttons", where *p* = 0.163. In the pairwise comparison of the assessments for the action of dressing the upper limbs, putting on the splints and buttoning buttons, no statistical significance was detected (*p* > 0.05). In the action of dressing the lower limbs, statistically significant differences were obtained for all assessments, except between the baseline and second assessments and between the second and third assessments (*p* ≥ 0.05). In the action of putting on shoes, significant differences were not found between the baseline and second assessments (*p* = 0.32), between the baseline and fourth assessments (*p* = 0.06) or between the third and fourth assessments (*p* = 0.10). In the action of putting on socks, there were no differences

between the baseline and second assessments (*p* = 1.00) or between the third and fourth assessments (*p* = 0.08). Regarding personal hygiene, only the changes between the baseline and fourth assessments and between the second and fourth assessments were significant (*p* = 0.02) (Table 8).


**Table 7.** Results and pairwise comparisons of spontaneous use, dynamic joint position and grasp–release action.

\* Statistically significant when *p* < 0.05. Results expressed in percentages % as median (IQR) measured with SHUEE.




**Table 8.** *Cont.*


**Table 8.** *Cont.*

\* Statistically significant when *p* < 0.05. Results expressed in median (IQR), measured with SHUEE on a scale of 1 to 5.

After analyzing the follow-up tables of the activities completed by each family, it was observed that five children performed the total dose (50 h), two children performed 48 h, and one of them performed 49 h. All of them completed the expected dose (48–50 h of mCIMT).

### **4. Discussion**

The deterioration of hand functionality causes a weakness present in the execution of activities of daily living in children with hemiplegia. There is an alteration compared to the healthy upper limb that manifests in the general slowness of movement, discontinuous movements, variability in the trajectory of the hand with compensations of the trunk and the presence of inadequate coordination in the grasp strength of the affected hand [45]. The improvement in grasp strength and stability occurs from the third to the fourth measurement due to an increase in hand strength. The increase was observed only in the last measurement, which could be due to the need for a longer treatment time (5 weeks of intervention). These children with impaired fine motor adjustment, a lack of finger dissociation and deficient proprioception in their affected hand had greater experience (trial–error) to adapt the grasp to the shape, texture and weight of the object, allowing the execution of a previous thinking strategy (anticipatory control) to achieve precision in the grasp and adjustment of the strength to grasp the object adequately. The improvement of grasp stability and strength allows a functional grasp when picking up objects of different characteristics and holding them while performing selective and precision activities, such as throwing a small ball at a target, keeping a fork steady with the affected hand and bringing food to the mouth during the feeding phase. Most children with unilateral brain injury do not develop adequate grip strength in the affected upper limb to coordinate one-handed activities. There is a pathological pattern or an immature state of grasp for their age, leading to an inadequate synergy of the coordination strength that is related to the deterioration of the manual ability of the affected hand depending on the level of injury [46].

Children obtained significant changes in the functional activities of daily life assessed, except in dressing the upper limbs, putting on splints and buttoning buttons. This could suggest the need for improvements in visuomotor coordination and bimanual coordination, and greater strength and precision in the affected grasp to support objects and to perform the activities, which require great ability in the affected upper limb.

There are few studies that propose a home-based therapy intervention [12,13,17,19,25]. Among such studies, only three of them combine this proposal with a modification based on the application of low doses of treatment [12,19,25]. None of them were developed in Spain; thus, our preliminary results provide interesting and unpublished data of the application of mCIMT.

The advantage of combining a low dose of treatment with the application of therapy in the child's own home is that this modification is better accepted by both parents and the child, as reported by

authors such as Eliasson et al. [12], who showed better rates of parental competence among those who had applied low doses of treatment. Likewise, some children showed higher levels of frustration or low tolerance was shown by both the child and the family with higher doses of treatment. To minimize such feelings, some authors have proposed adapting the original protocol, suggesting the use of the containment only during the intervention period, reducing the dose and using a protocol that is "child friendly" and enhances children's engagement [22,40,47]. Thus, our proposal could positively affect these aspects.

Although the objective of this study was not to analyze the cost–benefit of mCIMT, the positive results obtained in it demonstrate that this type of intervention is also a low-cost treatment compared to the application of botulinum toxin [11,14,48–50].

On the other hand, the age of the participants is also an important aspect to take into account when applying this treatment modality. The results of our study coincide with those of Chen et al. (2016), since the younger children with cerebral palsy responded better to home-based CIMT on some areas of upper limb functions than older children. When the child does not receive treatment, the choice of using the upper limbs to carry out a unimanual action will depend on the characteristics of the injury, levels of disability, experience and level of frustration and motivation in carrying out activities, among other factors. Learning "not to use" the unaffected upper limb by means of mCIMT intervention can provide an increased spontaneous participation of the affected upper limb in unimanual and bimanual tasks [32], observed in all measurements performed by SHUEE evaluation. The greatest increase was observed from the first to the second measurement, reaching 15.56% of the total value obtained in the last measurement, 18.87% being the median. This suggests that, when children do not depend on their dominant hand, they learn to use their affected upper limb early and acquire a greater representation within their body schema, developing functional strategies for the execution of daily tasks that allow them to overcome the "disuse" of the affected upper limb due to a lack of integration. Spontaneous use continued to evolve throughout the intervention as the children overcame the lack of experience of use. The improvement in functional performance was reflected in activities of daily living, where a degree of independence and greater participation of the affected limb was reached in the last measurement for the execution of the tasks of dressing upper limbs and lower limbs; putting on socks, shoes and splints; and personal hygiene. The activity carried out with the greatest ability and participation of both upper limbs was putting on pants, for which the most significant statistical results were obtained in the pairwise comparison of measurements. The increase in participation of the affected limb observed from the first measurement to the last, and in some activities accentuated in the fourth measurement, was due to an increase in the quality of bimanual coordination, as a result of the greater integration of the affected upper limb. The parents of the participants provided information about the use of the affected limb in their usual environment, such as during meals to actively support the healthy limb, without the need for the parents to give a verbal order to the child to use it; execution of school activities; playing (symbolic play with dolls); and in extracurricular activities such as dancing, where greater integration and earlier activation of the affected segment was observed, among others, which allowed reducing frustration and abandoning the disuse of the affected upper limb. The families showed great satisfaction with this protocol, which was reflected in the adherence to the therapy, since the children completed 96% of the total dose, thus suggesting the importance of the family and the setting. The natural environment (home) offered less distress during constraint-induced movement therapy practice for both children with cerebral palsy and their parents. Furthermore, the training schedule can be tailored to fit the family's daily routine. A home-based intervention can also save the family time and money in terms of commuting, and parents can be more involved throughout the process, increasing opportunities for parent–child interaction [51]. There could be a continuity in the maintenance of the gains obtained in the functionality of the affected upper limb after executing the intervention, since the daily treatment session was carried out at home, simulating the situation of normal life for the child, in addition to the design of the treatment protocol to simulate activities of daily living. The improvement in the quality of movement was shown by the greater progress in the results

obtained between the baseline and fourth measurements of the variables of dissociated movements and grasp. Appreciable benefits were obtained in each measurement produced by the acquisition of a more corrected posture of the trunk, head and shoulders in the execution of the grasp activities, present from the second measurement (after a week of treatment with mCIMT). A dynamic joint position occurred in wrist and elbow extension, increasing the median value for wrist extension by 21◦ from the baseline to the fourth measurement. In addition, the value reached in the fourth measurement for the median was 81.50◦ in the active supination of the forearm, allowing for greater control and support of the body structures for the execution of dissociated movements, grasp, weight bearing and protective extension due to the improvement of both active movements. There was a favorable evolution in the dynamic joint position of the affected upper limb due to the gain of active degrees for movement restriction in extension and supination, with an increase in this variable of 10.42% at the end of the treatment. In this way, the activities proposed during the evaluation were performed with greater ease of movement, such as eating a cookie, touching the opposite ear, picking up coins, opening a bottle or throwing a large ball, which require the selective motor control of certain muscles. In comparison with the results obtained for the quality of movement of the affected upper limb in the present study, we highlight an investigation [52] on mCIMT, which showed positive results for the assessment of the quality of movement of motor skills (measured through the QUEST scale) using an intervention protocol of 3 weeks of treatment with an intensity of 6 h per day of restriction and repetitive work. This study demonstrated the effectiveness of the intervention, as it had a larger sample and a control group (18 children with hemiplegia, nine children in the experimental group and nine children in the control group). In 2011, a different study [53], conducted exclusively with a girl with hemiplegia, used mCIMT for one hour per day for two weeks. No significant results were obtained at the end of the treatment, which was thereby prolonged for one more week of intervention. This last assessment showed an increase in the overall percentage of total quality of movement (also measured by the QUEST scale), appreciable in activities that involve the affected upper limb. Thus, the protocol chosen for the intervention and the initial functionality of the patient are two important factors to take into account in increasing the results obtained in the measurements. When comparing the results obtained in the second assessment after 20 h of treatment with those obtained after the treatment (50 h), improvements were detected in the former (20 h), although the greatest improvements and relevant changes were identified in the latter (50 h). At the end of the treatment, the families observed these functional changes (daily activities assessed) at different natural moments. Thus, it would be necessary to apply 50 h of dose instead of 20 h in children aged 4–8 years to increase the affected upper limb functionality, since the manual ability and age would be factors to consider in the dose to apply.

### *Limitations and Future Lines of Research*

As this is an uncontrolled trial [54] due to the absence of a control group, it cannot be guaranteed that the observed response (changes produced throughout the intervention with respect to the baseline situation) is exclusively due to the mCIMT protocol used, since other uncontrolled factors may also have had an influence on it. Therefore, the effectiveness of the mCIMT in increasing the functionality of the affected upper limb seen in the present study cannot be generalized to the population of children with hemiplegia, and thus, the fundamental utility of this study is descriptive. It is important to highlight the statistically significant results obtained in the different studied variables throughout the 5 weeks of the intervention (It can be seen at Figures S1–S11. Progression of different variables in the four assessments over 5 weeks of mCIMT). This suggests that, although there was no control group, it could be inferred that the changes obtained in the improvement of the functionality were due to the efficacy of the treatment, since it was a short period of time, where important differences were observed in the affected upper limb, which is unlikely to occur due to the maturation effect (learning and natural development of the child over time).

In Spain, this is the first study to provide preliminary data on the use of mCIMT. Although the results should be interpreted with caution due to the small sample size, the development of this pilot scheme is the first step for this type of therapy, which has already demonstrated its effectiveness in other contexts; therefore, it should be taken into account by therapist and researchers who develop their work with this type of children in Spain.

This study leads us to open different lines of research, such as including a control group to assess the effectiveness of the treatment in the functionality of the affected upper limb, to verify whether the gains obtained after the intervention are maintained over time (6 months or one year after therapy), and to study the influence of age on the obtained results due to neuronal plasticity and active participation of the subject. Likewise, it paves the road for clinicians and researchers to develop new treatment proposals in Spain.

### **5. Conclusions**

A low dose (50 h) of modified Constraint-Induced Movement Therapy can increase the functionality of children diagnosed with congenital hemiplegia between 4 and 8 years of age with moderate manual ability.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/2227-9067/7/9/127/s1. Supplementary figures: Figures S1–S11. Progression of different variables in the four assessments over 5 weeks of mCIMT.

**Author Contributions:** Conceptualization, R.P.-C. and R.-P.R.-G.; methodology, E.P.-P.; software, F.G.-M.S.J.; validation, R.P.-C., P.L.-M. and H.R.-B.; formal analysis, E.P.-P.; investigation, R.P.-C.; data curation, P.L.-M.; writing—original draft preparation, R.-P.R.-G. and R.P.-C.; writing—review and editing, R.-P.R.-G.; visualization, F.G.-M.S.J.; supervision, F.-G.-M.S.J. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Acknowledgments:** The authors thank the collaboration of Carmen López Zarzuela, a rehabilitation doctor of the "Virgen de la Salud" Hospital in Toledo, the parents who agreed to participate and allow the intervention to be carried out successfully due to their great dedication, and especially the children who participated in this study for their continuity and daily improvement.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

*Article*

## **Participation in Everyday Activities of Children with and without Neurodevelopmental Disorders: A Cross-Sectional Study in Spain**

**Nerea Blanco-Martínez 1,2,**† **, Laura Delgado-Lobete 1,\* ,**† **, Rebeca Montes-Montes <sup>3</sup> , Nuria Ruiz-Pérez 4,5, Marcos Ruiz-Pérez <sup>6</sup> and Sergio Santos-del-Riego <sup>1</sup>**


Received: 5 August 2020; Accepted: 27 September 2020; Published: 1 October 2020

**Abstract:** Children with neurodevelopmental disorders (NDDs) often report significant difficulties performing activities of daily living (ADLs), which may restrict their daily participation. The aim of this study was to investigate the differences in ADLs participation between children with NDDs and typically developing (TD) children, and to explore the associations between different daily participation contexts. A cross-sectional study was conducted that included twenty children with a medical diagnosis of an NDD and 26 sex- and age-matched TD controls. The daily participation across home, community, school, and instrumental living activities was measured using the Child and Adolescent Scale of Participation (CASP). The results show that children with NDDs engaged in lower participation in all CASP contexts (∆ = 1.7–5.5, *p* < 0.001) and had a significantly higher prevalence of moderate or severe restricted participation than their TD peers (OR = 23.4, 95% CI = 3.6–154.2, *p* < 0.001). Additionally, a strong association was found between the different contexts of participation (*r* = 0.642–0.856). Overall, the children with NDDs experienced significant participation restrictions on their daily activities. This study adds to the growing evidence showing that intervention strategies in this population should adopt a participation-oriented approach.

**Keywords:** neurodevelopmental disorders; autism spectrum disorders; attention deficit and hyperactivity disorder; motor coordination disorder; activities of daily living; participation; occupational therapy

### **1. Introduction**

Neurodevelopmental disorders (NDDs) are a group of conditions with onset in the early developmental period and comprise a broad group of developmental deficits in the brain function that affect physical, social, academic, and occupational functioning [1,2]. These disorders constitute the most frequent conditions for disability and participation restriction during childhood and are more frequent in males than females [3]. The estimated prevalence of NDDs in school-aged children worldwide

ranges from 4 to 13%, but these data vary across countries and estimation methods [3–6]. In most of the cases, the NDDs persist into late adolescence and adulthood, with persistent consequences for daily living functioning [6]. The most prevalent and usually reported NDDs in childhood are attention deficit and hyperactivity disorder (ADHD), developmental coordination disorder (DCD), and autism spectrum disorders (ASDs). In a comprehensive review on ADHD prevalence conducted by Polanczyk et al., it was reported that 9.5% of children had a lifetime diagnosis of ADHD worldwide during the past three decades [7], while this prevalence ranges between 4.1% and 8.8% among Spanish children and adolescents [8–10]. Reported estimates of the prevalence of DCD in schoolchildren usually range between 5 and 6% [11–13] but recent studies have found that up to 12.0% of Spanish children are at risk of DCD [14,15]. While the prevalence estimates of ASDs are significantly lower, with approximately 1–2% of children presenting with an ASD, the consequences of these disorders in everyday living are much more severe and restrictive, thus making ASDs some of the most studied neurodevelopmental conditions [9,16–19]. Overall, approximately 9.4% of Spanish children have a neurodevelopmental condition that interferes with their functional development [9].

Children with NDDs face significant difficulties in activity performance from early childhood onward, particularly in self-care activities, play and mobility, social cognition, and instrumental activities [20–24]. However, not only activity performance is restricted in these children but daily participation is compromised as well. Participation in meaningful activities of daily living (ADLs) is defined as involvement in life contexts or situations, where the International Classification of Functioning, Disability and Health (ICF) of the World Health Organization consider this aspect to be a crucial part of children's healthy psychological, emotional, and functional development [25].

Additionally, satisfactory participation in a particular daily context is hypothesized to be intimately related to participation in other daily contexts, and thus, restrictions or difficulties in one occupational area are expected to lead to restrictions in other activities of daily living as well [26–28]. This hypothesis is supported by research that shows an interrelation in participation between different daily contexts in typically developing (TD) children [29–31]. Moreover, performance difficulties or participation restrictions are rarely present in only one particular area but are reported in most daily contexts and activities [32].

Several studies have reported that motor, social, behavioral, and processing impairments of children with NDD lead to participation restrictions, and that this population is more likely to show lower and less meaningful participation in daily contexts than typically developing children [32–36]. Three literature reviews using the ICF have concluded that more studies should be conducted to explore the consequences of ADHD, DCD, and ASDs in the context of participation restrictions [35–37]. Investigating potential limitations in daily, naturalistic contexts is relevant for intervention strategies as well, as scientific research shows that activity- and participation-oriented approaches should be used to promote daily performance and family participation in children with neurodevelopmental conditions [11,38–42]. However, very few studies have focused on daily participation differences in Spanish children with and without NDDs. To the best of our knowledge, only two studies have explored social participation among children with ADHD, ASDs, or co-occurring disorders in Spain [43,44], but participation in self-care, instrumental, or community contexts was not assessed. Thus, little is known about daily participation restrictions in Spanish children with neurodevelopmental conditions. Additionally, exploring how children with and without typical development perform and participate during their daily activities is crucial for designing meaningful interventions [45].

Daily participation is influenced by cultural and country constraints, where this effect may differ between participation contexts, even between similar European countries. For instance, a recent study found significant differences in daily participation in self-care activities between typically developing Spanish and Dutch children [12]. Another study found that Spanish boys participated less than German boys in self- and house-care activities [46]. While this research focused on typically developing children, it was expected that differences in daily participation patterns would be found in children with neurodevelopmental conditions as well. As there is no information regarding differences in

daily participation in Spanish children with and without NDDs, a study that evaluated participation restrictions between both groups in Spain was needed. Therefore, the aims of this study were (a) to explore differences in participation between Spanish children with and without NDDs and (b) to explore the association between participation in different daily contexts.

### **2. Materials and Methods**

### *2.1. Research Ethics, Procedure, and Participants*

A multicenter, comparative cross-sectional study was carried out. This study was approved by the Autonomic Research Ethics Committee of Galicia, Spain (code 2018-544). All participants consented to take part in the study anonymously and confidentially.

The total sample comprised forty children aged 5 to 12 years. This sample size was estimated in order to measure the differences in ADLs participation between children with and without developmental disorders with a minimal bias (α < 0.01, power (1 − β) > 0.99) [31]. Children with NDDs (*n* = 20) were recruited from two private rehabilitation centers in Vigo and A Guarda (Spain). Children in this group had a medical diagnosis of a neurodevelopmental condition (girls = 35.0%; mean age = 7.8 years, standard deviation (SD) = 1.7 years; ASDs = 50.0%, motor coordination/psychomotor disorders = 25.0%, ADHD = 15.0%, pervasive developmental disorder/not otherwise specified = 10.0%). A second sample of children without developmental or learning disorders (TD group; *n* = 20) with a similar sex and age distribution was recruited to serve as a sex- and age-matched control group (girls = 35.0%; mean age = 8.7 years, SD = 1.9 years). Children in this group attended three different mainstream schools in Vigo (Spain) and were excluded beforehand if their parents reported a medically diagnosed neurodevelopmental condition. Table 1 shows the sociodemographic characteristics of both groups. Children of both groups were similar in age, sex distribution, education track, and type of school.


**Table 1.** Sociodemographic characteristics of the sample (*n* = 40).

TD—typically developing, NDDs—neurodevelopmental disorders, SD—standard deviation, NA—not applicable.

Parents of the participants received the Child and Adolescent Scale of Participation (CASP) questionnaire as well as an informative letter about the study between December 2018 and June 2019 via school or rehabilitation center intermediation. The letter included the e-mail address and telephone number of the first author such that parents could reach the research team for clarification of the study or the questionnaire. Only parents who consented to anonymously participate completed and returned the CASP to the schools or rehabilitation centers from where they were retrieved by the first author.

### *2.2. The Child and Adolescent Scale of Participation*

The Spanish parent-reported version of the CASP was used to measure daily participation and participation restrictions [31,47]. This scale is part of the Child and Family Follow-up Survey, which is a parent-reported measure informed by the ICF that was originally developed by the occupational therapist Gary Bedell to monitor the needs and outcomes of children and adolescents with an acquired brain injury and their families. The CASP has been validated in multiple settings across different countries and contexts with children and adolescents with other conditions, including NDDs [31]. The CASP has reported an excellent internal consistency and temporal stability (Cronbach's alpha = 0.96; intraclass correlation (ICC) = 0.94) and a good convergent validity with the Pediatric Evaluation of Disability Index (*r* = 0.51–0.75) and the Child and Adolescent Scale of Environment (*r* = 0.43–0.57) [31].

The CASP consists of 20 items and it is available in both parent-reported and self-administered versions. The purpose of this scale is to measure the extent of participation and participation limitations of children and adolescents in comparison to an age-expected performance in four different contexts (home = six items, community = four items, school = five items, and home and community living activities = five items) [31,47].

While "home" and "community" scales cover social-oriented or self-maintenance activities within the home and community, respectively, the "home and community living activities" scale refers to those activities that support daily life within the home and community, which often require more complex tasks and interactions (i.e., instrumental activities) [26,31]. In the present study, we did not include item 20—work activities and responsibilities (e.g., completion of work tasks, punctuality, attendance, and getting along with supervisors and co-workers)—as the Spanish legal working age is 18 years, and therefore none of the children enrolled in the study were expected to participate in that activity. This decision was made based on the recommendation of two Spanish occupational therapists who independently revised the items.

Each item describes a daily activity and it is rated on a four-point Likert scale (age expected = 4, somewhat limited = 3, very limited = 2, unable to participate = 1). An additional response of "not applicable" is available if the item is not appropriate for the child's age, and items rated as "not applicable" do not receive a score. The item scores are summed and divided by the maximum possible score based on the number of items scored. This score is then multiplied by 100 such that total scores for the subscales and total scale range from 0 to 100, where higher scores indicate a greater level of participation [48].

In addition to the standard analysis of the CASP, two different item-level score analyses of the scale were conducted in the present study to analyze the severity of participation restrictions in each context and for global ADLs [48]. First, item scores of 1, 2, or 3 were considered to explore mild or moderate participation limitations between groups. For instance, if a child was rated as "participation somewhat limited" or lower in at least one activity, they would be defined as presenting with at least mild participation restrictions. Second, we explored moderate or severe participation limitations by considering item scores of 1 or 2. In these cases, if a child was rated as "participation very limited" or "unable to participate," they would be defined as presenting with moderate or severe participation restrictions. Therefore, using the standard analysis of the CASP we could compare the level of daily participation between children with and without NDDs, while using the categorized item-level scores, we could explore how many children showed mild-to-severe participation restrictions in each context.

### *2.3. Data Analysis*

The sample size estimation was performed using G\*Power version 3.1.9.4. (Heinrich-Heine-Universität Düsseldorf, Düsseldorf, Germany) [49]. The statistical analyses were conducted using SPSS version 25 (SPSS Inc., Chicago, IL, USA). Prior to conducting the analysis of the data, the internal consistency of the CASP in the sample was tested using Cronbach's alpha to ensure that the removal of item 20 did not alter the reliability of the questionnaire. Values of 0.7 or higher were considered indicators of good internal consistency. The data were examined to determine whether it had a normal distribution using visual inspection, skewness, and kurtosis [50].

Student *t*-tests for independent samples were used to analyze the differences in participation scores in the CASP total and subscale scores between children with and without NDDs. The effect size of these differences was estimated with Glass's delta (∆) using the standard deviation of the typically developing group [51]. Differences in the prevalence of participation restrictions in the different contexts and global participation between both groups were analyzed using chi-squared tests. Additionally, the odds ratios (ORs) and OR 95% confidence of intervals (95% CIs) were calculated to estimate the risk for participation restrictions between children with and without NDDs [51]. Finally, the association between the different contexts of participation was examined in both the total sample and within groups using Spearman correlation coefficients.

### **3. Results**

The internal consistency values of the CASP were adequate for both the overall sample (Cronbach's alpha = 0.9) and for the two groups of participants (TD = 0.7, NDDs = 0.9). Most of the items were scored, indicating that the activities were relevant for the child, and only item 9, item 14, and item 19 received no applicable scores (*n* = 1, *n* = 1, and *n* = 14, respectively). There were no significant differences in the no applicable scores in those items between both groups (*p* = 0.311–0.320).

As shown in Table 2, significant and strong differences were found in all contexts of participation between children with and without NDDs. Community participation, home participation, and general participation were the contexts that revealed stronger differences between groups (∆ = 4.0–5.5, *p* < 0.001).


**Table 2.** Participation in activities of daily living in children with and without NDDs (*n* = 40).

TD—typically developing, NDDs—neurodevelopmental disorders, SD—standard deviation, ∆—effect size, CASP—Child and Adolescent Scale of Participation.

Children with NDDs showed a higher prevalence of participation restrictions or limitations in all contexts (*p* < 0.01). While up to 35.0–60.0% of children with typical development reported a mild or moderate participation limitation in at least one ADL (see Table 3), all the children in the NDDs group reported moderate-to-severe limitations or were unable to participate in at least one instrumental ADL, and most of them faced participation restrictions in home, community, and school settings as well (Table 3). Overall, children with NDDs were 23.4 times more likely to suffer significant participation limitations during their daily living.

Finally, the Spearman correlations exposed significant and moderate-to-strong associations between the different contexts of participation in the total sample and within the NDDs group (Table 4). The TD group showed significant correlations between community and school participation, and between community and home and community instrumental living activities participation (*r* = 0.484–0.604), although the correlation between home and community participation was close to significance (*p* = 0.068). The strongest correlations were found between school and home participation, and between school and community participation in both the total sample and the NDDs group (*r* = 0.641–0.856).


**Table 3.** Participation limitations in daily living in children with and without NDDs (*n* = 40).

ADL—activity of daily living, TD—typically developing, NDDs—neurodevelopmental disorders, OR—odds ratio, CI—confidence interval.

**Table 4.** Associations between the different contexts of participation (*n* = 40).


† *p* = 0.068, ‡ *p* = 0.052, \* *p* < 0.05, \*\* *p* < 0.01, \*\*\* *p* < 0.001.

### **4. Discussion**

Neurodevelopmental conditions that affect a child's typical development are present in approximately 9% of school-aged children in Spain [9]. These children usually face more performance difficulties and participation restrictions than their TD peers, which might have further consequences on their emotional and psychological wellbeing [20–24,32–38]. To the best of our knowledge, no studies on NDDs have so far compared participation restrictions in the different daily contexts in Spanish children with and without NDDs.

### *4.1. Di*ff*erences in Daily Participation between Children with and without NDDs*

Significant differences in daily participation between children with and without neurodevelopmental conditions were found in the present sample. As expected, children with ASDs, motor coordination disorders, or ADHD participated significantly less and faced more participation restrictions in all daily contexts than TD children. Such a pattern is consistent with previous research [32–38]. Our results regarding the level of participation in daily contexts between children with and without developmental disorders are highly similar to the findings of Bedell in the validation study for the CASP [31], who found significant differences between both groups with an analogous outcome in children from North America, Australia, and Israel. Additionally, children with ADHD, DCD, or ASDs usually have co-occurring sensory processing issues [44,52,53], which may further restrict their daily participation [12,44,53–56]. However, it is worth noting that children with neurodevelopmental conditions who are not currently referred to rehabilitation intervention may have different participation patterns. Although most of the NDD children in this study were enrolled in ordinary education, other children with NDDs who are not in inpatient treatment could also show potential differences in their participation and performance [52].

In the present study, participation restrictions in self-care and instrumental ADLs in home and community contexts were significantly higher in children with NDDs. These activities are oriented toward taking care of one's own body and supporting daily life within the home and community [26]. Our findings in these contexts are in line with previous studies that explored children from Europe, America, and Asia, which is of particular interest given that self-care and home maintenance activities may be different between cultures. Self-care and instrumental activities are one of the most problematic contexts for children with ASDs, ADHD, and DCD [21,24,33,35]. In addition, van der Linde et al. found that children with DCD both participated significantly less and performed significantly worse than TD children in self-care activities, while daily performance and participation were interrelated as performance predicted participation [23].

In our sample, children in the NDDs group showed lower levels of participation in the school context. Previous studies have highlighted the specific school-related issues that children with developmental disabilities encounter during their daily performance. Difficulties in numerical and mathematical comprehension, reading, academic fine motor activities, and peer socialization have been reported in children with motor coordination problems [33,57,58]. Children with ASDs face significant issues with learning processing and social participation, which negatively affect their academic performance and participation [44,59]. In addition, children with ADHD are more likely to face academic difficulties and to not pursue high school or university studies due to behavioral and attention deficits [44,60].

Social participation in children with neurodevelopmental disabilities is one of the most restricted contexts and one major concern for the families of these children. While the CASP does not consider a specific context of social participation, this aspect is thoroughly evaluated in all subscales, especially in the home and community participation contexts [31,47], which were significantly restricted in the NDDs group. This outcome is consistent with previous research. While exploring the level of participation in several leisure and social activities, Kaljaˇca et al. found that children with neurodevelopmental disabilities not only participated significantly less than TD children, but also that their activities were mainly stereotypical and highly structured, and were mostly supervised by parents [61]. Social cognition has been reported to be limited in these children as well, and it is associated with motor parameters, which further restricts daily participation [20,24,62]. Similarly, in the study conducted by Kreider et al., the social networks and participation with others of children with ASDs, ADHD, and learning disorders were explored [63]. The study concluded that children with neurodevelopmental conditions show specific limitations in physical, recreational, social, and informal activities with family and/or friends.

Children's participation in family activities in home and community contexts was significantly more restricted in children with NDDs. This finding adds to the evidence that family involvement and functioning is affected in families with children with NDDs, which was previously reported [56,64–66]. Family engagement in the child's activities is important for their overall development, quality of life, and participation involvement [67,68]. Overall, family support is essential for promoting ADLs

performance and participation, as parental self-efficacy and satisfaction predict children's participation involvement and enjoyment [67,69].

An interesting finding of the present study was that 35% of the overall parents (TD = 25%, NDDs = 45%, *p* = 0.320) considered the last item of this context (item 19—using transportation to get around in the community, e.g., to and from school, work, social, or leisure activities) as not applicable, meaning that their child was not expected to participate in that particular activity regardless of the presence of a neurodevelopmental condition. While this could be related to the young age of the participants (<12 years), it would be interesting to examine the subjective relevance perceived by the children about engaging in different daily contexts.

### *4.2. Associations between the Daily Contexts of Participation*

Finally, all daily contexts were strongly associated with each other in our sample. This finding suggests that daily participation is a complex and multifaceted construct comprised of several subareas of different but interrelated occupations, as it has been proposed by previous research [26,30]. Interestingly, school participation showed the strongest correlations with both home and community participation in the total sample and the NDDs group while being the context where children with NDDs faced greater participation restrictions. Additionally, correlations between contexts were different between children with and without NDDs, with them being significantly higher in the latter group. This particular result may suggest that participation restrictions in one daily context may have a greater influence on participation restrictions in other contexts in children with neurodevelopmental conditions in comparison with their typically developing peers, and therefore it may be necessary to pay particular attention when these children first show participation restrictions in their daily activities. Overall, these findings add to the existing literature recommending the use of family-centered and occupation- and participation-based interventions, such as occupational therapy in schools [70–73]. This is of particular relevance given than most children with DCD or ADHD are enrolled in ordinary schools, which do not usually provide this kind of intervention in Spain [52].

Overall, the findings from this study support that children with NDDs present with lower levels of daily participation and greater and more severe participation restrictions in their daily activities than typically developing children. This outcome is consistent with previous research from different countries and cultures, and altogether may point toward a cross-cultural model of participation restrictions in children with NDDs. Additionally, our results suggest that participation difficulties in one particular context may lead to participation difficulties in other contexts, especially in children with neurodevelopmental conditions. Therefore, these findings have implications for research on neurodevelopmental disabilities, both in Spain and in international populations, and they could be used on clinical intervention to assess for difficulties in overall daily participation as soon as a child shows participation restrictions in one particular area, such as school or self-care contexts.

### *4.3. Limitations and Future Research Directions*

This study has several limitations that need to be disclosed. First, a limited sample size was used; although this sample size was calculated to examine differences in daily participation between children with and without NDDs with minimal bias [31], the study findings should be interpreted cautiously and should be supported by larger studies. Nevertheless, this. Second, it is important to note that most of the children in the NDDs group had ASDs, which is usually associated with greater daily challenges than other conditions like ADHD or DCD. This could partially explain the greater variance of the participation abilities observed in this group. Additionally, this imbalance in diagnosis distribution within the NDDs group does not allow for a more in-depth analysis of potential differences between the different diagnostic groups. Third, the participants had a large age range (5 to 12 years). Even though both groups had a similar age distribution, differences in daily participation according to age could not be explored due to the limited sample size; therefore, this may pose a further limitation that should be explored in future studies. Additionally, there are other factors that could further

restrict daily participation, such as psychosocial and behavioral issues, which are present in a large percentage of children with NDDs [34–37,74,75]. Finally, we used a parent-reported questionnaire to assess daily participation, which could be subject to subjectivity. However, parental questionnaires can provide valuable information that would not have been recorded otherwise in a clinical setting [76,77], and therefore parent-based measures, such as the CASP, are useful for reporting information about children's participation in the daily, naturalistic contexts. Future studies should assess differences in participation between larger samples of children with different neurodevelopmental diagnoses that involve considering other environmental, psychosocial, and child-related factors that may be influencing daily participation.

### **5. Conclusions**

Findings from this study support the belief that participation difficulties and restrictions are present in Spanish children with NDDs in comparison with their TD peers. Home, community, and school contexts seem to be particularly affected, which may further restrict the quality of life and future development of children with neurodevelopmental issues. These consequences for daily functioning highlight the need for tailored, participation-oriented intervention strategies.

**Author Contributions:** Conceptualization, N.B.-M., L.D.-L., and S.S.-d.-R.; methodology, N.B.-M. and L.D.-L.; formal analysis, N.B.-M., L.D.-L., and R.M.-M.; investigation, N.B.-M., L.D.-L., R.M.-M., N.R.-P., M.R.-P., and S.S.-d.-R.; resources, N.B.-M. and L.D.-L.; data curation, N.B.-M. and L.D.-L.; writing—original draft preparation, N.B.-M., L.D.-L., and R.M.-M.; writing—review and editing, N.B.-M., L.D.-L., R.M.-M., N.R.-P., M.R.-P., and S.S.-d.-R.; supervision, L.D.-L. and S.S.-d.-R.; funding acquisition, S.S.-d.-R. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Acknowledgments:** The authors would like to thank the participants of the study for their cooperation. The authors would like to extend their gratitude and acknowledgments to the rehabilitation centers and schools that helped with data gathering.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Review* **Effectiveness of Virtual Reality-Based Interventions for Children and Adolescents with ADHD: A Systematic Review and Meta-Analysis**

**Dulce Romero-Ayuso 1,\* , Abel Toledano-González <sup>2</sup> , María del Carmen Rodríguez-Martínez <sup>3</sup> , Palma Arroyo-Castillo <sup>1</sup> , José Matías Triviño-Juárez <sup>4</sup> , Pascual González <sup>5</sup> , Patrocinio Ariza-Vega <sup>1</sup> , Antonio Del Pino González <sup>6</sup> and Antonio Segura-Fragoso <sup>2</sup>**


**Abstract:** This review aims to evaluate the effectiveness of virtual reality-based interventions (VRbased interventions) on cognitive deficits in children with attention deficit hyperactivity disorder (ADHD). A systematic review and meta-analysis were performed according to the PRISMA statement and the Cochrane Handbook guidelines for conducting meta-analyses. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) was used to assess the quality of the evidence. Clinical trials published up to 29 October 2020, were included. The meta-analysis included four studies, with a population of 125 participants with ADHD. The magnitude of the effect was large for omissions (SMD = −1.38; *p* = 0.009), correct hits (SMD = −1.50; *p* = 0.004), and perceptual sensitivity (SMD = −1.07; *p* = 0.01); and moderate for commissions (SMD = −0.62; *p* = 0.002) and reaction time (SMD = −0.67; *p* = 0.03). The use of VR-based interventions for cognitive rehabilitation in children with ADHD is limited. The results showed that VR-based interventions are more effective in improving sustained attention. Improvements were observed in attentional vigilance measures, increasing the number of correct responses and decreasing the number of errors of omission. No improvements were observed in impulsivity responses.

**Keywords:** virtual reality; ADHD; rehabilitation; cognition; attention; impulsivity

### **1. Introduction**

Among neurodevelopmental disorders, one of the most common is attention deficit hyperactivity disorder (ADHD) [1]. It is characterized by the presence of a persistent pattern of inattention and/or hyperactivity and impulsivity that interferes with cognitive functioning and participation in different activities, at least during the last six months. The diagnosis occurs after seven years of age and lasts throughout life [2]. Attention deficits are observed, for example, because the child frequently changes the focus of attention, particularly in monotonous and repetitive activities.

Occasionally, they do not pay attention to details, causing them to make mistakes or omit relevant information in the performance of tasks. It may also appear that they are not listening, even when people speak to them directly. They are easily distracted and forget

**Citation:** Romero-Ayuso, D.; Toledano-González, A.; Rodríguez-Martínez, M.d.C.; Arroyo-Castillo, P.; Triviño-Juárez, J.M.; González, P.; Ariza-Vega, P.; Del Pino González, A.; Segura-Fragoso, A. Effectiveness of Virtual Reality-Based Interventions for Children and Adolescents with ADHD: A Systematic Review and Meta-Analysis. *Children* **2021**, *8*, 70. https://doi.org/10.3390/children 8020070

Received: 20 December 2020 Accepted: 18 January 2021 Published: 21 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

details of activities of daily living (ADL) or lose objects. In addition, when impulsivity and hyperactivity appear, they can make decisions without reasoning, or not respect their turn in a conversation or play, generate difficulties in delaying gratification and inhibiting emotional reactions. This is mainly explained as a neurodevelopmental disorder of the prefrontal lobe, which concerns the development of executive functions in childhood. It includes inhibitory control, working memory, cognitive flexibility, planning, execution of goal-oriented behaviors, as well as self-behavior monitoring [3].

Some authors indicate that there is a long delay between the onset of symptoms and diagnosis and treatment, which affects the children's functional performance, acquisition of skills, and development in a broad sense [4]. There are several systematic reviews of the use of technologies, both for assessment and intervention (serious games (SG), robots, virtual reality (VR), etc.) in children with neurodevelopmental disorders [1,4–7]. However, there are no reviews or meta-analyses on the effectiveness of virtual-reality interventions in children with ADHD. Previous studies suggest that it is necessary to perform specific research with a higher level of evidence of technological interventions [4,8]. Among the different technological systems, the potential of VR is highlighted. It began to be applied in the health field in the 1990s [9]. The differences between the applications of SG and VR systems is difficult, especially in the field of childhood, where the tasks usually consist of playing a game. Regardless of the game itself, there are aspects that clearly define VR: (1) the use of external tools that provide sensory information (mainly visual, auditory, and haptic) to interact with the virtual environment; (2) internal tools, which allow the collection of information on the user's movement and position regarding their interaction with the system (for example, through gloves, trackers, exoskeletons, or a mouse); (3) systems for reproducing graphic images created by the virtual environment; and (4) software and databases that shape objects in the virtual world, using their shape, texture, or movement [8]. Furthermore, VR shares with the brain the mechanism of generating "embodied simulations" [8]. In fact, VR could be considered as an "embodied technology" [8] that provides the sensation of presence and immersion [10], allowing interaction, enhanced by the incorporation of virtual agents [11]. According to the hypothesis of predictive coding, it is understood that VR generates simulations of one's own body in the world, through the developed scenario. It allows exploring and manipulating that environment, improving self-regulation and learning through the representation from the prediction of the internal (of the body itself) and external (environmental) sensory stimuli. These contribute to improving the movements, actions, and emotions adapted to the context [8,12]. Recent developments in augmented reality (AR), which allows the adding of information and the superimposing of information on the real world, facilitate the transfer of learning to everyday life, and achieve a high level of relevance and motivation [1]. In the case of children, both VR and AR have been considered positive technologies, since they improve the quality of experience, motivation, and learning [10]. They have the potential to allow the design of rehabilitation tasks focused on the child and according to the child's interests, increasing their motivation in a safe environment and monitoring the their performance. These make VR a relevant tool in the field of rehabilitation.

In light of the above, the aim of this study was to evaluate the effectiveness of the use of VR-based interventions in children with ADHD, which is the most frequent neurodevelopmental disorder [4].

### **2. Materials and Methods**

The systematic review was conducted according to the Cochrane Handbook for Systematic Reviews [8] (https://handbook-5-1.cochrane.org), and it was informed by the PRISMA Declaration guidelines [10]. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system was used to assess the quality of the evidence [10]. This systematic review has been registered in PROSPERO under number code CRD42020152677.

### *2.1. Search Strategy*

To assess the validity, applicability, and scope of this systematic review and metaanalysis, a search strategy was carried out in the Web of Science (WOS), Scopus, Cochrane and PsycINFO databases from 29 June to 29 October 2020. The terms included in the search are shown in Table 1. This string was applied to the title + abstract + keywords fields. The electronic search was completed with a review of the bibliographic references of the included studies.

**Table 1.** Key Search Terms.


### *2.2. Selection of Studies*

Covidence (a web-based systematic review program that aims to make evidence synthesis a more proficient process, www.covidence.org) was used for the selection of articles. Randomized controlled clinical trials (RCT) or non-randomized controlled clinical trials that conducted VR-based interventions in children with a diagnosis of ADHD and that had been published in English or Spanish were included. Studies using immersive or semi-immersive VR or AR for the improvement of cognitive functioning were also selected. Articles (1) that did not include children or adolescents with ADHD; (2) that did not include VR-based interventions or AR, with robots or simple SG; (3) that were case series; (4) that were RCT protocols or clinical-trial protocols that did not present results; or (5) that focused on the application of VR exclusively to the educational field as a method of teaching a subject, were excluded.

No restrictions were placed on the minimum sample size of the studies or on the follow-up time. Two independent reviewers selected studies and extracted relevant data (A.T.-G. and D.R.-A.). Discrepancies were resolved by a third independent reviewer (M.C.R.-M.).

### *2.3. Data Extraction and Analysis*

Four researchers (P.A.-C., A.T.-G., M.C.R.-M., D.R.-A.) carried out the bibliographic searches, and two reviewers (A.T.-G. and D.R.-A.) subsequently reviewed the relevance of the titles and abstracts according to the inclusion and exclusion criteria. Data extraction was performed by four researchers (P.A.-C., A.T.-G., M.C.R.-M., D.R.-A.). Clinical efficacy was defined by whether study results showed that a test or a treatment improved any symptoms [11]. A data-extraction form was developed that included the following data from each study: article title, country, year, author, type of study, sample size, followup time, main characteristics of the participants, description of the intervention, main outcomes, and conclusions.

### *2.4. Summary Measures*

The objective measures of effect used to assess the effect of the intervention were Conners' Continuous Performance Test (CPT) and the Virtual Classroom Task Assessment (VCTA). The CPT assesses sustained attention and vigilance for a simple task over a time interval [12]. It can be used in children aged four years to adulthood. The child must press a key when a letter appears on the computer screen, except when it is the letter "X". The test lasts approximately 14 min. There are 324 non-X stimuli interspersed with 36 X

stimuli, presented in blocks of trials with interstimulus intervals of 1, 2, and 4 s, which vary between blocks. Results for CPT performance include number of hits, number of omission errors, number of commission errors, hit reaction time (RT), perceptual sensitivity (d'), and response bias (B). The omissions refer to the lack of response to the target stimuli. Therefore, omissions indicate inattention, which can be motivated by a temporary lack of response or by looking away when the stimuli are presented. Commission errors, which refer to the responses given to stimuli other than the target stimulus, are caused by the inability to inhibit motor responses as a consequence of an impulsive response trend. Perceptual sensitivity (d'), which refers to errors due to difficulties in the discrimination of perceptual characteristics, is related to selective attention. Response bias (B) represents an individual's response tendency.

The VCTA [13] is based on a continuous performance test and consists of a virtual system that uses a head-mounted display (HMD) that recreates a classroom with three rows of desks, a blackboard, and the teacher's desk. One of the sides of the classroom is a window that overlooks a playground with buildings, vehicles, and people, while the other three sides are walls. The virtual teacher instructs the children to press the mouse when they see the letter "K" preceded by the letter "A". There are auditory distractors (e.g., footsteps) visual distractors (e.g., a paper plane flying in the classroom), and mixed distractors (auditory and visual), like a car rumbling outside the window. The experiment consists of five blocks (over a period of 100 s each) with 20 targets (AK). Five hundred stimuli were presented throughout the task (500 s). The test registers the number of correct answers (number of cases in which an answer occurred together with target "AK") and commission errors (number of cases in which a joint response occurred with a non-target).

### *2.5. Synthesis of Results (Statistical Analysis Plan)*

The pre and post-intervention means with their standard deviations (SD) were used, and differences between scores after and before intervention (post-pre differences) were calculated. The SD not reported for these differences were calculated by imputing a correlation coefficient that was calculated in studies with the adequate information, from the preand post-SD and the SD of the difference. The weighted mean of these coefficients (*r* = 0.86) was calculated and applied to the rest of the studies. The effect size was determined using the adjusted Hedges G standardized mean difference (SMD) with a 95% confidence interval. The overall effect size was weighted by the sample size of the studies using the inverse variance method and a random-effects model. The 95% confidence interval (95% CI) and statistical significance were calculated using the z test. The magnitude of the effect was interpreted using Cohen's criteria: 0.2: small effect; 0.5: medium; and 0.8: large effect. Satisfactory values ≥ 0.6 were considered [13]. The individual and the combined effect of all studies were plotted by forest plot using RevMan software [14], including assessment of risk of bias for individual studies. Due to the small sample size of the studies included in the meta-analysis, separate effects were calculated for each of the VCTA and CPT subscales: omissions, commissions, correct hits, reaction time, and perceptual sensitivity. All of these scales could not be combined into a single global estimate of the intervention effect.

### *2.6. Assessment of Risk of Bias in Individual Studies*

An independent reviewer (A. S.-F.) assessed the methodological quality of the studies using the items included in RevMan: random sequence generation (selection bias), allocation concealment (selection bias), blinding of participants and personnel (performance bias), blinding of outcome assessment (detection bias), incomplete outcome data (attrition bias), and selective reporting (reporting bias). These items were categorized as 'high', 'low' or 'unclear'.

### *2.7. Assessment of the Degree of Evidence of the Set of Studies*

The GRADE system was used [9] to consider eight factors to assess the possible downgrade or upgrade of the level of quality of the evidence.

The GRADE system defines four levels of quality of evidence: high, moderate, low, and very low. The meta-analysis was based on controlled trials (randomized or nonrandomized), so it was based on a high-quality level of evidence. Based on this, a series of factors were considered to downgrade or upgrade the final level of evidence.

The factors that downgraded the level of evidence were: (1) risk of bias of the set of studies; (2) inconsistency (heterogeneity) between studies; (3) indirect evidence; (4) imprecision; and (5) publication bias. The factors considered that upgraded the level of evidence were: (6) large effect magnitude (SMD > 0.8); (7) dose-response gradient; and (8) control for confounding factors.

### 2.7.1. Risk of Bias of the Set of Studies

The studies that met the selection criteria were assessed with the items included in the review manager, according to the following characteristics: generation of the randomization sequence, concealment of the allocation sequence, blinding of participants and personnel, blinding of the outcome assessment, incomplete outcome data, and selective reporting of results. Each of these items was categorized as high, low, or unclear. Based on the joint assessment of the six items, the risk of bias of the study was categorized as follows: (1) low risk when all items were at low risk of bias; (2) unclear risk when one or more items have unclear risk; and (3) high risk when one or more items were at high risk of bias.

### 2.7.2. Heterogeneity

To measure the degree of variability or heterogeneity in the effects of the intervention between the different studies, the I<sup>2</sup> statistic (% of the variability of SMD attributable to heterogeneity and not to chance) was used, and the chi-square test was used to assess its statistical significance. It was assessed by visual examination of the forest plot and the chi-square statistic. The I<sup>2</sup> was interpreted as absent (0%), low (25%), moderate (50%), or high heterogeneity (≥75%) [14]. The chi-square test was used to assess whether the differences observed between the studies were compatible with simple hazard [9].

### 2.7.3. Indirect Evidence

Direct evidence is understood as that obtained from research that directly compares the interventions of interest, assessed in the type of patients in whom we are interested, and that measures relevant outcomes for patients. The level of evidence decreases when the population studied, the intervention, or the results measured are not adequate.

### 2.7.4. Imprecision

Imprecision was assessed by the calculation of the optimal information size (OIS) [15], which is a conventional calculation to detect an SMD equal to the minimum clinically important difference and the post-SD of the control group. The calculation was performed using the comparison of the post-pre means in independent groups, based on the use of ANOVA of repeated measures (group-time interaction). Accepting an alpha risk of 0.05 and a beta risk of 0.1 (90% statistical power) in a two-sided contrast, the OIS would be 28 subjects in the experimental group and 28 in the control group to detect a standardized SMD effect size = 0.2.

### 2.7.5. Publication Bias

Publication bias was assessed by visual examination of the funnel plot complemented with the DOI plot, both created with METAXL [16]. In addition, the Begg test and the Egger test calculated with Stata, and the Luis Furuya-Kanamori index (LFK) were used [17]. The Begg and Egger tests contrast the null hypothesis of absence of publication bias. Begg uses the rank correlation between the effect of the standardized intervention and its standard error [18]. Egger uses linear regression of the estimate of the intervention effect against its standard error, weighted by the inverse of the variance of the estimate of the intervention effect [19]. The LFK index uses a new graphical method, the DOI plot, to visualize the skew

between studies. An LFK ≤ 1 was considered as no skewness = no publication bias; >1 ≤2 as minor skewness = low risk of publication bias; and > 2 as major skew = high risk of publication bias.

### **3. Results**

### *3.1. Study Selection and Characteristics*

A total of 471 studies were identified. Of these, 123 studies were duplicates and automatically discarded by Covidence. After de-duplicating, 348 manuscripts were reviewed by title and abstract, excluding 245 studies (124 included the wrong population; 75 were non-VR based interventions; 20 were case studies; 6 were protocols without results; and 20 were studies focused on the application of VR exclusively to the educational field), leaving 103 manuscripts for a complete reading. Of these 103 studies, 36 were not clinical trials; 39 included the wrong population (population with only autism spectrum disorder (ASD), ADHD with ASD comorbidity, or other neurodevelopmental disorders (NDD); 11 had a subject age > 18; and 11 used non-VR based interventions. The review of the full texts reported six studies. Of these, four were included in the meta-analysis (Figure 1; Table 2). The two studies not included in the meta-analysis were those of Tabrizi et al. [20] and Bul et al. [21]. Tabrizi et al. conducted a study in order to verify the effectiveness of virtual-reality systems to improve memory in children with ADHD. Eighteen children between seven and 12 years old participated in this study, divided into three groups: experimental, with pharmacological treatment, and control group. The type of sampling was intentional. The intervention consisted of 10 sessions. The virtual environment was a classroom in which different target stimuli appeared, and the child had to remember them at the end of the session. In addition, auditory and visual stimuli were incorporated, demanding greater inhibition of interference in the children. As the intervention advanced, the difficulty of the task also increased, increasing the number of targets and distractors. The results before and after the intervention were compared using the SNAP-4 questionnaire (a test of attention span and working memory (WISC-IV)) and the Raven Progressive Matrix test. After the intervention, the children who obtained the best results were those who followed VR. The results showed that VR-based interventions, similar to pharmacological treatment, improved memory in children with ADHD in both the post-test and follow-up stages. The study by Bul et al. [21] was the only one found that aimed to improve daily life skills, such as planning, time management, and social skills, through an SG named "Plan-It-Commander", in which the child must perform 10 different missions. The sample was composed of 170 children diagnosed with ADHD according to the DSM-IV TR between eight and 12 years old, and of these, 88 of them received the intervention. The period of the intervention was 10 weeks, and it was used as a complementary treatment to a pharmacological one. To know the changes after the intervention (post-pre differences), a time-management questionnaire, the planning subscale of the Behavior Rating Inventory of Executive Function (BRIEF) questionnaire, and the cooperation subscale of the Social Skills Rating System (SSRS) were used [21]. According to the parents' report, the children significantly improved their time-management skills (*p* = 0.04), social skills (*p* = 0.04), and working memory (*p* = 0.002), compared to the control group. The effects of the experimental group were maintained 10 weeks after the intervention.

### *3.2. Characteristics of the Studies Included in the Meta-Analysis*

The four studies finally included in the meta-analysis are summarized in Table 2. Of these, three were RCT [22–24] and one was a non-randomized controlled trial [25]. The type of experimental intervention was immersive VR using a head-mounted display (HMD), and in some studies using a head tracker with 3 degrees of freedom (DOF) [25]. The study presented by Lee et al. [25] also performed EEG recordings, and in Cho's study [22], neurofeedback was used at the same time as VR, although the neurofeedback protocol was not reported. The total number of participants included in the meta-analysis was 125; of

these, 44 were in the experimental group and 81 were in the comparison one. The mean age was 12.9 years. The vast majority were male (92%).

The clinical profile of the participants was characterized by presenting the clinical symptoms for the diagnosis of ADHD according to the DSM-IV. Three of the four studies indicated that they had no previous experience with the use of VR-based interventions, and that all the participants did so voluntarily and could withdraw at any time from the study. *Children* **2021**, *8*, x FOR PEER REVIEW 7 of 20

**Figure 1.** PRISMA flow diagram. ADHD**:** Attention deficit hyperactivity disorder; ASD: autism spectrum disorder; NDD: **Figure 1.** PRISMA flow diagram. ADHD: Attention deficit hyperactivity disorder; ASD: autism spectrum disorder; NDD: Neurodevelopmental disorder; VR: Virtual Reality; SG: Serious games.

*3.2. Characteristics of the Studies Included in the Meta-Analysis* 

The four studies finally included in the meta-analysis are summarized in Table 2. Of these, three were RCT [22–24] and one was a non-randomized controlled trial [25]. The type of experimental intervention was immersive VR using a head-mounted display (HMD), and in some studies using a head tracker with 3 degrees of freedom (DOF) [25].

[22], neurofeedback was used at the same time as VR, although the neurofeedback protocol was not reported. The total number of participants included in the meta-analysis was 125; of these, 44 were in the experimental group and 81 were in the comparison one. The

Neurodevelopmental disorder; VR: Virtual Reality; SG: Serious games.

mean age was 12.9 years. The vast majority were male (92%).

*Children* **2021**, *8*, 70





presented.

#### *3.3. Interventions* All VR-based interventions used HMD systems with immersive, through cognitive

*3.3. Interventions* 

study.

All VR-based interventions used HMD systems with immersive, through cognitive tasks of attention, where the child was instructed to keep their attention focused on the activity, to select targets, and to inhibit responses to target stimuli. In all the studies, the interventions were individual. In Lee's study [25], the setting was a game with a dinosaur. In the other three studies, the virtual environment simulated a classroom. The type of control interventions in the case of the study by Lee et al. [25] received no intervention. In the study by Bioulac et al. [23], there were two comparison groups, one of which received only pharmacological treatment with QUASYM, and the other of which received only supportive psychotherapy and psychoeducation. In the two studies by Cho et al. [22,24], there were two comparison groups, one of which received cognitive rehabilitation through computerized tasks, while the control group received no treatment. The intervention period in three of the four studies was two weeks. Only the most recent study by Biolac et al. [23] lasted six weeks. The number of sessions varied from eight to 12 (Table 2). tasks of attention, where the child was instructed to keep their attention focused on the activity, to select targets, and to inhibit responses to target stimuli. In all the studies, the interventions were individual. In Lee's study [25], the setting was a game with a dinosaur. In the other three studies, the virtual environment simulated a classroom. The type of control interventions in the case of the study by Lee et al. [25] received no intervention. In the study by Bioulac et al. [23], there were two comparison groups, one of which received only pharmacological treatment with QUASYM, and the other of which received only supportive psychotherapy and psychoeducation. In the two studies by Cho et al. [22,24], there were two comparison groups, one of which received cognitive rehabilitation through computerized tasks, while the control group received no treatment. The intervention period in three of the four studies was two weeks. Only the most recent study by Biolac et al. [23] lasted six weeks. The number of sessions varied from eight to 12 (Table 2).

The clinical profile of the participants was characterized by presenting the clinical symptoms for the diagnosis of ADHD according to the DSM-IV. Three of the four studies indicated that they had no previous experience with the use of VR-based interventions, and that all the participants did so voluntarily and could withdraw at any time from the

#### *3.4. Effect of the VR-Based Interventions on the Different Factors of Sustained Attention and Impulsivity in Children with ADHD 3.4. Effect of the VR-Based Interventions on the Different Factors of Sustained Attention and Impulsivity in Children with ADHD*

Figure 2 shows the forest plot with the results of the individual studies and the metaanalysis on omissions. In summary, the four studies with seven comparisons showed a large magnitude of the effect on omissions (SMD = −1.38; *p* = 0.009). Figure 2 shows the forest plot with the results of the individual studies and the metaanalysis on omissions. In summary, the four studies with seven comparisons showed a large magnitude of the effect on omissions (SMD = −1.38; *p* = 0.009).


*Children* **2021**, *8*, x FOR PEER REVIEW 10 of 20

**Figure 2.** The results of the individual studies and meta-analysis on omissions. **Figure 2.** The results of the individual studies and meta-analysis on omissions.

The four studies with eight comparisons showed a moderate magnitude of the effect on commissions (SMD = −0.62; *p* = 0.002) (Figure 3). The four studies with eight comparisons showed a moderate magnitude of the effect on commissions (SMD = −0.62; *p* = 0.002) (Figure 3).

The four studies with seven comparisons showed a large magnitude of the effect for correct hits (SMD = −1.50; *p* = 0.004) (Figure 4).

Finally, the four studies with five comparisons on the reaction time showed a moderate magnitude of the effect (SMD = −0.67; *p* = 0.03) (Figure 5) and a large magnitude of the effect on perceptual sensitivity (SMD = −1.21; *p* <0.001) (Figure 6).


*Children* **2021**, *8*, x FOR PEER REVIEW 11 of 20


**Figure 4.** The results of the individual studies and meta-analysis on correct hits.

**Figure 4.** The results of the individual studies and meta-analysis on correct hits.

**Figure 4.** The results of the individual studies and meta-analysis on correct hits.

the effect on perceptual sensitivity (SMD = −1.21; *p* <0.001) (Figure 6).

the effect on perceptual sensitivity (SMD = −1.21; *p* <0.001) (Figure 6).

Finally, the four studies with five comparisons on the reaction time showed a moderate magnitude of the effect (SMD = −0.67; *p* = 0.03) (Figure 5) and a large magnitude of

Finally, the four studies with five comparisons on the reaction time showed a moderate magnitude of the effect (SMD = −0.67; *p* = 0.03) (Figure 5) and a large magnitude of



#### *3.5. Degree of Evidence from the Set of Studies 3.5. Degree of Evidence from the Set of Studies*

#### *3.5. Degree of Evidence from the Set of Studies*  3.5.1. Risk of Bias of Individual Studies

3.5.1. Risk of Bias of Individual Studies Figure 7 shows the risk of bias for each study in each of the assessed criteria, using three colors: green = low risk; yellow with question mark = unclear; and red = high risk. All studies except the one by Lee et al. showed a low risk in the generation of the randomization sequence; three of the studies showed an unclear risk of bias in the allocation concealment, while in the study of Lee et al., this risk was high. All were at high risk of bias in the blinding of participants and personnel, and three of them in the blinding of outcome assessment. Only one study had a low risk of bias in incomplete outcome data and in 3.5.1. Risk of Bias of Individual Studies Figure 7 shows the risk of bias for each study in each of the assessed criteria, using three colors: green = low risk; yellow with question mark = unclear; and red = high risk. All studies except the one by Lee et al. showed a low risk in the generation of the randomization sequence; three of the studies showed an unclear risk of bias in the allocation concealment, while in the study of Lee et al., this risk was high. All were at high risk of bias in the blinding of participants and personnel, and three of them in the blinding of outcome assessment. Only one study had a low risk of bias in incomplete outcome data and in Figure 7 shows the risk of bias for each study in each of the assessed criteria, using three colors: green = low risk; yellow with question mark = unclear; and red = high risk. All studies except the one by Lee et al. showed a low risk in the generation of the randomization sequence; three of the studies showed an unclear risk of bias in the allocation concealment, while in the study of Lee et al., this risk was high. All were at high risk of bias in the blinding of participants and personnel, and three of them in the blinding of outcome assessment. Only one study had a low risk of bias in incomplete outcome data and in selective reporting.

#### selective reporting. selective reporting. 3.5.2. Heterogeneity

Total heterogeneity was high in omissions (I<sup>2</sup> = 81%; *p* < 0.009) and correct hits (I<sup>2</sup> = 80%; *p* < 0.004). It was moderate in perceptual sensitivity (I<sup>2</sup> = 59%; *p* < 0.04), low in reaction time (I<sup>2</sup> = 28%; *p* < 0.23), and null in commissions (I<sup>2</sup> = 0%; *p* < 0.82). This indicated inconsistency in the results for omissions, correct hits, and perceptual sensitivity.


**Figure 7.** The risk of bias in the individual studies. green = low risk of bias; yellow with question mark = unclear risk of bias; red = high risk of bias. **Figure 7.** The risk of bias in the individual studies. green = low risk of bias; yellow with question mark = unclear risk of bias; red = high risk of bias.

### 3.5.3. Indirect Evidence

3.5.2. Heterogeneity Total heterogeneity was high in omissions (I² = 81%; *p* < 0.009) and correct hits (I² = 80%; *p* < 0.004). It was moderate in perceptual sensitivity (I² = 59%; *p* < 0.04), low in reaction None of the studies found substantial differences among the study population, the intervention or the outcomes measured, and the criteria established in the meta-analysis.

#### time (I² = 28%; *p* < 0.23), and null in commissions (I² = 0%; *p* < 0.82). This indicated incon-3.5.4. Imprecision

sistency in the results for omissions, correct hits, and perceptual sensitivity. 3.5.3. Indirect Evidence None of the studies found substantial differences among the study population, the The total sample size of the studies included in the meta-analysis was 125 for omissions, commissions, and correct hits (44 in the experimental group and 81 in the control group), and 74 for reaction time and perceptual sensitivity (28 in the experimental group and 46 in the control group).

intervention or the outcomes measured, and the criteria established in the meta-analysis.

### 3.5.5. Publication Bias

3.5.4. Imprecision The total sample size of the studies included in the meta-analysis was 125 for omis-Figures 8 and 9 show the assessment of publication bias using the funnel plot and DOI plot. *Children* **2021**, *8*, x FOR PEER REVIEW 14 of 20

There was a certain asymmetry between the comparisons whose SMD was below or

of the plot) had smaller standard errors and therefore larger sample sizes. It could be considered that there is a deficit of results in the lower-right quadrant of the plot. It would be

a consistent bias in the non-publication of small studies with little or no effects.

**Figure 8.** Publication bias: funnel plot. **Figure 8.** Publication bias: funnel plot.

**Figure 8.** Publication bias: funnel plot.

There was a certain asymmetry between the comparisons whose SMD was below or above the mean of all comparisons. Comparisons with larger effects (left of the plot) also had smaller sample sizes (larger standard errors), while those with smaller SMDs (right of the plot) had smaller standard errors and therefore larger sample sizes. It could be considered that there is a deficit of results in the lower-right quadrant of the plot. It would be

a consistent bias in the non-publication of small studies with little or no effects.

There was a certain asymmetry between the comparisons whose SMD was below or above the mean of all comparisons. Comparisons with larger effects (left of the plot) also had smaller sample sizes (larger standard errors), while those with smaller SMDs (right of the plot) had smaller standard errors and therefore larger sample sizes. It could be considered that there is a deficit of results in the lower-right quadrant of the plot. It would be a consistent bias in the non-publication of small studies with little or no effects.

There was a certain asymmetry similar to that described in the funnel plot. The LFK index showed minor asymmetry (LFK = −1.34). The statistical significance of the Begg test was *p* < 0.001, and that of the Egger test was *p* < 0.001.

### **4. Discussion**

The aim of this meta-analysis was to know the effectiveness of VR-based interventions in ADHD children and adolescents. According to Ioannidis [26], the present study tries to overcome the drawbacks that many meta-analyses have. Therefore, our research sought to promote evidence-based practice. Our meta-analysis showed that there are few studies of VR-based interventions aimed at the cognitive rehabilitation of ADHD children. Most VR studies in ADHD populations have focused on validating the assessment of attention in a virtual classroom environment [4,13,27–29]. Furthermore, as in other NDD [30], there is a lack of consensus on the outcome measures used in the different studies, which means that there are few studies that can be compared. However, in our review, we have only included studies with the same outcome measures. In this sense, this represents a strength of our meta-analysis, since this is a consequence of establishing rigorous criteria on the type of outcome measures and type of intervention, with the aim of knowing more clearly the effectiveness of VR-based interventions, and considering the difficulties noted by other meta-analysis that have addressed the effectiveness of psychosocial treatment in ADHD [31]. Moreover, our study attempted to respond to the criticism made by another meta-analysis of meta-analysis of psychosocial treatments in ADHD, in which the authors address this issue as the problem of: *Apples and Apples or Apples and Oranges*? [31].

All the studies except for Lee's [25] used a classroom as a virtual environment and focused on cognitive tasks, similar to those required by the CPT-type sustained attention tests. In this way, there was a notable reduction in the type of demands for vigilance and sustained attention that have been raised in the studies. It is noticeable that this is so when, in addition to attention deficits, children with ADHD have significant executive deficits, such as planning, execution, and supervision of action, which requires the development of self-regulation. In addition, the deficits of these children also affect the scope of activities of daily living, social and recreational activities. Only the study by Bul et al. [21] attempted to address this type of difficulties, although it did so using SG. On the other hand, the study by Tabrizi only tried to address the deficits in working memory in these children, but again in the virtual classroom setting [20].

### *4.1. Effect of VR-Based Interventions on Each Type of Outcome (Omissions, Commissions, Correct Hits, Reaction Time, Perceptual Sensitivity)*

In summary, although there were a limited number of studies, the results suggest that VR-based interventions help to improve the cognitive performance of children and adolescents with ADHD in vigilance and sustained-attention tasks, reducing the number of omissions, and increasing the number of correct responses to the target stimuli with large effect size. Meanwhile, a medium effect on performance was observed in the reaction time to the target stimuli and the number of errors per commission. This suggests that the effect is more notable on vigilance, and less on the improvement of impulsivity or control inhibitory. These results are of interest because they suggest that VR-based interventions could improve inattention symptoms and therefore be very useful in children with ADHD of the inattention subtype, in whom a greater number of omissions and fewer correct answers have been observed [1].

On the other hand, the results of the Bioulac study [23] showed that children who received VR-based interventions can better inhibit distractors. Moreover, they also showed less impulsivity (with a lower number of commissions). The authors of this study also indicated the good acceptability of this type of intervention. However, no improvements were observed by parents according to the results of the ADHD-RS, showing that although there was an improvement in the parameters of the attention tasks, there was no transfer to activities of daily living [32], which suggests that it is necessary to increase the ecological validity of this type of intervention, as mentioned by the authors themselves. The simple fact of modifying the environment of the task does not mean that the task has ecological validity (pressing a button when a stimulus appears). It is necessary to differentiate between tasks and environment and carefully analyze the task and adapt it so it has ecological validity in accordance with age-appropriate demands, and with significant value for the child [33].

### *4.2. GRADE Quality of Evidence*

### 4.2.1. Assessment of Risk of Bias in the Individual Studies

Risk of bias is considered high, as all studies had one or more items with a high risk of bias. For this reason, it is considered to downgrade the level of evidence in omissions, commissions, correct hits, reaction time, and perceptual sensitivity.

### 4.2.2. Assessment of Heterogeneity

In omissions and correct hits, evidence was downgraded a level for heterogeneity because the studies presented a high heterogeneity. Downgrading was not considered for commissions, reaction time, and perceptual sensitivity.

### 4.2.3. Assessment of Indirect Measurement

It was not considered to downgrade the level of evidence by indirect measurements, since all measurements were direct.

### 4.2.4. Assessment of Imprecision

Because the sample size equaled or exceeded the OIS in all outcomes, it was not considered to downgrade one level due to imprecision.

### 4.2.5. Assessment of Publication Bias

Despite the fact that there was a certain degree of asymmetry and risk of publication bias, and the statistical significance of the Begg and Egger tests lead to rejecting the null hypothesis of the absence of publication bias, the LFK index showed less asymmetry (LFK = −1.34). Therefore, it was not considered to downgrade a level due to publication bias.

The criteria used to upgrade the level of evidence were: (1) Large effect size—it was considered to upgrade one level in omissions and correct hits. It was not considered to upgrade a level of evidence in commissions, reaction time, and perceptual sensitivity, since the observed effect showed a moderate or small improvement; (2) Dose-response effect—it was not considered to upgrade the level of evidence because the dose-response effect was not assessable in this meta-analysis (there was not a gradual exposure to the intervention in any of the studies included); and (3) Control of blinding factors—although it was considered that the control of blinding was relatively good in the case of the three RCTs and the non-randomized controlled trial, it was not considered to upgrade a level of evidence, since most of the studies had small sample sizes. Therefore, in summary, the GRADE level of evidence on the effects of VR-based interventions remains as seen in Table 3.

**Table 3.** The assessment of the quality of the studies.


Note: RCT: randomized control trial; CI: confidence interval; SMD: standardized mean difference; (−1) = downgrade one level; (-) = same level.

> This study has a number of limitations. First, it is likely that not all studies were identified, despite using extensive search strategies. Second, the variety of clinical settings, evaluation protocols, and interventions did not allow the findings to be generalized. Third, the methodological shortcomings of the studies and the absence of post-intervention followup did not allow us to know if the effect achieved is lasting. Fourth, the studies included have small sample sizes. Meta-analyses based on studies with small sample sizes could produce heterogeneous effect sizes. In this meta-analysis, heterogeneity was high in some results, indicating an inconsistency in the findings of the various studies, so they should be interpreted with caution.

> Although the number of studies with VR-based interventions as a complementary therapy in ADHD has increased, the vast majority are intra-subject design research [17], without a comparison group [17–19] or cross-sectional studies conducted to find the specific performance in a virtual task [20,21]. The results of our study show the need for more robust RCTs with larger sample sizes and with methodological planning that includes blinding of participants and personnel, blinding of outcome assessments, and allocation concealment. As stated in a recent systematic review, there is a limited number of RCTs; studies on VR-based interventions are scarce, mostly without control groups and with small sample sizes [11], and have very different outcomes. It is necessary to encourage studies that include VR-based interventions with higher methodological quality that allow conclusions to be drawn and provide evidence of their effectiveness.

> Given the advances in VR systems, which allow simulating the body and interoceptive and proprioceptive perception [34], future studies could improve self-regulation in children

with ADHD in different contexts of daily life, and it is recommended that the tasks implemented through VR have ecological validity [33]. Furthermore, it is necessary to develop more studies that address the planning and supervision of the action and organizational skills, since they are symptoms that continue into adulthood and are hardly addressed in current approaches [35], being basic skills for personal autonomy in ADL [33,36]. Likewise, other studies have indicated the need for the technologies used with children with NDD to be flexible and adaptable, given the heterogeneity of the needs of each child [4,37]. Finally, we note that VR-based interventions, in children between six and 11 years old, could make it difficult to for them to distinguish between memories based on VR and real ones [13]; therefore, the use of VR-based interventions should always be supervised by a healthcare professional [9]. In the future, we consider studies in which brain activity is recorded simultaneously with the use of VR-based interventions to be of interest.

### **5. Conclusions**

To the best of our knowledge, this is the first systematic review and meta-analysis on the effectiveness of VR-based interventions in children and adolescents with ADHD. The studies included in this meta-analysis were three RCTs and one non-randomized controlled trial that analyzed the effect of VR-based interventions to improve cognitive functioning in children with ADHD and that were published between 2001 and 2020.

The review analyzed 469 articles. We selected four studies that included immersive or semi-immersive VR-based interventions [22–25]. No RCTs or non-randomized controlled trials with augmented or semi-immersive reality were found. The total number of participants among all the studies included was 125 children and adolescents with ADHD (115 males and 10 females); the ages ranged from eight to 18 years, with a mean age of 12.9 years.

Our study provides relevant results for scientific advancement in the design and implementation of new VR-based interventions. VR-based interventions were effective in improving cognitive performance in ADHD, such as sustained vigilance, which showed a decrease in omissions.

Future RCTs of VR-based interventions should consider the following recommendations: (1) New studies should include other virtual environments alternative to the classroom, such as free-play environments (for example, a school playground, park, etc.), basic activities of daily living, environments where the main demand is social, or the use of the environment, such as that of the instrumental activities of daily living; (2) New interventions should include tasks that require the child to plan and supervise action, adherence to rules, correction of errors, and working memory, because these are core deficits in ADHD; (3) In addition, it would be convenient for the new RCTs to include different ADHD comparison groups, especially due to its prevalence in children with only inattentive subtypes and with hyperactive-impulsive predominance (to date there are none); (4) The new VR-based interventions should allow graduating these cognitive and social demands according to the age of the child and the deficits severity; and (5)new studies should include follow-up measures to determine if the improvement is maintained over time. All of these recommendations will help researchers and clinicians to design studies and tools with greater ecological validity.

**Author Contributions:** Conceptualization, D.R.-A., J.M.T.-J.; methodology, D.R.-A., A.S.-F., A.T.-G., M.d.C.R.-M., P.A.-C.; formal analysis, A.S.-F., D.R.-A.; writing—original draft preparation, D.R.-A., A.S.-F.; writing—review and editing P.G., J.M.T.-J., M.d.C.R.-M., A.T.-G., P.A.-V., A.D.P.G.; funding acquisition, D.R.-A., P.G. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Ayudas para el Apoyo y Fomento a la Investigación en Materia de Igualdad e Inclusión del programa 50 del Plan Propio de Investigación y Transferencia de la Universidad de Granada. Convocatoria 2020 INV-INC83-2020. In addition, it was also funded by the Spanish Ministry of Economy and Competitiveness and by EU FEDER funds under project grant PID2019-108915RB-I00.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


*Review*

## **The E**ff**ectiveness of Hippotherapy to Recover Gross Motor Function in Children with Cerebral Palsy: A Systematic Review and Meta-Analysis**

**Laura De Guindos-Sanchez 1,2, David Lucena-Anton 3,\* , Jose A. Moral-Munoz 3,4,5 and Alejandro Salazar 4,5,6 , Ines Carmona-Barrientos <sup>3</sup>**


Received: 27 July 2020; Accepted: 17 August 2020; Published: 19 August 2020

**Abstract:** Cerebral palsy (CP) is a permanent disorder of the posture and movement, which can result in impairments of gross motor function, among others. Hippotherapy (HPT) is an emerging intervention to promote motor recovery in patients with neurological disorders, providing a smooth, precise, rhythmic, and repetitive pattern of movement to the patient. The main objective of this systematic review and meta-analysis of randomized controlled clinical trials was to analyze the effectiveness of HPT interventions on gross motor function in subjects with CP. The following databases were searched in May 2019: PubMed, Scopus, Embase, and Web of Science. The methodological quality of the randomized controlled trials was assessed using the Physiotherapy Evidence Database (PEDro) scale. A total of 10 studies were analyzed in this review, involving 452 participants. Favorable effects were obtained on the gross motor function (Gross Motor Function Measure-66, standardized mean difference (SMD) = 0.81, 95% confidence interval (CI) = 0.47–1.15, Gross Motor Function Measure-88 dimension A SMD = 0.64, 95% CI = 0.30–0.97, dimension B SMD = 0.42, 95% CI = 0.09–0.75, and dimension E SMD = 0.40, 95% CI = 0.06–0.73). The results obtained in the present review show the potential benefit of HPT intervention in improving gross motor function in children with CP.

**Keywords:** hippotherapy; cerebral palsy; equine-assisted therapy; physical therapy; gross motor function

### **1. Introduction**

Cerebral palsy (CP) is the main source of physical disability in children [1]. The prevalence of CP is 2.11/1000 live births since 1985 in high-income developed countries. Children with CP usually present several limitations in terms of postural control, balance, walking, and gross motor function, as well as sensory and perceptual disturbances, spasticity, visual impairment, mental retardation, epilepsy, etc. [2]. These disorders are responsible for inefficient and ineffective movements and activities and it often leads to limitations in carrying out activities of daily living [2]. The neurodevelopmental therapies are usually used in the neurological rehabilitation of children with CP. These therapies are

focused on decreasing excessive tone, giving the patient a sense of normal position and movement, and easing normal movement patterns [3].

Hippotherapy (HPT) is an equine-assisted therapy that applies the specific movement of horses in the rehabilitation of patients with neurological disorders [4], improving the neurological functions and sensory processes [5,6]. The research in HPT has increased in recent years as a complementary therapy to traditional treatments [6]. HPT is based on two main action mechanisms: (i) the transmission of the warmth and (ii) the transmission of three-dimensional movements with rhythmic impulses from the horse to the patient's body. The pelvis of the patient is moved in a repetitive, rhythmic, and soft pattern, which is similar to the movement carried out during human gait. This three-dimensional movement stimulates balance reactions, improves postural balance and the trunk straightening [4]. This therapy provides movements in all the movement planes, coming from the alternating elevation of the horse's back that originate anteversion/retroversion, elevation/decrease, and lateral movement with rotation [5]. In addition, HPT provides sensory input and induces greater postural control and motor responses [6]. Several favorable physical effects of HPT were found in muscle coordination, muscle tone, balance, posture, strength, endurance, and flexibility, improving gait and patterns of abnormal movement. In addition, it also showed positive improvements at the social, cognitive, and psychological levels [6]. Furthermore, several recent reviews suggested that HPT could be effective for the neurological rehabilitation of subjects with CP: Novak et al. stated that HPT was a successful allied health therapy to improve muscle symmetry in subjects with CP [7]; Mendizábal Alonso [8] suggested that HPT was effective to improve postural alignment in subjects with CP; Martin-Valero et al. [9] also reported benefits in the performance of the activities of daily living and quality of life; and Zadnikar and Kastrin [10] also obtained favorable results on postural balance in subjects with CP.

Regarding the motor function of children with CP, the main aim of therapeutic interventions is to increase the performance of the gross motor skills that are key components of the functional mobility [11]. To the best of our knowledge, only a systematic review carried out in 2012 by Whalen and Case-Smith [12] suggested that HPT could produce benefits on gross motor function in subjects with CP. Therefore, the current evidence through meta-analysis analyzing the use of HPT to recover gross motor function in patients with CP is limited. Consequently, the aim of this systematic review and meta-analysis is to evaluate the effectiveness of HPT for improving gross motor function in children with CP.

### **2. Materials and Methods**

### *2.1. Search Strategy*

The present review was carried out following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) [13] recommendations for systematic reviews. The literature search was carried out using the databases: PubMed, Web of Science (WoS), Scopus, and Embase. The search covered up to May 2019, without a limit in the starting date. It was performed by combining the following keywords: "hippotherapy" and "cerebral palsy". No filters were applied in relation to the publication dates or language, but the results were filtered to obtain only studies that corresponded to randomized clinical trials (RCTs).

### *2.2. Selection Criteria*

The articles included in this review met the following inclusion criteria based on the PICOS model: (P) population: subjects diagnosed with CP; (I) intervention: HPT; (C) comparison: with conventional physical therapy intervention or placebo; (O) outcomes: gross motor function; and (S) study design: RCTs. The exclusion criteria were: (I) studies that involved healthy participants; (II) more than one intervention compared in the study; and (III) an intervention performed using HPT simulators. In addition, we excluded articles in which the intervention was based on therapeutic riding because the instructors may not always be medical professionals using an interdisciplinary team approach [10].

### *2.3. Study Selection Process and Data Extraction*

First, a literature search was conducted in the scientific databases by combining keywords. Afterwards, we identified and excluded the duplicated articles. After this first selection, the titles and abstracts of the articles found were reviewed. Next, a second exclusion process was made of those studies that did not fulfill the inclusion criteria. These articles obtained after this last selection were evaluated in depth to fulfill the specific inclusion criteria. Finally, the studies that form part of this review were included. Two reviewers (L.D.-G.S. and D.L.A.) independently selected, reviewed, and extracted data form the studies. An additional reviewer (I.C.B.) participated in the consensus of the decisions. We extracted the following information from each study: author, year of publication, number of participants from both groups, average age, gender, levels of the gross motor function classification system (GMFCS), type of CP, intervention carried out, frequency, duration, outcomes, measuring instruments, and results.

### *2.4. Assessment of the Methodological Quality of the Studies*

The PEDro [14] was used to assess the methodological quality of the studies. This scale comprises different items in terms of the following domains: performance, selection, information, detection, and attribution bases. A higher score shows a higher methodological quality. A study with a PEDro score of 6 or higher is considered as evidence level 1 (6–8 is good; 9–10 is excellent), and a study with a score of 5 or less is considered as evidence level 2 (4–5 is acceptable; <4 is poor) [15].

### **3. Results**

Once the database searches were completed, using the different keywords, a total of 276 documents were obtained, as shown in Figure 1. Finally, 10 studies met the inclusion criteria for review. *Children* **2020**, *7*, x FOR PEER REVIEW 4 of 15

**Figure 1.** Flow diagram of included articles. **Figure 1.** Flow diagram of included articles.

### *3.1. Methodological Quality of the Studies*

Table 1 shows the PEDro scores achieved by the articles reviewed in this study. Three of ten articles were considered to have a high methodological quality: McGibbon et al. [16]; Kwon et al. [17], and Lucena-Antón et al. [5]. Matusiak-Wieczorek et al. [18] achieved the lowest score. The overall methodological quality was acceptable (average total score = 5.1, range 3–7).


**Table 1.** Analysis of the methodological quality of the studies (PEDro scores).

Range: 0–10. Item 1 is not included in the total score. Item 1: Eligibility criteria; Item 2: Random allocation; Item 3: Concealed allocation; Item 4: Baseline similarity; Item 5: Subject blinding; Item 6: Therapist blinding; Item 7: Assessor blinding; Item 8: >85% follow up; Item 9: Intention-to-treat analysis; Item 10: Between-group statistical comparison; Item 11: Point and variability measures.

### *3.2. Main Characteristics of the Studies Included in the Systematic Review*

Regarding the age of the participants, the highest average age among the control groups was found in the study by McGibbon et al. [16] (8.8 years), while among the intervention groups, it was found in the study by Lucena-Antón et al. [5] (9.6 years). The lowest average age in both groups was presented in the study by Kwon et al. [17] (5.9 and 5.7 years, respectively). In terms of the sample size, the study by Kwon et al. [17] achieved the highest sample size with a total of 91 participants. The overall sample size ranged from 15 to 73 subjects. Table 2 shows the main clinical and demographic characteristics of the participants.

Concerning the different effects analyzed in the different studies, three studies [17,22,23] analyzed the effects of HPT interventions on gross motor function, four studies [17,18,20,23] analyzed the effects on balance, two studies [5,23] analyzed the spasticity, and two studies [16,19] analyzed the muscle activity through electromyography. The main intervention characteristics of the studies included in the systematic review are shown in Table 3.


**Table 2.** Main clinical and demographic characteristics of the participants.

*Children* **2020**, *7*, 106


*Children* **2020**, *7*, 106

McGibbon et al., 2009 [16]

IG: (*n* = 25) CG: (*n* = 19)

Phase 2: CG: Exercises

Phase 2:

Once a week Phase 2: 40 min Phase 2:

36 weeks

on a barrel

IG: (*n* = 6)

One session

Hip adductors muscle activity SEMG

(*p* < 001; *d* = 1.32)

Phase 2: After 12 weeks, 4 of 6 children improved the muscle symmetry of hip adductors




4–8.

#### *3.3. Meta-Analysis of the Study Groups 3.3. Meta-Analysis of the Study Groups 3.3. Meta-Analysis of the Study Groups*  The groups were created according to the measuring instrument used to assess the gross motor The groups were created according to the measuring instrument used to assess the gross motor

*3.3. Meta-Analysis of the Study Groups* 

*3.3. Meta-Analysis of the Study Groups* 

The groups were created according to the measuring instrument used to assess the gross motor function. Accordingly, seven groups were set up: (i) GMFM-66 total scores; (ii) GMFM-88 total scores; and (iii–vii) GMFM-88 dimensions A–E. The groups were created according to the measuring instrument used to assess the gross motor function. Accordingly, seven groups were set up: (i) GMFM-66 total scores; (ii) GMFM-88 total scores; and (iii–vii) GMFM-88 dimensions A–E. The groups were created according to the measuring instrument used to assess the gross motor function. Accordingly, seven groups were set up: (i) GMFM-66 total scores; (ii) GMFM-88 total scores; and (iii–vii) GMFM-88 dimensions A–E. The gross motor function measure (GMFM) is commonly used in neurological rehabilitation to function. Accordingly, seven groups were set up: (i) GMFM-66 total scores; (ii) GMFM-88 total scores; and (iii–vii) GMFM-88 dimensions A–E. The gross motor function measure (GMFM) is commonly used in neurological rehabilitation to function. Accordingly, seven groups were set up: (i) GMFM-66 total scores; (ii) GMFM-88 total scores; and (iii–vii) GMFM-88 dimensions A–E. The gross motor function measure (GMFM) is commonly used in neurological rehabilitation to assess the gross motor function in subjects with CP. The GMFM-66 scale is an updated version of

*Children* **2020**, *7*, x FOR PEER REVIEW 11 of 15

The gross motor function measure (GMFM) is commonly used in neurological rehabilitation to assess the gross motor function in subjects with CP. The GMFM-66 scale is an updated version of GMFM-88. It includes 66 of the original 88 items providing more information to encourage the goal setting process [11]. Both scales include different items that assess how much of an activity can be carried out rather than the quality of performing the activities [25]. Both versions have been validated to evaluate changes in children with CP. A higher score is an indicator of better gross motor function [26]. The gross motor function measure (GMFM) is commonly used in neurological rehabilitation to assess the gross motor function in subjects with CP. The GMFM-66 scale is an updated version of GMFM-88. It includes 66 of the original 88 items providing more information to encourage the goal setting process [11]. Both scales include different items that assess how much of an activity can be carried out rather than the quality of performing the activities [25]. Both versions have been validated to evaluate changes in children with CP. A higher score is an indicator of better gross motor function [26]. assess the gross motor function in subjects with CP. The GMFM-66 scale is an updated version of GMFM-88. It includes 66 of the original 88 items providing more information to encourage the goal setting process [11]. Both scales include different items that assess how much of an activity can be carried out rather than the quality of performing the activities [25]. Both versions have been validated to evaluate changes in children with CP. A higher score is an indicator of better gross motor function [26]. Two studies analyzed the differences in gross motor function using the GMFM-66. The overall assess the gross motor function in subjects with CP. The GMFM-66 scale is an updated version of GMFM-88. It includes 66 of the original 88 items providing more information to encourage the goal setting process [11]. Both scales include different items that assess how much of an activity can be carried out rather than the quality of performing the activities [25]. Both versions have been validated to evaluate changes in children with CP. A higher score is an indicator of better gross motor function [26]. Two studies analyzed the differences in gross motor function using the GMFM-66. The overall GMFM-88. It includes 66 of the original 88 items providing more information to encourage the goal setting process [11]. Both scales include different items that assess how much of an activity can be carried out rather than the quality of performing the activities [25]. Both versions have been validated to evaluate changes in children with CP. A higher score is an indicator of better gross motor function [26]. Two studies analyzed the differences in gross motor function using the GMFM-66. The overall result of this study group was favorable (Figure 2).

Two studies analyzed the differences in gross motor function using the GMFM-66. The overall result of this study group was favorable (Figure 2). Two studies analyzed the differences in gross motor function using the GMFM-66. The overall result of this study group was favorable (Figure 2). result of this study group was favorable (Figure 2). result of this study group was favorable (Figure 2).


**Figure 2.** Forest plot for gross motor function measured by the GMFM-66 scale. **Figure 2.** Forest plot for gross motor function measured by the GMFM-66 scale. **Figure 2.** Forest plot for gross motor function measured by the GMFM-66 scale. Regarding the GMFM-88 scale, it is divided into five dimensions (A: lying and rolling, B: sitting, Regarding the GMFM-88 scale, it is divided into five dimensions (A: lying and rolling, B: sitting,

Regarding the GMFM-88 scale, it is divided into five dimensions (A: lying and rolling, B: sitting, C: crawling and kneeling, D: standing, and E: walking, running, and jumping). The total score ranges from 0 to 100. For the GMFM-88 total score, the overall result of the meta-analysis was not conclusive (Figure 3). Regarding the GMFM-88 scale, it is divided into five dimensions (A: lying and rolling, B: sitting, C: crawling and kneeling, D: standing, and E: walking, running, and jumping). The total score ranges from 0 to 100. For the GMFM-88 total score, the overall result of the meta-analysis was not conclusive (Figure 3). Regarding the GMFM-88 scale, it is divided into five dimensions (A: lying and rolling, B: sitting, C: crawling and kneeling, D: standing, and E: walking, running, and jumping). The total score ranges from 0 to 100. For the GMFM-88 total score, the overall result of the meta-analysis was not conclusive (Figure 3). C: crawling and kneeling, D: standing, and E: walking, running, and jumping). The total score ranges from 0 to 100. For the GMFM-88 total score, the overall result of the meta-analysis was not conclusive (Figure 3). C: crawling and kneeling, D: standing, and E: walking, running, and jumping). The total score ranges from 0 to 100. For the GMFM-88 total score, the overall result of the meta-analysis was not conclusive (Figure 3).


**Figure 3.** Forest plot for gross motor function measured by the GMFM-88 total scores. **Figure 3.** Forest plot for gross motor function measured by the GMFM-88 total scores. **Figure 3.** Forest plot for gross motor function measured by the GMFM-88 total scores. **Figure 3.** Forest plot for gross motor function measured by the GMFM-88 total scores. **Figure 3.** Forest plot for gross motor function measured by the GMFM-88 total scores.

Regarding the different dimensions included in the GMFM-88 scale, the overall result of the meta-analysis was favorable in GMFM-88 dimensions A, B, and E, while the overall result of the Regarding the different dimensions included in the GMFM-88 scale, the overall result of the meta-analysis was favorable in GMFM-88 dimensions A, B, and E, while the overall result of the meta-analysis was inconclusive for GMFM-88 dimensions C and D. The results are shown in Figures Regarding the different dimensions included in the GMFM-88 scale, the overall result of the meta-analysis was favorable in GMFM-88 dimensions A, B, and E, while the overall result of the meta-analysis was inconclusive for GMFM-88 dimensions C and D. The results are shown in Figures 4–8. Regarding the different dimensions included in the GMFM-88 scale, the overall result of the meta-analysis was favorable in GMFM-88 dimensions A, B, and E, while the overall result of the meta-analysis was inconclusive for GMFM-88 dimensions C and D. The results are shown in Figures Regarding the different dimensions included in the GMFM-88 scale, the overall result of the meta-analysis was favorable in GMFM-88 dimensions A, B, and E, while the result of the meta-analysis was inconclusive for GMFM-88 dimensions C and D. The results are shown in Figures 4–8.

**Figure 4.** Forest plot for GMFM-88 Dimension A. **Figure 4.** Forest plot for GMFM-88 Dimension A. **Figure 4.** Forest plot for GMFM-88 Dimension A. **Figure 4.** Forest plot for GMFM-88 Dimension A.


**Figure 5.** Forest plot for GMFM-88 Dimension B. **Figure 5.** Forest plot for GMFM-88 Dimension B. **Figure 5.** Forest plot for GMFM-88 Dimension B.

**Figure 5.** Forest plot for GMFM-88 Dimension B.

**Figure 5.** Forest plot for GMFM-88 Dimension B.

*Children* **2020**, *7*, x FOR PEER REVIEW 12 of 15

**Figure 6.** Forest plot for GMFM-88 Dimension C. **Figure 6.** Forest plot for GMFM-88 Dimension C. **Figure 6.** Forest plot for GMFM-88 Dimension C.


**Figure 7.** Forest plot for GMFM-88 Dimension D. **Figure 7.** Forest plot for GMFM-88 Dimension D. **Figure 7.** Forest plot for GMFM-88 Dimension D.

**Figure 8.** Forest plot for GMFM-88 Dimension E. **Figure 8.** Forest plot for GMFM-88 Dimension E.

**Figure 8.** Forest plot for GMFM-88 Dimension E.

**Figure 8.** Forest plot for GMFM-88 Dimension E.

### **4. Discussion 4. Discussion**

**4. Discussion**  The objective of this systematic review and meta-analysis of RCTs was to analyze the effectiveness of HPT interventions on improving gross motor function in subjects with CP. A total of **4. Discussion**  The objective of this systematic review and meta-analysis of RCTs was to analyze the effectiveness of HPT interventions on improving gross motor function in subjects with CP. A total of ten RCTs were analyzed in the systematic review, involving 452 participants. In view of our results, The objective of this systematic review and meta-analysis of RCTs was to analyze the effectiveness of HPT interventions on improving gross motor function in subjects with CP. A total of ten RCTs were analyzed in the systematic review, involving 452 participants. In view of our results, The objective of this systematic review and meta-analysis of RCTs was to analyze the effectiveness of HPT interventions on improving gross motor function in subjects with CP. A total of ten RCTs were analyzed in the systematic review, involving 452 participants. In view of our results, we could conclude that HPT could be an effective intervention to improve gross motor function in children with CP.

ten RCTs were analyzed in the systematic review, involving 452 participants. In view of our results, we could conclude that HPT could be an effective intervention to improve gross motor function in children with CP. From a clinical perspective, the findings obtained in the present review suggest that HPT stands for an emerging intervention in neurological rehabilitation, which could be used in addition to neurodevelopmental based methods. The findings on the GMFM-66 scale and GMFM-88 dimensions A, B, and E showed that HPT interventions had significant improvements on gross motor function and, more specifically, on the ability to perform lying and rolling, sitting, and walking. We can hypothesize that the rhythmic and symmetrical movement of the horse could stimulate the proprioception and balance reactions. Furthermore, according to Casady and Nichols-Larsen [27], HPT could stimulate the motor learning and subjects could be able to transfer the movement patterns learned from HPT to other usual environments. According to Bertoti [28], and considering that three of four studies [17,18,20,23] did obtain significant effects on balance and two studies [5,23] reported we could conclude that HPT could be an effective intervention to improve gross motor function in children with CP. From a clinical perspective, the findings obtained in the present review suggest that HPT stands for an emerging intervention in neurological rehabilitation, which could be used in addition to neurodevelopmental based methods. The findings on the GMFM-66 scale and GMFM-88 dimensions A, B, and E showed that HPT interventions had significant improvements on gross motor function and, more specifically, on the ability to perform lying and rolling, sitting, and walking. We can hypothesize that the rhythmic and symmetrical movement of the horse could stimulate the proprioception and balance reactions. Furthermore, according to Casady and Nichols-Larsen [27], HPT could stimulate the motor learning and subjects could be able to transfer the movement patterns learned from HPT to other usual environments. According to Bertoti [28], and considering that three of four studies [17,18,20,23] did obtain significant effects on balance and two studies [5,23] reported significant effects on the spasticity of hip adductors, we can suggest that the positive effects obtained we could conclude that HPT could be an effective intervention to improve gross motor function in children with CP. From a clinical perspective, the findings obtained in the present review suggest that HPT stands for an emerging intervention in neurological rehabilitation, which could be used in addition to neurodevelopmental based methods. The findings on the GMFM-66 scale and GMFM-88 dimensions A, B, and E showed that HPT interventions had significant improvements on gross motor function and, more specifically, on the ability to perform lying and rolling, sitting, and walking. We can hypothesize that the rhythmic and symmetrical movement of the horse could stimulate the proprioception and balance reactions. Furthermore, according to Casady and Nichols-Larsen [27], HPT could stimulate the motor learning and subjects could be able to transfer the movement patterns learned from HPT to other usual environments. According to Bertoti [28], and considering that three of four studies [17,18,20,23] did obtain significant effects on balance and two studies [5,23] reported significant effects on the spasticity of hip adductors, we can suggest that the positive effects obtained From a clinical perspective, the findings obtained in the present review suggest that HPT stands for an emerging intervention in neurological rehabilitation, which could be used in addition to neurodevelopmental based methods. The findings on the GMFM-66 scale and GMFM-88 dimensions A, B, and E showed that HPT interventions had significant improvements on gross motor function and, more specifically, on the ability to perform lying and rolling, sitting, and walking. We can hypothesize that the rhythmic and symmetrical movement of the horse could stimulate the proprioception and balance reactions. Furthermore, according to Casady and Nichols-Larsen [27], HPT could stimulate the motor learning and subjects could be able to transfer the movement patterns learned from HPT to other usual environments. According to Bertoti [28], and considering that three of four studies [17,18,20,23] did obtain significant effects on balance and two studies [5,23] reported significant effects on the spasticity of hip adductors, we can suggest that the positive effects obtained on balance and muscle spasticity contributed to improvements in the functional outcomes and, thus, to the significant results obtained in the GMFM-66 and GMFM-88 dimensions.

significant effects on the spasticity of hip adductors, we can suggest that the positive effects obtained on balance and muscle spasticity contributed to improvements in the functional outcomes and, thus, to the significant results obtained in the GMFM-66 and GMFM-88 dimensions. Regarding the intervention characteristics, most studies included more than 30 participants, a high number considering the difficulty to recruit patients with CP. All intervention groups received HPT in addition to physical therapy, and all of them carried out their HPT interventions through a walking pace, except for Alemdaroğlu et al. [23] and Deutz et al. [24] that did not specify it. Most studies used protocols with 8–12 weeks as the total duration and two times/week as the frequency. The session duration used in the studies was around 30 min, with unusual interventions of more than 45 min. The effects found were similar and several authors suggested that longer durations could on balance and muscle spasticity contributed to improvements in the functional outcomes and, thus, to the significant results obtained in the GMFM-66 and GMFM-88 dimensions. Regarding the intervention characteristics, most studies included more than 30 participants, a high number considering the difficulty to recruit patients with CP. All intervention groups received HPT in addition to physical therapy, and all of them carried out their HPT interventions through a walking pace, except for Alemdaroğlu et al. [23] and Deutz et al. [24] that did not specify it. Most studies used protocols with 8–12 weeks as the total duration and two times/week as the frequency. The session duration used in the studies was around 30 min, with unusual interventions of more than 45 min. The effects found were similar and several authors suggested that longer durations could cause fatigue in children, which was not positive for achieving the intended improvements. on balance and muscle spasticity contributed to improvements in the functional outcomes and, thus, to the significant results obtained in the GMFM-66 and GMFM-88 dimensions. Regarding the intervention characteristics, most studies included more than 30 participants, a high number considering the difficulty to recruit patients with CP. All intervention groups received HPT in addition to physical therapy, and all of them carried out their HPT interventions through a walking pace, except for Alemdaroğlu et al. [23] and Deutz et al. [24] that did not specify it. Most studies used protocols with 8–12 weeks as the total duration and two times/week as the frequency. The session duration used in the studies was around 30 min, with unusual interventions of more than 45 min. The effects found were similar and several authors suggested that longer durations could cause fatigue in children, which was not positive for achieving the intended improvements. Therefore, we can suggest that HPT interventions based on 8–12 week programs with sessions of 30– Regarding the intervention characteristics, most studies included more than 30 participants, a high number considering the difficulty to recruit patients with CP. All intervention groups received HPT in addition to physical therapy, and all of them carried out their HPT interventions through a walking pace, except for Alemdaro˘glu et al. [23] and Deutz et al. [24] that did not specify it. Most studies used protocols with 8–12 weeks as the total duration and two times/week as the frequency. The session duration used in the studies was around 30 min, with unusual interventions of more than 45 min. The effects found were similar and several authors suggested that longer durations could cause fatigue in children, which was not positive for achieving the intended improvements. Therefore, we can suggest that HPT interventions based on 8–12 week programs with sessions of 30–45 min two times a week could be proper for children with CP to recover motor function.

cause fatigue in children, which was not positive for achieving the intended improvements. Therefore, we can suggest that HPT interventions based on 8–12 week programs with sessions of 30– Therefore, we can suggest that HPT interventions based on 8–12 week programs with sessions of 30– 45 min two times a week could be proper for children with CP to recover motor function. 45 min two times a week could be proper for children with CP to recover motor function. Concerning the methodological quality of the studies included in the present review, the main limitation was found in the application of double-blind. Blinding of the participants and therapists

45 min two times a week could be proper for children with CP to recover motor function.

was not possible in most studies due to the unconcealable nature of the intervention. In addition, the concealed allocation was only possible in two studies [16,19] and the assessor blinding was carried out by four studies [5,16,17,24]. Nevertheless, the overall quality of the studies was acceptable.

Our results matched with the findings of the systematic review conducted by Whalen and Case-Smith [12] in 2012, in which they stated that HPT could produce benefits on gross motor function in children with CP, but the authors highlighted that the evidence was limited. Other findings were found in different pathologies, such as Down syndrome and autism disorder. De Miguel Rubio et al. [29] suggested that HPT could not be effective to improve gross motor function in subjects with Down syndrome, and Srinivasan et al. [30] analyzed the effects of HPT interventions in subjects with autism disorder, obtaining positive effects on motor skills.

The present systematic review presented some limitations. Potential useful articles that were indexed in other scientific databases could not be included. In addition, the lack of long-term follow-up and the heterogeneity of the protocols suggests the need to unify the HPT intervention programs, specifically, in subjects with CP. Moreover, despite assessing the same outcomes between the different studies included in the review, the statistical comparison was not always possible due to studies used different scales and measuring instruments to assess the clinical differences. Thus, only two studies were included in the meta-analysis. Therefore, the results obtained should be taken with caution since a limited number of studies was analyzed.

### **5. Conclusions**

In conclusion, we could state that HPT interventions were effective to improve gross motor function in subjects with CP. Specifically, favorable results were obtained in the GMFM-66 total scores and GMFM-88 dimensions A, B, and E. Furthermore, positive effects have been showed on balance recovery and muscle spasticity reduction.

Despite the different HPT protocols used, evidence shows that 30–45 min sessions, twice weekly for 8–12 weeks, could produce significant effects on gross motor function in children with CP.

This study can be helpful in neurological rehabilitation of children with CP using HPT interventions, as well as by providing key factors to determine which specific factors of the HPT protocols have a greater weight to achieve the desired effects in future interventions. Nevertheless, it will be necessary to carry out more randomized controlled trials with larger sample sizes and specified protocols.

**Author Contributions:** Conceptualization and methodology: L.D.G.-S., I.C.-B., J.A.M.-M. and D.L.-A.; statistical analysis: A.S.; writing—original draft preparation: L.D.G.-S., D.L.A., A.S. and J.A.M.-M.; writing—review and editing: D.L.-A., J.A.M.-M. and I.C.-B. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Department of Nursing and Physiotherapy (University of Cadiz).

**Acknowledgments:** The authors are grateful for the funding from the Department of Nursing and Physiotherapy (University of Cadiz).

**Conflicts of Interest:** The authors declare no conflict of interest.

### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **Pain and Communication in Children with Cerebral Palsy: Influence on Parents' Perception of Family Impact and Healthcare Satisfaction**

**Inmaculada Riquelme 1,2,\* , Álvaro Sabater-Gárriz 1,2,3 and Pedro Montoya <sup>1</sup>**


**Abstract:** Cerebral palsy (CP) is an impacting chronic condition. Concomitant comorbidities such as pain and speech inability may further affect parents' perception of the pathology impact in the family quality of life and the provided care. The objective of this cross-sectional descriptive correlational study was to compare parental reports on family impact and healthcare satisfaction in children with CP with and without chronic pain and with and without speech ability. Parents of 59 children with CP (age range = 4–18 years) completed several questions about pain and speech ability and two modules of the Pediatric Quality of Life Measurement Model: The PedsQLTM 2.0 Family Impact Module and the PedsQLTM Healthcare Satisfaction Generic Module. Our findings revealed that children's pain slightly impacted family physical health, social health and worry. In children without pain, speech inability increased the perceived health impact. Parents' healthcare satisfaction was barely affected by pain or speech inability, both increasing parents' satisfaction in the *professional technical skills* and *inclusion of family* domains on the care plan. In conclusion, pain and speech inability in children with CP can impact family health but not healthcare satisfaction. Regular assessment and intervention in family health is essential for the design of family-centred programs for children with CP.

**Keywords:** cerebral palsy; pain; speech; family impact; healthcare satisfaction

### **1. Introduction**

Children with cerebral palsy (CP) have unique demands, causing significant impact to the quality of life of their families. The care that people with cerebral palsy require throughout their lives involves a great financial burden, a significant investment of time and significant repercussions on work and social activities [1–3]. Undoubtedly, all this can generate chronic stress in the family and caregivers who take care of these persons [4,5], thus compromising their health and well-being [3,6,7]. Thus, parents of children with CP have reported poorer physical and mental health than the general population, with higher levels of depression, musculoskeletal pain and fatigue [7–12]. Furthermore, the family impact does not appear to be associated with the dependency level, age, type or severity of CP [4,10,11,13]. Even improvements in the child's motor function do not produce changes in the quality of life of the parents [14]. Rather, parents develop negative feelings due to reductions or difficulties in the health, social skills, behaviours or emotions experienced by their children [4,15,16]. In this context, pain may be an important factor influencing family well-being, as concern for the child's pain is one of the most reported causes of emotional stress in parents of children with CP [13,17].

More than half of children with CP experience frequent moderate to severe pain at multiple body locations [18,19]. Recurrent pain produces an increase of behavioural and emotional problems in children with CP, reducing their quality of life and negatively

### **Citation:** Riquelme, I.;

Sabater-Gárriz, Á.; Montoya, P. Pain and Communication in Children with Cerebral Palsy: Influence on Parents' Perception of Family Impact and Healthcare Satisfaction. *Children* **2021**, *8*, 87. https://doi.org/10.3390/ children8020087

Academic Editor: Dulce María Romero-Ayuso Received: 11 December 2020 Accepted: 22 January 2021 Published: 27 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

affecting their participation in daily and social activities and the satisfaction of parents with performing these activities [20–22]. Pain constitutes an additional burden for the health system, producing more demand for health services than the severity of the pathology [23], with more frequent visits to the family physician, more prescription of analgesics [18] and more recurrent use of conventional and alternative therapies [24]. Furthermore, pain is one of the main concerns of parents when their children are faced with a health intervention [25] and one of the main factors influencing parents' evaluation of the intervention success [26]. Therefore, frequent pain reduces satisfaction with motor rehabilitation, while a low level of post-operative pain increases satisfaction after recovery and effective pain management is considered to improve the quality of healthcare [27,28].

The child's inability to speak is an important risk factor for parental stress and depression [17,29], increasing the vulnerability perceived by parents in interventions that affect health, such as surgery [30], and poor perception of the child's health-related quality of life [31]. Although parents are able to detect pain in their children in spite of their speech impairments [32,33], parent report increased frequency, duration and intensity of musculoskeletal pain in more severely affected children who are unable to self-report [34]. Moreover, discrepancies between parents and health professionals in pain detection are greater in children with speech problems [32].

Although pain and speech are important factors affecting parents' quality of life and satisfaction with health services, little research has focused on their specific associations. This study aims to compare parental reports on family impact and healthcare satisfaction in children with cerebral palsy with and without chronic pain, as well as with and without speech ability.

### **2. Materials and Methods**

### *2.1. Participants*

This is a cross-sectional descriptive correlational study, with purposive sampling and a survey method for data collection. The staff, which is responsible for 11 specialized centres dedicated to education, care or leisure for individuals with disabilities in Majorca (Spain), identified the participants with cerebral palsy. The inclusion criterion was a diagnosis of CP and age between 4 and 18 years. Most of the participants were identified in care centres for children with cerebral palsy (80% of families), while a smaller percentage were identified in educational or leisure centres that support different developmental conditions. The parents of 70 children with CP were initially contacted through a letter explaining the objectives and protocol of the study. Moreover, informative meetings were held with the families at the centres to explain the objectives and methods of the study throughout the last quarter of 2016. Parents of 59 children with CP (age range = 4–18, mean age = 11.58 (4.61), 34 girls) agreed to participate in the study and provided written informed consent. In addition, children with CP with a sufficient cognitive level expressed verbal or gestural willingness to participate. The study was approved by the Ethics Committee of the Regional Government of the Balearic Islands (Reference code: IB3156/16 PI).

### *2.2. Interview and Questionnaires*

The week following the signing of the informed consent, each family was assigned an anonymous code for data collection. Parents completed a semi-structured interview, and two 2-report questionnaires were delivered to be completed at home. The semistructured interview (Supplementary Table S1) consisted of several questions about demographic data (Table 1), as well as about the pain and communication characteristics of their children. A member of the research team was in permanent telephone contact with the families to resolve possible doubts while the questionnaires were being completed. Parents were asked to return the completed questionnaires to the centre in a sealed envelope, and a member of the research team collected them each week. The type of CP, the cognitive level and the level of motor impairment, determined by the Gross Motor Function Classification System (GMFCS-R) [35], were obtained from the children's medical history. Table 2 displays the clinical characteristics of children with CP.


**Table 1.** Families' sociodemographic characteristics.

**Table 2.** Clinical characteristics of children with cerebral palsy. GMFCS = Gross motor function classification system (1 = walks without limitations, 5 = transported in a manual wheelchair) [35].


Children's pain was measured using the following information from the interview: (1) Whether they were experiencing chronic pain (pain lasting more than 3 months) or not (yes/no response); (2) ratings of current and worst pain in the last week using a 11-point numerical rating scale (0 = no pain, 10 = unbearable pain); and (3) location of painful body regions using a human figure drawing (QL07/00 Pediatric Pain Questionnaire) [36]. Speech ability was assessed with a yes/no question.

Parents completed two questionnaires included in the Pediatric Quality of Life Measurement Model (PedsQLTM, Lyon, France) [37] to assess family impact and healthcare satisfaction. The PedsQLTM 2.0 Family Impact Module consists of 36 items comprising 8 dimensions: *Physical functioning*, *emotional functioning*, *social functioning*, *cognitive functioning*, *communication*, *worry*, *daily activities* and *family relationships*. Items are rated on a Likert scale from 0 (never) to 4 (almost always) and are transformed into a score from 0 to 100, with higher scores indicating better functioning (less impact). The 8 dimensions are combined into 3 total scores: The *Total impact score*, *Parent health-related quality of life summary score* and *Family functioning summary score*. The PedsQLTM Healthcare Satisfaction Generic Module consists of 24 items comprising 6 dimensions: *Information*, *inclusion of family*, *communication*, *technical skills*, *emotional needs* and *overall satisfaction*. Items are rated on a Likert scale from 0 (never) to 4 (always) and are transformed into a score from 0 to 100, with higher scores indicating greater satisfaction. The PedsQLTM model and its different questionnaires have proven to be valid and reliable for assessing different aspects of paediatric health-related quality of life [36].

### *2.3. Statistical Analyses*

Multivariate analyses of variance (MANOVA) were performed on family impact and healthcare satisfaction separately. The factors PAIN (children with pain vs. children without pain) and SPEECH (children with speech ability vs. children without speech ability) were used as between-subject factors in the statistical design. In addition, the factor DIMENSION was used as within-subjects to assess effects on the different subscales of each module. In case of significant effects due to DIMENSION, separate ANOVAs on the scores of each subscale were planned to further explore the differences due to PAIN and SPEECH. All results were adjusted using Bonferroni corrections for post-hoc comparisons. In addition, Pearson and Spearman correlations were performed to establish associations among the different dimensions of the family impact and healthcare satisfaction questionnaires with pain characteristics. The missing data were not replaced or completed by statistical methods and were discarded from the analyses. Significance levels were set at *p* < 0.05.

### **3. Results**

Once the recruitment performed, the sample size was estimated at 30 children. Pain was reported in 51% of the children (*n* = 30), and 40.7% had speech problems (*n* = 24). Parents reported moderate impact (mean = 69.14 (17.03), range = 26.39–96.53) and healthcare satisfaction (mean = 64.96 (21.94), range = 2–100). The descriptive data of the different dimensions for each of the four groups are displayed in Table 3.

In the Family Impact Module questionnaire, the MANOVA revealed only a main effect due to DIMENSION (F (7,27) = 17.77, *p* < 0.001), indicating that scores in the different subscales were significantly different. To further explore this effect, separate ANOVAs were performed on each dimension to examine the effects due to PAIN and SPEECH (Table 4). For the *physical functioning* dimension, a significant effect due to PAIN × SPEECH (F (1,39) = 5.04, *p* = 0.031) was yielded, indicating that parents of children without speech ability reported lower scores (higher negative impact) than those of children with verbal speech when children have no chronic pain (*p* = 0.044) (Figure 1). By contrast, no differences due to speech ability were observed on physical impact when children have chronic pain (*p* = 0.268). For the *social functioning* dimension, a significant interaction PAIN × SPEECH (F (1,39) = 4.38, *p* = 0.044) was also found, revealing that parents of children without speech ability reported lower scores (higher impact) than those of children with verbal speech when children report no chronic pain (*p* = 0.087) (Figure 1). No differences due to speech ability were observed on social functioning when children report chronic pain (*p* = 0.260). No other significant effects were found in the rest of the domains.


**Table 3.** Mean (standard deviation) and range of the different domains of the PedsQLTM 2.0 Family Impact Module and the PedsQLTM Healthcare Satisfaction Generic Module in every group of children.

**Table 4.** Statistical values of group comparisons in all the different dimensions of the Family Impact and Healthcare Satisfaction modules. Two-way ANOVAs, with PAIN (children with pain vs. children without pain) and SPEECH (children with speech ability vs. children without speech ability) as between-subject factors. \* *p* < 0.05, \*\* *p* < 0.01.


Healthcare Satisfaction Generic Module

Cognitive functioning F = 1.43, *p* = 0.241 F = 0.19, *p* = 0.665 F = 0.47, *p* = 0.499 Communication F = 1.12, *p* = 0.298 F = 0.62, *p* = 0.437 F = 1.66, *p* = 0.207 Worry F = 0.24, *p* = 0.629 F = 0.89, *p* = 0.768 F = 0.89, *p* = 0.768 Daily activities F = 0.65, *p* = 0.425 F = 0.79, *p* = 0.379 F = 1.89, *p* = 0.178 Family relationships F = 0.24, *p* = 0.629 F = 1.46, *p* = 0.235 F = 0.15, *p* = 0.703

Global score F = 0.96, *p* = 0.335 F = 0.88, *p* = 0.354 F = 1.73, *p* = 0.197 Information F = 1.14, *p* = 0.294 F = 0.24, *p* = 0.629 F = 0.20, *p* = 0.657 Inclusion of family F = 2.50, *p* = 0.123 F = 0.38, *p* = 0.540 F = 8.24, *p* = 0.007 \*\* Communication F = 0.62, *p* = 0.437 F = 0.51, *p* = 0.482 F = 0.66, *p* = 0.423 Technical skills F = 0.27, *p* = 0.609 F = 6.80, *p* = 0.014 \* F = 0.70, *p* = 0.410 Emotional needs F = 1.06, *p* = 0.312 F = 0.16, *p* = 0.901 F = 0.93, *p* = 0.343 Overall satisfaction F = 0.89, *p* = 0.768 F = 0.52, *p* = 0.478 F = 0.14, *p* = 0.907

**Figure 1.** Mean and standard errors of the four groups of children in the significant dimensions of the Family Impact and Healthcare Satisfaction modules. \* *p* < 0.05, \*\* *p* < 0.01. **Figure 1.** Mean and standard errors of the four groups of children in the significant dimensions of the Family Impact and Healthcare Satisfaction modules. \* *p* < 0.05, \*\* *p* < 0.01.

In addition, the number of pain locations correlated negatively with scores in the *worry* dimension of the family impact (r = −0.37, *p* = 0.033), indicating increased parents' worry (lower functioning) when the number of painful body locations increased. The worst pain during the week correlated with the *communication* dimension (r = 0.515, *p* = 0.029), revealing lower impact in family communication with higher intensity of the In addition, the number of pain locations correlated negatively with scores in the *worry* dimension of the family impact (r = −0.37, *p* = 0.033), indicating increased parents' worry (lower functioning) when the number of painful body locations increased. The worst pain during the week correlated with the *communication* dimension (r = 0.515, *p* = 0.029), revealing lower impact in family communication with higher intensity of the child's pain.

child's pain. In the Healthcare Satisfaction Module questionnaire, the MANOVA revealed a main effect due to DIMENSION (F (5,27) = 6.17, *p* < 0.001), indicating that scores in the different subscales were significantly different. In order to further explore differences due to PAIN and SPEECH, separate ANOVAs were performed on the scores of these subscales (Table 4). In the *technical skills* dimension, a significant main effect due to SPEECH (F (1,36) = 6.80, *p*  = 0.014) revealed higher satisfaction in parents of children without speech ability compared with parents of children with verbal speech. In the *inclusion of family* dimension, a significant interaction PAIN x SPEECH (F (1,37) = 8.24, *p* = 0.007) indicated that (1) parents of children without speech ability reported higher satisfaction when children report In the Healthcare Satisfaction Module questionnaire, the MANOVA revealed a main effect due to DIMENSION (F (5,27) = 6.17, *p* < 0.001), indicating that scores in the different subscales were significantly different. In order to further explore differences due to PAIN and SPEECH, separate ANOVAs were performed on the scores of these subscales (Table 4). In the *technical skills* dimension, a significant main effect due to SPEECH (F (1,36) = 6.80, *p* = 0.014) revealed higher satisfaction in parents of children without speech ability compared with parents of children with verbal speech. In the *inclusion of family* dimension, a significant interaction PAIN × SPEECH (F (1,37) = 8.24, *p* = 0.007) indicated that (1) parents of children without speech ability reported higher satisfaction when children report chronic pain than when they report no pain (*p* = 0.009), whereas no differences were observed in children with verbal speech ability (*p* = 0.293); and (2) parents of children with pain reported higher satisfaction when children did not have speech ability than when they have speech ability (*p* = 0.021), whereas no differences were found in children without chronic pain (*p* = 0.113) (Figure 1). There were no other significant ANOVA effects or significant correlations regarding parents' healthcare satisfaction.

### **4. Discussion**

The objective of the present study was to explore the mutual influence of children's pain and speech ability on parental perception about the family impact and healthcare satisfaction in children with cerebral palsy. Our findings point to a slight impact of pain on family functioning. Only a few dimensions, such as *physical functioning*, *social functioning* and *worry*, seem to be affected by the presence of pain, which modulates the least perceived impact when the child has verbal speech. Parental satisfaction with healthcare was barely affected by pain or the lack of speech, increasing both the parental satisfaction with professional technical skills and inclusion in the plan of care.

Pain affected physical and social family functioning and the number of pain locations impacted on parental worrying. Pain has been reported to be a factor that increases the demand for care also in other chronic paediatric pathologies such as osteogenesis imperfecta [38]. Other studies have reported a good parental understanding of children's expressions of pain, even when they cannot communicate verbally [32,39]. In the present study, we observed that families of children with chronic pain (with and without speech abilities) reported equal impact on physical and social health. A periodic evaluation of the physical, social and psychological status of the parents should be included in the protocols of the family-centred care models for children with CP in order to detect the specific areas (e.g., worry about child's pain) that deserve specific attention [10,16,23]. In this sense, some experiences, such as web-based intervention programs that provide training in daily care to mothers of children with CP or home-based programs that use augmentative and alternative communication, have proven to improve the experience of care and quality of life of the caregiver [40,41]. Interestingly, the greater intensity of the children's pain produced a lesser impact on family communication. Other studies with challenging situations, such as chronic life-threatening illnesses, have shown that parents concentrate on solution-focused communication, deferring potentially distressing discussions [42]. Thus, it seems that the presence of pain can help to promote pragmatic communication among family members to solve critical problems.

Inclusion in healthcare decisions, encompassing all phases of assessment, intervention and evaluation, is a critical determinant of high-quality care for parents and chronically ill children [43,44]. Families understand inclusion as the ability to communicate, understand the care plan and participate with the health team in decision-making [45]. Another factor that promotes family satisfaction with healthcare is professional competence [46], which is defined in a complex way and encompasses the attributes of emotional and communication skills (providing empathy for child/family, explaining procedures, answering questions) [47–49]. By contrast, misunderstanding of the problem or differences in intervention priorities negatively affect the parent-professional relationship [25,50]. Challenging situations, such as a lack of verbal speech or pain, require focusing on the problem, and require health professionals to improve their competences, reinforcing parental satisfaction with healthcare.

Limitations. The questionnaires were answered by one of the parents (mostly the mother), even in divorced families. Therefore, the perception of the other partner may differ. Our sample was small and biased toward participants with high motor difficulties (76.3% of the sample had GMFCS levels from 3 to 5), since most of the participants were identified at specialized centres for children with cerebral palsy (80% of the families). Similarly, our prevalence of pain and speech disability was slightly higher than that reported by other studies [51,52], probably due to the overrepresentation of children with greater impairments. These facts do not reflect the general distribution in the CP population, and the generalizability of the findings should be limited to children with the most severe impairments.

In conclusion, pain and, to a lesser extent, the ability to speak in children with CP can have an impact on the physical, social and psychological health of their families, although it does not seem to affect healthcare satisfaction. Periodic assessment and intervention of the family's health and needs should be considered in the design of family-centred programs for children with CP.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/2227-906 7/8/2/87/s1, Table S1: Interview questions.

**Author Contributions:** Conceptualization, I.R. and P.M.; methodology, I.R.; formal analysis, I.R.; investigation, I.R.; resources, P.M.; writing—original draft preparation, I.R.; writing—review and editing, I.R., Á.S.-G. and P.M.; visualization, I.R. and Á.S.-G.; supervision, P.M.; project administration, P.M.; funding acquisition, P.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Spanish Ministry of Economy and Competitiveness and European Regional Development Funds, grant number #PSI2017-88388-C4-1-R.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of the Regional Government of the Balearic Islands (protocole code: IB3156/16 PI, approved on 24 February 2016).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

### **References**


MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*Children* Editorial Office E-mail: children@mdpi.com www.mdpi.com/journal/children

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34 Fax: +41 61 302 89 18

www.mdpi.com ISBN 978-3-0365-4391-8