Thinking about Inclusion: Designing a Digital App Catalog for People with Motor Disability

Abstract

Some apps serve as assistive technologies or digital therapeutic tools that can be used by rehabilitation professionals in the motor disability context, bringing benefits to therapists and people with disabilities. However, websites or catalogs do not provide reliable information, easy search, and intuitive access to these apps, causing access information difficulties. Therefore, this work proposes to develop a digital catalog of software focused on motor disability. This work performed a systematic search of websites and catalogs related to motor disability, a systematic search of the apps that the digital catalog would show, and a quality evaluation of selected apps using the Mobile Application Rating Scale tool. The digital catalog was developed with the information obtained in the previous phases combining the Prototyping and User Experience criteria, then assessed by final users, software testers, and a web accessibility evaluation tool. The catalog obtained a satisfactory quality score based on the end-users’ and therapists’ satisfaction when finding technological resources to use in their professional and health-care activities. This research aims to contribute to those interested in developing software for people with disabilities and encouraging them to create and design their implementations based on this study.

1. Introduction

Disability is part of the human condition, and many people will have temporary impairments at some stage in their lives. Moreover, those who reach old age will experience annoying difficulties in their daily lives. Disability is complex, and the interventions to overcome the disadvantages of disability are diverse and vary according to the person [1].

The International Classification of Functioning, Disability, and Health (ICF) defines disability as an umbrella term for impairments, activity limitations, and participation restrictions. Over a billion people are estimated to live with some disability. These data correspond to about 15% of the world’s population. Out of this number, between 110–190 million people have very significant difficulties in functioning, often requiring healthcare services. Furthermore, the rates of disability are increasing due to aging populations and an increase in chronic health conditions [2].

Information and Communication Technology (ICT) allows people with disabilities (PWD) to integrate better socially and economically, into their communities. For PWD, access to technology means living more independent lives, accessing information and services. Therefore, the Web promotes a considerable change for PWD, removing communication and interaction barriers compared to the physical world’s challenges [3]. Furthermore, ICT offers excellent levels of flexibility and customization for users with disabilities. Therefore, ICT plays a fundamental role in creating opportunities for PWD in every stage of life [4].

Technology can support the functional characteristics of people involved in the disability context. For example, in motor disability, the main stakeholders are relatives of PWD, rehabilitation professionals, and patients [5]. They can use apps to search for professional counseling, control their treatment, assess their therapeutic activities, perform diagnostic evaluations, and help patients to complete tasks independently [6].

A study by Kadijevich et al. [7] examined the extent to which individuals use ICT by comparing people with and without disabilities. This study found a directly proportional relationship between ICT use and life quality improvement for PWD.

Nowadays, the Web is considered an important tool that can be utilized to achieve numerous tasks. It is also used to obtain knowledge and to learn new skills. A wide variety of devices can be used to access the Web [8].

PWD can deal with three types of barriers when engaging with the surrounding environment: environmental, attitudinal, and digital. The digital barriers arise when the technology cannot be changed into one more layout accessible by assistive technologies [8].

Assistive technologies are hardware or software tools that individuals with handicaps use to access and utilize various technologies. The Web contents can be in text, video, sound, or graphics. Web ease of access indicates that the website site can be accessed and used efficiently by people with and without disabilities [9]. PWD need to surf and utilize the web easily; this can help them conquer physical barriers by benefiting from various online services.

Kirchner [10] determined that the primary problem in Web accessibility is that most websites are not appropriate for all types of disabilities. Nevile [11] recommended that, in order to develop Web content that is universally accessible, a programmer has to develop redundant components with different methods to access the contents.

Providing an accessible Web will make it possible for disabled people to be more independent and more efficient. The National Company on Disability [12] estimated the growth of PWD who utilize the web is twice that of non-disabled.

In the United States and Europe, assistive technology is an integral part of the health system. Additionally, the government provides devices and systems through rehabilitation, specialized education agencies, insurance companies, and non-governmental organizations [13].

On the other hand, Latin America and the Caribbean are among the regions of the world with the most significant inequality. This region is highly unequal concerning incomes and exhibits unequal access to education, health, water, electricity, and technology [14]. Despite economic and social advances in this century, the Economic Commission for Latin America and the Caribbean has warned on several occasions [15] that Latin America is still “an unequal region on the planet”.

Because PWD represent approximately 13% of the population in Latin America and the Caribbean, it is easy to understand the cost of not taking full advantage of this group’s production, consumption, and tax payment potential. The price of exclusion is high; the problem for PWD is that they are less likely to be employed than their peers. The lower skill levels observed are due, in part, to exclusion from education, lower promotion rates, and unemployment. When hired, these jobs are more likely to be poorly paid and in the informal sector [16]. Another issue is that many Latin American countries still perceive ICT as a leisure-related luxury good and not for working. As a result, people do not realize technology can empower them to promote topics relevant to the local and national agendas and improve their basic needs [17].

We address the following research questions in this work:

RQ1. How can a digital catalog support the rehabilitation process of people with motor disabilities?

RQ2. What is the acceptance of the app by end-users?

The current work presents the design and development of a digital catalog of easy access and intuitive navigation that provides reliable information about apps related to motor disability.

The current work is structured as follows: Section 2 presents the background. Section 3 performs the methodology followed to create the digital catalog for motor disability apps. Section 4 shows the main results. Section 5 contains a brief discussion of the results obtained. Finally, Section 6 offers some conclusions.

2. Background

Disabilities are diverse and can lead in some cases to poor health and extended healthcare needs. A report indicates that PWD seek healthcare services and have higher unmet needs more often than people without disabilities [2].

Motor disability is a condition that arises from a permanent or temporary deficiency of the locomotor system due to some abnormality in the functioning of the bone, joint, or muscular system. This condition can include posture problems, and difficulty in movement, coordination, or manipulation [18].

2.1. Habilitation and Rehabilitation

The World Health Organization (WHO), in its world report on disability, defines rehabilitation as a variety of measures that help PWD to maintain optimal functioning concerning their environment.

Habilitation focuses on people who have a congenital disability or who they have acquired in the early stages of life.

Rehabilitation is aimed at people who have lost functionality and helps them regain it to a greater or lesser degree [1].

Therapy is the process of restoring activity, compensating functionality, and preventing deterioration in areas of a person’s life. This activity is performed by rehabilitation professionals, like physical therapists and occupational therapists [1].

Physical therapy provides services to maintain, develop, and restore maximum movement and functional skills. The purpose of physical therapy is to help people at any stage in their life cycle when movement and functionality have been affected by age, trauma, injuries, disease, health disorders or environmental conditions [19].

Occupational therapy is the therapeutic use of self-care, work, and play activities to increase independent functionality, improve development, and prevent disability. Occupational therapy can include the adaptation of the environment and activities to enhance people’s quality of life [20].

2.2. Motor Disability and Technology

Therapeutic digital tools connect with the use of resources such as estimation teams, diagnostic-oriented evaluations, and evaluations aimed at measuring functionality. These resources, combined with the professional knowledge of the therapist, can generate a series of rehabilitation actions. There are technological tools such as numerical tests, exercise guides, therapeutic content guides, control and management systems, which can assist the professional in his or her clinical, training or research activity [6].

Assistive technologies refer to practical tools that increase the independence of PWD. They are any part, equipment, product or system acquired or modified used to increase, maintain, or improve individuals’ functional capabilities.

Some examples of assistive technology tools are screen magnifiers, audiobooks, font magnification software, alternative digital communication boards, voice synthesizers, geolocation, and accessible mobility software, architectural barrier identification software, and systems to replace the standard input and output interfaces of the computer [21].

The use of mobile technology can encourage PWD to change their behavior and obtain help in their daily life. ICT can extend the learning process for PWD beyond the school, maintaining functional skills while working with rehabilitation professionals face-to-face [22].

Apps with interactive content allow empowerment and promote self-esteem through activities and measures of achievement. Tablets, such as iPads, have been used as classroom assistive technology to provide educational support to students with disabilities [23].

Motor disabilities can also affect speech for individuals, Augmentative and Alternative Communication (AAC) software can help with these types of disabilities [24]. People with motor disabilities can interact through music, dance, and the visual arts, thanks to specialized apps [25].

Free assistive technology software is the key against paid apps for motor disabilities. Open-source software and freeware are available at online repositories like Raising the Floor, EmpTech, Adaptech, Athena, OATSoft, and Project Possibility. Moreover, devices and apps can be compared according to their essential features and suggested information, improving the classification process [26].

3. Method

The current research uses a method based on design science research (DSR), proposed by Hevner et al. [27]. This framework is active in technology, participating in the creation of technical artifacts that affect people and organizations. It focuses on solving problems, but usually only considers a simplified view of the people and organizational environment that must work in it.

The purpose is to reproduce prototypes and user-oriented evaluations [28], taking as reference the guidelines of DSR for the current work [29]. The DSR [30] process includes six steps:

The phases to perform this work are as follow:

• Problem identification and motivation;
• Objectives of the solution;
• Research existing websites and catalogs for people with motor disabilities;
• Design and development;
• Demonstration;
• Evaluation.

3.1. Problem Identification and Motivation

The rapid development of technologies such as web apps and mobile apps is of great help to the professionals of physical and occupational therapy and the patient with motor disabilities, either during their treatment or in their daily lives. These technologies can intervene in processes such as searching for information and disseminating therapeutic activities [6]. However, in Latin America, the problems concerning these resources are the inexperience of the people who could benefit from their use and the limited accessibility of these resources.

For example, in Ecuador, the National Council for Disability Equality (CONADIS) has established that there are 215,751 people who have some degree of motor disability [31]. According to the National Agenda for Disabilities, one of the factors that affect people with disabilities, is the inexperience of patients and therapists when it comes to finding existing technological resources and information on their benefits [32]. Although the highest percentage of persons with disabilities registered in the country categorizes their condition as a motor disability, in Ecuador, there is no digital catalog that facilitates the search for applications that could be used in the treatment and daily activities of people with this type of disability.

3.2. Objectives of the Solution

The objective is to create an interactive digital catalog that allows users access to resources for motor disability therapy. To achieve the objective, the following needs to be carried out:

• Establish a categorization for the apps.
• Design of the catalog should be a user-friendly interface.
• The catalog needs a community feedback area for the users, in order to improve the apps.
• The apps contained in the digital catalog are intended to encourage the use of new technologies for therapy for people from 19 years old.

3.3. Research Existing Websites and Catalogs for People with Motor Disability

The technique used to search for catalogs and websites was Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [33]. This method is used to perform systematic reviews and has a broad range of uses in several study fields.

The search performed used the Google search engine. The language was Spanish and the search strings were “Software Discapacidad Física”, “Catálogo de Software Discapacidad Física”, “Software Discapacidad Motora”, “Catálogo de Software Discapacidad Motora”, “Software Discapacidad Motriz”, and “Catálogo de Software Discapacidad Motriz”. The following exclusion criteria were applied: duplicate digital catalogs, blogs and information sites, non-functional sites, sites with irrelevant or out-of-context content, and catalogs and lists in PDF format.

Figure 1 shows the catalog searching and exclusion process. After that, selected websites were analyzed to identify essential characteristics and improvement chances.

Table 1. Websites and Catalogs Analysis.

ID	Name	Country	General Features
01	SOLCA	Panama	– It provides free software that allows motor disabled people to use computers. – Organized by Functionality – Content Nesting Level 3 – Ease of Use Medium
02	Free Resources for the disabled (Recursos Gratuitos para Discapacitados)	Spain	– It provides free software that allows motor disabled people to use computers. – Organized by Functionality – Content Nesting Level 3 – Ease of Use Low
03	Tecno Accesible	Spain	– It provides free assistive software for PWD. – Organized by Functionality – Content Nesting Level 3 – Ease of Use High
04	BJ Adaptations	Spain	– It provides paid assistive software. – Organized by Functionality – Content Nesting Level 3 – Ease of Use High
05	Wikinclusion	Uruguay	– It is a wiki to improve the interaction between technology and disabilities. – Organized by Life competence areas, disability type, and alphabetically. – Content Nesting Level 6–7 – Ease of Use Medium
06	ENESO	Spain	– It provides paid software to allow PWD to use technology. – Organized by Functionality – Content Nesting Level 5–6 – Ease of Use High
07	Disability portal (Portal Discapacidad)	Spain	– It provides free software that helps motor disabled people to use computers. – Organized by Functionality – Content Nesting Level 4 – Ease of Use Low
08	ADAPTA	Chile	– It provides software for PWD education and rehabilitation. – Organized by Feature – Content Nesting Level 4 – Ease of Use High
09	DISCAPNET	Spain	– It offers assistive software for motor disabled people. – Organized by Functionality – Content Nesting Level 4 – Ease of Use Medium
10	Connect equality (Conectar Igualdad)	Argentina	– It provides free software that allows PWD to use computers. – Organized by disability type. – Content Nesting Level 3 – Ease of Use Medium
11	Antonio Saco	Argentina	– It provides free software and information for PWD. – Organized by No categorization criteria. – Content Nesting Level 4 – Ease of Use Low
12	ARASAAC	Spain	– It provides software to improve communication for PWD. – Organized by Functionality – Content Nesting Level 5–6 – Ease of Use High
13	Physiotherapy software (Software para Terapia Física)	Spain	– It provides paid software for physical therapy professionals. – Organized by feature – Content Nesting Level 4 – Ease of Use Low
14	Innovations (Innovaciones)	Colombia	– It offers paid software and equipment for therapeutic activities. – Organized by No categorization criteria. – Content Nesting Level 5 – Ease of Use Low

Results. After discarding duplicate results, irrelevant and out-of-context websites, PDF files, and non-functional sites, the sample for the analysis was 14 websites. The following main characteristics were found:

• On these websites, users must do 3–7 clicks to access an app.
• Only three of the analyzed websites allow user feedback on the apps.
• Two types of websites were identified:

  ○

  Developer websites belong to projects or institutions that develop health and disability solutions, with the support of health professionals and technology experts. For instance, BJ Adaptations creates customized solutions to improve disabled people’s autonomy.

  ○

  Promoter websites collect information about support and health technology products, share them, and provide access. For instance, the Aragon Portal of Alternative Communication (ARSAAC), gathers information and resources to improve people’s functional difficulties autonomy.

• None of the websites records the history of apps obtained or comments made by a user.
• None of the websites presents any content restriction with apps that have sensitive medical information that represents a risk for any patient without professional supervision.

Additionally, each website and catalog evaluated its content using the Web Accessibility Assessing Tool (WAVE). WAVE is a suite of evaluation tools that helps authors make their web content more accessible to individuals with disabilities. WAVE can identify accessibility and Web Content Accessibility Guideline (WCAG) [34] errors and facilitate the human evaluation of web content.

Each of the websites and catalogs analyzed were evaluated by WAVE to obtain metrics of their accessibility characteristics.

Table 2. Websites and Catalogs Accessibility evaluation—WAVE.

ID	Name	Errors	Contrast Errors	Alerts	Features	Structural Elements	ARIA
01	SOLCA	12	12	21	5	55	32
02	Free Resources for the disabled (Recursos Gratuitos para Discapacitados)	9	9	16	1	21	0
03	Tecno Accesible	0	7	18	29	58	2
04	BJ Adaptations	19	70	31	16	68	2
05	Wikinclusion	1	1	85	35	53	8
06	ENESO	1	13	11	11	23	2
07	Disability portal (Portal Discapacidad)	5	20	108	8	19	0
08	ADAPTA	6	4	28	9	20	5
09	DISCAPNET	0	1	4	7	41	0
10	Connect equality (Conectar Igualdad)	1	18	4	2	25	0
11	Antonio Saco	0	11	2	4	3	0
12	ARASAAC	13	8	76	53	21	10
13	Physiotherapy software (Software para Terapia Física)	3	436	134	141	231	22
14	Innovations (Innovaciones)	20	49	72	36	45	0

shows the results of the WAVE evaluation. The main errors found were the contrast of colors and the lack of alternative texts in the elements of the HTML structure of the sites. These factors can cause readability problems and limit the possibility that a screen reader user can navigate and use a website.

Mobile Application Rating Scale (MARS) [35] is a tool to evaluate web and mobile app quality. In this research, only four quality sub-scales were considered: engagement, functionality, aesthetics and information.

MARS has been cited in more than 300 articles. For instance, Masterson et al. used the tool to assess apps related to heart failure symptom monitoring and self-care management [36], Mani et al. for iOS mindfulness-based apps [37], Santo et al. for apps related to medication reminders [38], Tinschert et al. for apps related to asthma [39], Sullivan et al. for travel and diet [40], and Grainger et al. for apps that assist people to monitor rheumatoid arthritis [41].

3.4. Design and Development

The prototyping criteria were followed to perform an incremental development of the catalog. Moreover, User Experience (UX) [42] measures were covered to focus functionality and design features on the user needs.

3.4.1. User Experience

User Experience allows design efforts to be focused on the primary needs of potential users. A UX framework proposed by Garret [43] was used to provide the advantages of the development of prototypes, and at the same time to create a catalog that promotes satisfactory user perceptions. During the digital catalog development, we addressed the five levels of UX defined by Garret; these levels are:

Strategy

Characteristics and improvement chances identified during websites searching helped to identify digital catalog objectives. Decision-making during digital catalog development was based on metrics that allowed the user experience and satisfaction to be perceived. HEART, an evaluation framework designed by Google [44], was taken as a reference for user perception metrics and indicators definition. To obtain real users feedback, recurred to physical and occupational therapists to answer the following questions: What is your level of technological knowledge? What difficulties do you encounter when you are looking for digital support for your work? How willing are you to use apps in your daily job? These questions allowed the researchers to collect information about their main activities in the motor disability environment.

Scope

This section defines the functional and content requirements for the digital catalog, and it is worthwhile because it indicates what would develop and what would not. Catalog requirements were designed to satisfy the following specific functionality aspects: apps access, apps feedback, comments, complete information on apps, access history, user registration, app searching, comment history, content restriction, events and news, and management modules.

The functional specifications proposed at this stage are:

• Initial view refers to the digital catalog’s home page displays the list of apps.
• App download differs by app type. In web apps, it opens the site directly in mobile apps, it redirects to platforms such as Apple App Store or Google Play Store. Finally, desktop apps provide the official installer download page.
• Feedback allows registered users to assign a five-star scale scoring to the apps.
• Comments on each app will help users to add new comments and reply to existing ones.
• App information contains its version, platform, category, requirements, number of accesses, and a description.
• User registration provides a form in which users will enter their information, user type, and authentication data.
• App search is located at the top of the home screen and helps users look for content.
• Comment history stores the records of the profiles and comments made by users.
• News and events show relevant and updated information for the users.
• Content restriction controls access to unauthorized users. The full access to apps of the category “Tools for professional use” is reserved only for “Therapist” accounts.
• App management role manages apps, including unsubscribing previously entered apps.
• Comment management allows an administrator user to block inappropriate comments.

Structure

Catalog design involved decisions about how to supply and display information to users. These decisions changed according to evaluations performed over the catalog prototypes, improving the ease of use. Some mechanisms were implemented to prevent errors that could appear during user interactions. Figure 2 shows a map of the website.

Skeleton

At this stage, a rapid prototype was developed using front-end languages. The navigation and usability tests performed on the prototype helped discover errors early, during the development cycle. The Bootstrap material design framework was used to create responsive and adaptive views for catalog interfaces. A vital prototype characteristic was that interfaces could adapt to any screen size.

Surface

Once the interaction flow and component arrangement on the screen were completed, the visual styles of all interfaces or sections of the website were defined, ensuring consistency and uniformity. Moreover, a brand was designed to give an identity to the digital catalog, which produces the right interpretation about catalog functionality in the user’s understanding. Additionally, WCAG 2.1 [45] was used as a reference to provide accessibility features to the site. The current work embraces the four following standard principles:

• Perceivable takes care of the way users can perceive information and user interface;
• Operable makes sure that user interface and navigation are appropriate;
• Understandable checks that the information and operation of the user are suitable;
• Robust focuses on the significance of content to be interpreted by users.

3.4.2. Prototyping

Prototypes can be used in the software development process to anticipate changes that may arise, and they can be a tool during the elicitation process and requirements validation [46]. A prototype goes through a stage of planning, definition, development, and evaluation. Additionally, to identify the prototype’s level of completeness with the final system, three fidelity levels are defined.

The low-fidelity prototype suggests the structure and relationship between the different interfaces of the catalog without a real view or functionality. The screen models were designed using the Balsamiq tool, which allows screen mockups to be created to capture the users’ initial sensations and define the accommodation of each screen’s elements. As an example of the prototypes developed, Figure 3 shows the home screen of the low-fidelity prototype. This screen aims to show the apps available from the moment the user enters the catalog. The main sections of this screen are the navigation bar on the top, the list of categories on the sidebar displayed on the left, and a list of selected apps in the center.

The middle-fidelity prototype is a navigable representation of the catalog, with real information from one app, a first logo model, and a proposed categorization parameter. This prototype was developed using front-end languages, Bootstrap, and JQuery to create navigation and functionality simulation. The buttons that lead to the apps, registration, account, and category sections were functional. Figure 4 shows the home screen of the middle fidelity prototype. The category bar is shown as a list, and the apps are shown by information cards that contain the image, star rating, and category. The categories of the catalog are Tech support, for therapist use, rehabilitation, and adaptation, motor improvement, consciousness. In this prototype, the range of colors and the position of some elements still present readability problems.

The high-fidelity prototype was created using MEAN stack MongoDB, Express Js, Angular, Node Js, a set of four web technologies that allowed the use of JavaScript as the programming language, both in the back-end and front-end [47]. Moreover, GitHub and Cloudinary were included to perform version control and media files management. Figure 5 shows the Category screen of the high-fidelity prototype. On this screen, the name and description of the category can be seen, and the apps are represented by information cards. The cards of the apps contain the name of the app, its category, the platform, the star rating, and a link to be able to read the description of the app without accessing the app page. All the screens of this prototype have the navigation menu as permanent elements. The categorization bar to facilitate mobility through the website, the final logo of the catalog, and Material Design characteristics were also included, giving the elements a sense of depth and hierarchy on each of the screens.

3.5. Demonstration

The research resulted in the creation of a digital catalog. This artifact was used effectively to UX and prototype to improve the therapy process of people with motor disabilities.

The immediate contribution of this research to the therapy process was implemented in the “Fundación Hermano Miguel”, located in Quito (Ecuador). The therapists indicated that the functionality of the digital catalog of software focused on motor disability allows easy access to applications focused on this type of disability; that is, the therapists were able to find appropriate tools according to the pathology or the patient they were treating at a certain moment. It even helped them replace the use of physical and printed artifacts with applications that digitize information during control and diagnostic activities. The usability evaluation evidenced the ease of understanding and use of the catalog by users. Therapists valued the digital catalog as a practical and handy tool when finding technological resources for their activities in the field of motor disability. In a broader context, the artifact serves as a valuable tool in the health system.

3.6. Evaluation

3.6.1. Low-Fidelity Prototype Evaluation

The goal of the low-fidelity prototype assessment was to identify sections that generate more significant user interaction and design problems that could be confusing on the site. A group of five professionals made up of three occupational therapists and two physical therapists performed the evaluation. Figure 6 shows how the tool Canvas Flip was used to get feedback and produce reports about user interaction. This tool allows test cases to be created and video clips to be recorded from user sessions.

Canvasflip generates interaction maps for a test. Figure 7 shows an interactive map in which the sections with the highest concentration of color represent a more significant number of clicks made, and therefore, greater interaction with the user.

Figure 8 shows the survey used during the evaluation of the low-fidelity prototype.

3.6.2. Middle-Fidelity Prototype Evaluation

The goals of this evaluation were: determine preferences regarding the categorization parameter and identify the prototype sections that require changes. A survey was designed and applied to a group of two physical and three occupational therapists. The closed questions were: Would you use a software catalog for your therapeutic activities? What type of users would use the catalog more frequently? Indicate the degree of importance of each section presented. Which of the classification patterns is most useful for your activities? The open questions were: Which of the items in the catalog was not clear during the demonstration? What is your opinion regarding the information in the digital catalog? Do you have any suggestions regarding the apps of the catalog? Any comments regarding the demo? Figure 9 shows the survey used during the evaluation of the middle-fidelity prototype.

3.6.3. High-Fidelity Prototype Evaluation

This prototype assessment was made up of four tests:

• The usability test consists of asking a group of users to carry out some interactions for which it was designed. The test was performed by a group of five therapists: two physical therapists and three occupational therapists. This test consisted of free use with the catalog, with the following activities: exploration of the catalog sections, review of the catalog content, and handling of the interface.
• The task-oriented test refers to a group of users who performed a set of specific tasks to be within the catalog. The time and the number of interactions each task took were measured.
• Accessibility tests using WAVE can identify WCAG related errors and facilitate the human evaluation of web content. The test was performed automatically by the tool.
• Quality assessment using MARS used the catalog evaluation and answered the items of the tool; 25 testers performed the evaluation.

Figure 10 shows the survey, which seeks to perceive user satisfaction in terms of content, navigation, structure and design, appearance, and functionality.

The WAVE tool was used to evaluate the accessibility characteristics of the web content in the digital catalog. The tool generated a report of the errors regarding accessibility, positive attributes, and elements that must be analyzed, as shown in Figure 11.

4. Results

From the evaluation of the low fidelity prototype, it can be identified that the sections with the most user interaction were the category navigation bar and the app’s information screen. It was possible to identify that the attributes that motivate users to select an app are the number of downloads and the app’s rating.

4.1. Canvas

The CanvasFlip tool recorded the interaction of five participants, who were able to observe and click on the 9 mock-ups of the prototype. The average time used to navigate the prototype sections was 4 min and 1 s, and the average clicks made for navigation was 93 keystrokes. According to the interaction Heatmap generated by Canvas Flip, it was determined that the elements with the highest number of interactions with the user were the following: on the home screen and the category screen, it was the category bar.

In the task-oriented test, a scenario with 9 navigation tasks was created. The test was applied to five users using the CanvasFlip.

Table 3. Tasks performed by users—CanvasFlip.

N	Task	Users	Completeness	Time (s)	Click
1	Enter a category	5	100%	7	2.4
2	Access an application and read its information	5	100%	6	2.8
3	Go to the NEWS section	5	100%	2	1
4	Enter the user registration screen	5	100%	20	9.8
5	Log in as a user and visit the profile	5	100%	26	9.4
6	Go to the MY DOWNLOADS section	5	100%	10	5.6
7	Access one of the MY DOWNLOADS apps	5	100%	8	4
8	Sign out of an account and access an app	5	100%	7	3.2
9	Make comments in an application	5	80%	4	3

shows each of the tasks performed by the users.

User Observations are:

• The navigation elements must be better defined to reach each section of the site.
• Each application should indicate the current version in operation.
• The information of the applications should include your required operating system (Android, IOS, etc.).
• Each category should have a description.
• The news section is only helpful if the information is up to date.
• There should be a section that explains the functionality of the site.

4.2. Prototype Results

In the evaluation of the middle-fidelity prototype, it was possible to identify that the most critical sections for users were the categorization bar and the app page. Users stated that the categorization parameter of apps should be according to their environment of use. The final set of categories were the following: apps for professional activities, self-care apps, digital badges, alternative communication apps, entertainment apps, accessible mobility apps.

The five therapists indicated their predisposition to use the catalog for their therapeutic activities. Figure 12 shows the degree of importance that users gave to each of the prototype sections. The five users indicated that the most important areas are the category bar and the app page.

Figure 13 shows users’ preferences for the categorization parameter used for the catalog. For example, 20% of users chose categorization by pathology, 20% chose categorization by target users (therapists, caregivers, patients), and 60% chose categorization by application focus (professional use, mobility, alternative communication, self-care, digital ramp).

User comments were as follows:

• Voice navigation is confusing, and it makes it difficult to use the site, as many people with neuronal damage could pronounce the commands correctly.
• You must specify who can add responses to the comments made.
• It should be specified that some applications should be for the exclusive use of professionals, as their content is sensitive.
• It must be indicated whether the applications are for the computer, the smartphone, or the internet.
• The project is a good initiative, as there are beneficial resources.
• Instead of using voice commands, access to applications should be simplified.
• The recommended applications section is not clear.
• The news section must be up to date. Additionally, it would be helpful if the information on events such as congresses and seminars, was updated.
• A guide or help section would be helpful to understand the functionality of the site.
• In the future, paid applications should be included since the user makes the last decision at download.

4.3. Usability Test Results

After the usability test was performed on the high-fidelity prototype, the therapists indicated their satisfaction with the utility, appearance, design, and functionality. They were also able to carry out the following actions on the catalog without any difficulty: search for an app, download an app, create a user account, post a comment, reply to existing comments, and access relevant news.

Table 4. Tasks performed—high-fidelity prototype.

N	Task	Users	Completeness	Click
1	Find an app	2	100%	2
2	Download an app	2	100%	3
3	Create a user account	2	100%	2
4	Post a comment	2	100%	2.5
5	Reply to an existing comment	2	100%	3.5
6	Access the original source of an event or news	2	80%	2

shows the tasks proposed during the evaluation of the high-fidelity prototype, the average completion of each task within the digital catalog, and the average number of clicks used to reach target sections.

Table 5. Observations—high-fidelity prototype evaluation.

N	Component	Result
1	Navigate	The participants used the category bar as a search mechanism for the required app.
2	Search	The participants used the search bar to enter a word that was not part of the name of an app, obtaining a satisfactory result.
3	Access	Participants were able to view the available apps from the moment they entered the catalog.
4	Download	Participants were able to access the applications. However, the quality of the network connection delayed the loading process of web apps.
5	Comments	The participants were able to make comments and answers about the apps, after creating a user and logged into their account.
6	News and events	Participants were able to enter the original source of a generic event recorded for the test. However, they also expressed that more information about the event should be shown without redirecting to the original source.
7	App information	Participants were able to view application information when they entered the application interface. In addition, the videos show their attraction to the image gallery of the use of the app.
8	Subsections of the apps	Participants were able to view the recently added apps section both on the home page and in the interfaces of each category.

shows the results obtained from the video captures made during the execution of the test. Again, observations are displayed according to the functional component of the digital catalog.

4.4. Accessibility Test Results

Accessibility test

Table 6. Results of the Accessibility test.

Alert	Section	Positive Attributes	Suggested Items
Main section	3	31	43
Category page	3	13	23
App page	3	5	18
News and events page	3	3	13
Registration page	3	3	12
Login page	3	3	12

shows the accessibility alerts, the positive attributes, and the elements that should be reviewed. In addition, the WAVE tool generated a report for the primary interfaces of the digital catalog. In the current work, the analysis conducted with WAVE allowed for the fixing of errors and features. For example, all contrast errors were reduced. The alternative text was implemented in each of the site’s elements.

4.5. MARS Results

The MARS evaluation was performed by 25 testers on the high-fidelity prototype; the following score was obtained for each quality subscale, as shown in Figure 14.

The average value for catalog quality in this evaluation was 4.45 on a scale of 1 to 5. Therefore, the final quality score is good and higher than the average. Therefore, user satisfaction for these elements means that the UX stages (such as structure, skeleton, and surface) accomplished their goals.

The analysis performed in Section 3.3 found 14 catalogs and websites. In addition, it was possible to provide the catalog with functionalities that some sites lacked, such as feedback, creation of user accounts, download and comment histories, and a management module to manage the website’s content.

Additionally, research existing mobile and web apps for included in the digital catalog of motor disability. A systematic search was performed to find apps. The sources used to apply the searching process were websites found in the previous search, and the Google Play Store platform. The language of the search was Spanish, and search strings were made according to this language. Apps that were selected and subsequently evaluated fulfilled an exclusion process according to the following criteria: duplicate items, articles and news sites, apps that require hardware interfaces, costly apps, and apps related to other types of disabilities were also excluded.

Figure 15 shows the search and selection process based on PRISMA [33]. The identification stage deleted duplicated items, articles, and news sites. The screening stage excluded irrelevant and out-of-context content. The eligibility stage filters by other disabilities, paid services, and non-functional apps. Finally, the Included step brings the final results filtered.

The filtered apps were installed on a computer or mobile device, and a therapist and an end-user performed the assessment. After the test, they answered the set of 18 quantitative questions posed by the MARS tool. Therapists and end-users justified the reason for their score in each MARS quality item. It was handy to carry out the test with these people to obtain the real perception of the interaction with the technology.

The searching process performed in Section 3.2 went through Google, catalogs, and digital platforms like Google Play Store, and returned 391 apps. After PRISMA exclusion criteria, 30 apps were selected to evaluate with the MARS tool.

Table 7. Apps MARS evaluation results.

ID	App Name	Engagement	Functionality	Aesthetics	Information	MARS
01	Skills	4.40	5.00	5.00	3.83	4.56
02	NeuroFit	4.00	5.00	5.00	4.00	4.50
03	Estiramiento Funcional	4.20	4.00	5.00	4.17	4.34
04	Reumapp	4.00	5.00	4.00	3.67	4.17
05	Caminos Accesibles	4.40	4.50	3.67	4.00	4.14
06	Ottobock	3.40	5.00	3.67	4.33	4.10
07	ReHand	4.20	4.50	3.67	4.00	4.09
08	Neuro RHB	3.60	4.50	4.00	4.00	4.03
09	Catch My Pain	3.60	4.50	3.67	4.17	3.98
10	Bocciapp	3.60	5.00	3.67	3.50	3.94
11	Indices de dependencia	3.40	5.00	3.33	3.83	3.89
12	EVA Facial Mouse	3.80	5.00	2.67	4.00	3.87
13	Mugikon	4.00	4.25	3.33	3.83	3.85
14	Mefacilyta Click	3.40	4.25	3.33	4.00	3.75
15	Stimulus	3.60	4.00	3.33	4.00	3.73
16	OrtoTrauma	3.00	5.00	3.67	3.17	3.71
17	Plafones	2.80	5.00	3.00	4.00	3.70
18	Ayudas para todos	3.20	5.00	3.00	3.50	3.68
19	RecuCoach	2.80	4.50	3.33	4.00	3.66
20	Dano Cerebral	3.20	4.25	3.33	3.83	3.65
21	Plaphoons	3.40	3.75	3.33	3.83	3.58
22	ListNote	1.80	4.75	3.67	3.50	3.43
23	Tremor Sense	2.60	4.75	3.00	3.17	3.38
24	Fisioterapia a tu alcance	3.00	4.00	3.00	3.33	3.33
25	HeptoGo	4.40	5.00	4.33	4.50	4.56
26	Anatomy 3D	4.40	4.00	5.00	3.83	4.31
27	PTX	4.40	2.50	3.00	4.83	3.68
28	EViaCam	4.40	5.00	3.33	4.17	4.23
29	Plaphoons Desktop	4.00	4.75	3.00	4.33	4.02
30	JAWS	4.00	5.00	3.33	4.00	4.08

shows the app’s quality assessment results.

It shows the rating that each app obtained in terms of its engagement, functionality, aesthetics, and information. The final value is the total quality obtained. According to the platform, apps are classified as three for desktop, three web apps, and 24 mobile apps. The results determined that 12 of the 30 evaluated apps need an internet connection for operation. Evaluating apps using MARS helped to ensure the quality of the content displayed in the catalog. The following averages were obtained for each quality sub-scale: for engagement 3.63, for functionality 4.56, for aesthetics 3.62, and for information 3.91. Functionality has the higher average of all sub-scales. Finally, 19 apps were included in the site’s inventory due to their quality value being higher than 3.75.

5. Discussion

Most sites analyzed required too many browsing interactions to reach a resource. Search functionality was created to solve this problem that runs within the catalog to achieve the lowest level of navigation required to get an app [48]. Identified sites differ significantly in the quality of resources and the information they provide. The websites that collect information about apps and share it generally do not show complete details, while the websites that develop customized technological solutions offer high-quality software.

Users assessed apps that serve as digital ramps or technical aids as beneficial, valuable resources. However, their qualifications were lower in aesthetics because the apps focus on functionality, not attractiveness. Apps useful in rehabilitation professionals’ activities did not reach the highest scores because they have extensive and sensitive information. For this reason, one of the changes suggested by users was to restrict access to content apps according to the user’s profile. Rehabilitation professionals requested a content restriction for some apps due to the nature of their medical information.

The evaluations performed on each of the prototypes from the beginning of development helped to increase functionality and, at the same time, satisfy the needs that users expressed.

The evaluation of the low fidelity prototype allowed for the fact that the most critical sections of the catalog were the categorization bar and the app screen to be identified. These sections were implemented with a higher hierarchy in the middle-fidelity prototype. From the evaluation of the middle-fidelity prototype, it was possible to obtain users’ feedback about the navigation of the site and the parameter of the apps’ categorization. These features are part of the high-fidelity prototype.

RQ1. Therapists of the “Fundación Hermano Miguel” indicated that the functionality of the digital software catalog focused on motor disability allows easy access to applications focused on this type of disability. In addition, therapists valued the digital catalog as a practical and handy tool when finding technological resources for their activities in the field of motor disability.

RQ2. The evaluations of the high-fidelity catalog allowed us to verify the usefulness of the catalog, its ease of use, and the correct organization of the information within it. Furthermore, it was beneficial to conduct tests oriented to the end-user and tests to software testers because it was possible to obtain the developed solution’s functional perspective and technical observations. When performing different evaluations on the high-fidelity prototype, it was possible to perceive other points of view when issuing an assessment or qualification. The review conducted with therapists and end-users helped to ensure usability. The evaluation managed by software testers helped to find errors, aesthetic facts, and technical aspects that can be improved.

6. Conclusions

The first requirement definition was the primary input to create the digital catalog’s prototype and delimit components. It is essential to mention that the list of functional and content requirements experienced modifications throughout the development process, due to the observations and needs that users exposed during each prototype assessment.

The use of prototyping combined with UX allowed for the development of partial versions of the digital catalog, used to perform tests, and the receiving of feedback from users during the entire developing process. Prototyping facilitated the increase in the catalog functionality, as well as the correction of errors from early stages, while UX helped ensure the satisfaction of user needs.

The categorization of the catalog embraces apps for professional activities, self-care apps, digital badges, alternative communication apps, entertainment and awareness apps, and accessible mobility apps. Categorization allowed for friendly and easy-to-understand navigation for users.

Rehabilitation professionals rated the “digital catalog” as a practical and useful tool to find technological resources for their activities in the motor disability context. The therapists adopted the catalog resources to speed up their activities and reduce physical barriers—for instance, the use of a postural test app instead of a mirror with a grid.

As for future work, the next step is to design apps for an intellectual disability digital catalog, visual disability and hearing disability to compliment the current research.