Web Accessibility Testing for Visually Impaired People in Indonesia ^†

Abstract

An equal society requires respect for differences and attention to minority groups. Web inclusivity for disabled people is closely related to how they can understand, use and utilize websites like non-disabled people. The objective of this research was to analyze whether web accessibility has been implemented in Indonesia based on the availability and functionality of the requirements for visually impaired people. Tests were performed manually and automatically on 30 samples. Automated tests were performed by validating the HTML DOM elements of the web page. The results showed that 222 out of 240 tests failed because specified requirements were not found. Based on this research, web accessibility in Indonesia is deficient. Websites should provide accessibility features to assist visually impaired people access information and services.

1. Introduction

People with disabilities have higher vulnerability to experiencing exclusion in society. The leading cause is their elaborate particular needs that are more intricate to apply to public services and facilities than the standard design used for non-disabled people. National development plans often do not meet their particular needs. There is a low possibility that disabled people can enjoy the results of national development, and it is difficult for them to play an active role in national development [1,2,3]. The government and private businesses need to actualize disabled people’s needs so they can be at ease without discrimination [4,5].

Disabled people’s awareness of their rights and their commitment to embody it must come from society. The government shows support for people with disabilities by enacting and issuing laws and regulations that set equal rights for them [1,6,7,8]. However, actualizing equal rights for disabled people has not been maximally implemented [1,8,9,10]. The biggest challenge to creating inclusivity for disabled people is the lack of their participation and their chances to socialize to make their needs acknowledged [11,12,13].

Information technology has been a significant support in people’s daily lives during the COVID-19 pandemic, where most activities are carried out online [14,15,16]. Therefore, information and public services have also shifted to digital platforms through websites [17,18]. Web applications are often the only available way for people to access some services, such as health, public administration, banking, education and entertainment.

Web inclusivity for disabled people is closely related to how they can understand, use and utilize web applications like non-disabled people. Web visual and functional design should not reduce their ability to access information and services. However, Indonesia is ranked 66th out of 120 other countries on the Inclusive Internet Index [19], with an accessibility score of 69.75, which does not meet the Web Content Accessibility Guideline [20] standards. This poor score is due to faint web accessibility implementation in Indonesia for people with disabilities on government and university websites, of which 95% do not support accessibility for disabled people [21].

This research focuses on the availability of web modalities for disabled people with limited vision. One of the hindrances of typical web design for visually impaired people is that web information is presented in small text/objects without audio information assistance. For instance, nearsighted users find it difficult to read information in small text sizes that non-disabled users can generally read.

Requirements that can be implemented on the web to help visually impaired users maximize the use of the web are object magnification, contrast settings, information in the form of sound and adjustable text sizes [20,22,23]. The purpose of this research is to analyze whether web accessibility requirements for users with limited vision have been implemented in a sample of websites that Indonesians widely use to obtain information and services. Hopefully, this research can provide information on web accessibility in Indonesia as a part of technology service improvements for visually impaired people.

2. Method

Based on the research flow in Figure 1, the first stage of the research is Problem Identification. At this stage, secondary research was conducted on the condition of disabled people’s access to information. Based on the 2018 National Socio-Economic Survey (SUSENAS), access to information for disabled people using mobile phones or laptops was only 34.89%, and internet access for people with disabilities was only 8.5% [24]. Considering Undang-Undang No. 8 of 2016 on Persons with Disabilities, the Ministry of Social Affairs seeks to provide disabled people with equal opportunities, one of which is equal access to information [25]. However, most government and university websites in Indonesia are not disabled friendly, and the accessibility of information technology has not been widely applied [21]. Therefore, this research analyzes the accessibility of several websites in Indonesia for users with limited vision.

The second stage is Accessibility Requirements Identification. Web accessibility requirements for visually impaired users were collected and selected through a literature review. Requirements related to visual impairment were obtained through sources in guidelines, laws and previous research. Then, these requirements were outlined and grouped based on similarities. The requirements listed in two or more sources were selected to be tested.

The third stage is Test Scenario Preparation. A test scenario was designed to verify the existence of web elements that implement the requirements’ functionality. The fourth stage is Website Sampling. This stage was carried out randomly on the websites that Indonesians use to obtain information and services in education, health, civil, employment, banking, law and entertainment.

The fifth stage is a Manual Test in the form of exploratory tests. The Manual Test was carried out to find unknown web elements in each sample. Once discovered, several accessibility feature usage scenarios were tested for functionality. The sixth stage is Elements and Keywords Identification. For each accessibility feature that passed the Manual Test, its HTML element was identified to find keywords that point to that element. Keywords were used as coordinates in the test automation script. The seventh stage is Test Automation. The automation was performed by accessing the HTML DOM of a web page, and it verified the presence of elements based on coordinates.

The last stage is the Conclusion of the Results. The results obtained from manual and automated testing were compared. If there were differences in the results between manual and automated tests, then the anomalies that affected the differences in results were rechecked. Then, a descriptive analysis was conducted on what requirements were applied and how they affected web accessibility in Indonesia.

3. Results and Discussion

Relevant data regarding web accessibility requirements for visually impaired users were collected from three sources.

Table 1. Web accessibility requirements for visually impaired users.

Requirement	Type of Visual Impairment	Code
Auditory information	Blindness; Low vision	A
Adjustable text size	Low vision	B
Object magnification	Low vision	C
Contrast settings	Low vision	D
Written information on a color-based action	Color blindness	E

lists information on the requirements identified in the sources reviewed. We gathered five requirements, each was given a representative alphabetical code, and visual impairment types included blindness, low vision and color-blindness.

Table 2. Accessibility standards and guidelines.

Resource	A	B	C	D	E
Americans with Disability Act (ADA)	O	O	O
The European Parliament and the Council of the European Union	O	O	O	O	O
Web Content Accessibility Guidelines (WCAG) 2.1	O	O		O

shows the association between the requirements and the party issuing the guidelines. An O indicates that the requirement was found at the source.

Based on the requirements appearing in two or more sources, four were selected from Table 2 to be used in this research. The requirements are as follows:

• R1: Auditory information;
• R2: Adjustable text size;
• R3: Object magnification;
• R4: Contrast settings.

The test scenario template was designed as below.

• Open browser;
• Open sample web page;
• Visually verify the availability of the requirements on the homepage;

  3.a.

  If one or more requirements are found;

  3.a.1.

  Check if the features can be utilized;

  R1

  Check if the audio can be played and produce a clear sound;

  R2

  Try all the available text size options and see if they work;

  R3

  Try playing with the magnifying feature and see if the object looks bigger than before;

  R4

  Try all the available contrast options and see if the visuals change;

  3.a.2.

  Record the results and proceed to step 4;

  3.b.

  If no requirements are found;

  3.b.1.

  Perform an exploratory test on some sample web pages to look for the existence of the requirements;

  3.b.1.1.

  If the requirements are found, proceed to step 3.a.1;

  3.b.1.2.

  If none are found, proceed to step 4;

• Close the browser.

The samples that were used were 30 HTML elements of all samples that passed the manual tests, and they were examined to acquire keywords. The collected keywords are listed in

Table 3. Keywords of the requirements.

Keyword	Requirement
resizer; font-smaller; font-larger; font-normal; font-readable; compact mode; text magnifier; readable font; adjust font sizing	Adjustable text size (R2)
content scaling	Object magnification (R3)
contrast; contrast = default; contrast = night; contrast = high contrast; tema; theme; terang; bright; gelap; dark; default perangkat; device default; id = “menu-darkmode”; class = “icon-svg icon-darkmode”; data-dark = “system”; data-dark = “dark”; data-dark = “light”; dark mode; highlight titles; highlight links; color adjustments; dark contrast light contrast; high contrast; monochrome; high saturation; low saturation; appearance; device theme; dark theme; light theme	Contrast settings (R4)

. The keywords were found in Indonesian and English. Each keyword was used with lowercase, uppercase and capitalization to increase the probability of finding the requirement, because the coordinates in the automation script were case sensitive. The implementation of the R1 requirement was not found. Instead, we used the <audio> tag; this was referred from [26] (Figure 2).

The automation script was written using the Robot Framework (Robot Framework Foundation, Helsinki, Finland). Lines 1–5 define the general settings of the test, such as a set of actions before–after running a suite, after running a test and the resource/library used. Lines 6–9 define variables to store constantly changed values for the sample’s URL, a path with each keyword in Table 3, and the timeout based on the responsibility of the web sample. Lines 10–16 define the test case developed based on the manual test scenario. First, it goes to a specified web address and then waits for the page to load completely. Next, it can directly verify the element, or it needs to go through a few steps before the element becomes visible. For example, the element was nested in a burger menu, so it clicked on the burger menu element before verifying the keyword element.

The results of the manual and automated tests are attached in

Table 4. Manual and automated test results.

Num.	Sample	Manual	Automated
1	The Indonesian Government’s Official Website Regarding COVID-19	X	X
2	Ministry of Communication and Information of the Republic of Indonesia	X	X
3	Tax Official Website of Indonesia	X	X
4	BPJS Kesehatan (Health Insurance)	X	X
5	Bank Rakyat Indonesia	X	X
6	Rumah belajar of the Ministry of Education, Culture, Research and Technology of the Republic of Indonesia	X	X
7	PT Kereta Api Indonesia (Indonesian Railways)	X	X
8	Kampus Merdeka	X	X
9	Indonesian National Police	X	X
10	Pedulilindungi	X	X
11	Indonesian Immigration Official Website	X	X
12	Population and Civil Registration Office of Salatiga City	X	X
13	Salatiga City District Court	X	X
14	Indonesian Agency for Meteorological, Climatological and Geophysics	X	X
15	Central Java Job Market	X	X
16	Alodokter	X	X
17	Pinterest	X	X
18	Tokopedia	X	X
19	Blog Ruangguru	X	X
20	Google Indonesia	R1 X R2 X R3 X R4 O	X
21	Persatuan Tunanetra Indonesia	R1 X R2 O R3 X R4 O	R1 X R2 O R3 X R4 O
22	Yayasan Mitra Netra	R1 X R2 O R3 X R4 O	R1 X R2 O R3 X R4 O
23	Youtube	R1 X R2 X R3 X R4 O	R1 X R2 X R3 X R4 O
24	CNN Indonesia	X	X
25	Kompas	R1 X R2 X R3 X R4 O	X
26	LinkedIn Indonesia	X	X
27	Kapanlagi	X	X
28	Halodoc	X	X
29	Facebook Indonesia	R1 X R2 O R3 X R4 O	X
30	Bank Central Asia	X	X

. Code X states that the requirements were not found, NW means that the requirements were found but did not work, and O means that the requirements were found and could be used. There were differences in the manual and automated testing results in the Google, Kompas and Facebook Indonesia samples. Details of the anomaly in the results are described in

Table 5. Anomalies of differences in test results.

Sample	Requirement	Anomaly
Google	Contrast settings (R4)	Failed to detect the element of contrast settings because Google blocked the robot used in the automated test.
Kompas	Contrast settings (R4)	The keyword id = “menu-darkmode” did not lead to one specific object, so the robot failed to detect it. This problem was resolved by changing the keyword to class = “icon-svg icon-darkmode” through XPath and had to be written the same for it to be detectable.
Facebook Indonesia	Adjustable text size (R2) and Contrast settings (R4)	The element’s path could not be read or encrypted. Example: class = “d2edcug0 hpfvmrgz qv66sw1b c1et5uql lr9zc1uh a8c37 × 1j fe6kdd0r mau55g9w c8b282yb keod5gw0 nxhoafnm aigsh9s9 d3f4 × 2em mdeji52x a5q79mjw g1cxx5fr lrazzd5p oo9gr5id hzawbc8m”

.

The results show that 80% of the samples were not accessible to visually impaired people. Samples number 1–15, government-owned websites in education, health, civil, employment, banking and law, did not implement any of the specified requirements. Samples number 16–19, 24, 26–28 and 30 were also absent of these requirements.

Based on the results, various parties pay little attention to visual impairment accessibility when developing websites. Furthermore, websites owned by organizations focusing on visually impaired people only implement two of the requirements, which are adjustable text sizes (R2) and contrast settings (R4).

The key to optimizing the proposed automation testing is building the precise path from the keyword used for locating the accessibility element. Finding the keyword that leads to a specific element can avoid the anomaly of the machine not being able to verify the accessibility element despite it being present on the web page.

4. Conclusions

The implementation of web accessibility requirements for visually impaired users is deficient. Based on the results, only 14 tests passed out of 240 tests, and only 4 failed due to technical anomalies of automated testing. Therefore, the government and various parties are encouraged to implement accessible features to help visually impaired users access information and services optimally. In the future, it is recommended to research other requirements and other types of disabilities to support information dissemination and the awareness of poor web accessibility in Indonesia. To achieve website inclusivity, we must care about accessibility for minority groups.