Next Article in Journal
Computational Simulation of an Agricultural Robotic Rover for Weed Control and Fallen Fruit Collection—Algorithms for Image Detection and Recognition and Systems Control, Regulation, and Command
Previous Article in Journal
Sensorless Control of Seven-Phase PMSM Drives Using NSV-SVPWM with Minimum Current Distortion
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Electronics
Volume 11
Issue 5
10.3390/electronics11050788
electronics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Conceptual Framework for Determining Quality Requirements for Mobile Learning Applications Using Delphi Method

by
Mohammed Amin Almaiah
1,*,
Fahima Hajjej
2,
Abdalwali Lutfi
3,
Ahmad Al-Khasawneh
4,
Tayseer Alkhdour
1,
Omar Almomani
5 and
Rami Shehab
1
1
Department of Computer Networks and Communications, College of Computer Sciences and Information Technology, King Faisal University, Al-Ahsa 31982, Saudi Arabia
2
Department of Information Systems, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
3
College of Business, King Faisal University, Al-Ahsa 31982, Saudi Arabia
4
Department of Cyber Security, Irbid National University, Irbid 21110, Jordan
5
Computer Network and Information Systems Department, The World Islamic Sciences and Education University, Amman 11947, Jordan
*
Author to whom correspondence should be addressed.
Electronics 2022, 11(5), 788; https://doi.org/10.3390/electronics11050788
Submission received: 20 January 2022 / Revised: 14 February 2022 / Accepted: 17 February 2022 / Published: 3 March 2022
(This article belongs to the Section Computer Science & Engineering)
Browse Figures
Versions Notes

Abstract

:
The development of mobile learning apps might fail due to poor selection of the suitable technical requirements for mobile devices. This will affect the quality of mobile learning applications and, thus, will increase the development cost of mobile learning apps. Due to the above issues, we need to determine the most appropriate technical quality requirements for the development of mobile learning apps that meet user requirements. To achieve that, we propose a comprehensive framework to capture the most suitable technical quality requirements for mobile learning apps. A Delphi method was used to collect, evaluate, and analyze the data for this study. As a result of our Delphi study, we have identified nineteen technical quality requirements, divided into six quality dimensions, for the development of mobile learning applications. The proposed framework is expected to be a guideline for mobile apps designers and developers to successfully develop mobile learning apps.

1. Introduction

The concept of Quality in this study reflects a degree of excellence of learning content quality and learning service quality of the M-learning system [1]. The importance of quality factors has been widely examined in prior research in several fields, such as e-learning quality [2,3,4], learning management system quality [5], and M-learning quality [6]. In the context of mobile learning, quality factors is defined as a set of requirements that ensure the high quality of learning activities, design of learning content, and quality of learning services. These requirements can be used as guidelines for ensuring the successful development of mobile learning systems [7,8,9]. Determining quality requirements for mobile learning systems is considered one of the critical steps during the development process [10]. If quality is a prerequisite for the success of the educational process, quality becomes a necessary problem for M-learning, in particular. The success of any educational system is highly dependent on its commitment to internationally agreed quality standards. Recently, the rapid transition to mobile learning application has won, and the knowledge gained can be applied [11]. This lesson, learned through COVID-19, will force a period of novel resolutions for future studies. Therefore, the quality of mobile learning is an important issue during COVID-19. Few studies discuss how to improve the mobile learning quality, for users, needed during COVID-19 [12].
Although previous studies have developed several models and frameworks into mobile learning [13,14,15,16], there are no clear and well defined technical quality requirements and specifications for mobile learning applications. In order to address this gap, the current study aimed at establishing framework to identify the most critical technical quality requirements for mobile learning that aid the development of mobile learning applications. By identifying the most significant technical quality dimensions and requirements, there exists a better opportunity for success in the implementation of a mobile learning applications.
Thus, this research provides an important contribution to the literature through answering the following two main questions:
  • What are the appropriate technical quality requirements for mobile learning applications?
  • What technical quality requirements from question one lead to the development of mobile learning applications successfully?

Related Works

According to the existing literature, a number of studies aimed to evaluate the main aspects affecting acceptance, adoption, usage, and implementation of M-learning, and the benefits universities derive from M-learning systems have been increased [17,18,19]. For example, Almaiah et al. [20] proposed a new model to identify the most important factors that could motivate students to accept and use of M-learning system. They identified four success factors of mobile learning, which were subdivided into categories, including (i) innovative factors (security, protection, similarity, relatively favorable position, and trust), (ii) hierarchical components (protection from change and technological availability), (iii) social elements, and (iv) quality variables (nature of the framework, nature of substance, and nature of administration). Then again, Almaiah et al. [21] inspected the impact of various components on M-learning applications’ improvement at three primary phases of utilization (static stage, association stage, and exchange stage).
The outcomes demonstrated that the main variables, identifying with the point of view of the user, to think about when creating M-learning in three phases were framework similarity, security, data quality, awareness, seen practical advantage, self-viability, accessibility of assets, and trust [22]. Quantitative investigation with 275 undergrad Jordanian understudies at the college of Jordan called attention to that trust, seen security, seen the convenience, and saw handiness, which are fundamental variables for effective selection and usage of M-learning framework. Additionally, Almaiah [23] proposed a structure for M-learning acknowledgment dependent on integrating the Technology Acceptance Model (TAM) with the refreshed DeLone and McLean’s model (DL&ML). The examination intended to research the impact of value components and individual variables on students’ fulfillment and expectations for the utilization of the M-learning network. The outcomes presumed that quality components, identifying with framework quality, data quality, and administration quality, are fundamental measurements for guaranteeing understudies’ fulfillment and goals with the utilization of the M-learning framework. Likewise, Barnes et al. [24] proposed a half-breed quality model for M-learning, dependent on researches of DeLone and McLean data framework achievement model (DL&ML) with the TAM model, to look at the impact of 10 quality measurements on M-learning framework acknowledgment. They uncovered that the most basic components, identifying with increments of the students’ acknowledgment, were content plan quality, usefulness, UI configuration, learning content quality, openness, responsiveness, personalization, and intelligence. Another investigation, Hsu et al. [25], created three structures for M-learning advancement dependent on quality variables received from the refreshed DeLone and McLean data framework achievement model.
Finally, a recent study conducted by [26] investigates the important factors that affect students’ acceptance of mobile learning. The study applied the Unified Theory of Acceptance and Use Technology (UTAUT) model and revealed that perceived data quality, similarity, trust, sense of awareness, and accessibility of assets, self-adequacy, and security are the principal sparks of understudies’ acknowledgment of the M-learning framework and, subsequently, achievement in the execution of M-learning projects.
The majority of M-learning studies have been considered students’ perceptions for determining the factors that affect acceptance, adoption, and usage of mobile learning, while few studies have paid attention to users’ perceptions of quality factors for M-learning applications. Previous papers ignored the fact that quality dimensions serve as important roles for meeting students’ perceptions and ensuring the successful development of M-learning applications, and, therefore, it is important for examining such factors. The existing literature offers little insight into the quality factors of M-learning systems. There are a limited number of examples that look at this, including the studies by [27,28], which identified various M-learning quality factors in educational institutions. However, as far as we have been able to determine, few of these studies have empirically identified the important quality factors of M-learning applications’ development. Consequently, this study aims to provide a comprehensive model to capture the most suitable technical quality requirements for mobile learning apps.

2. Background of the Study

For the successful development of technical quality requirements framework, several software quality models need to be taken into account [29,30,31]. Table 1 shows an overview of the previous models on software quality in the literature, information system quality models, and service quality models that play a role for the development quality standards, factors, and characteristics of software, systems, and applications [31,32,33]. Therefore, the quality models have been employed as a foundation for constructing the proposed framework for our research.
Table
Table 1. Review of Previous Quality Models and Their Dimensions.
Table 1. Review of Previous Quality Models and Their Dimensions.
TypeQuality ModelsQuality DimensionsAuthors
Software quality modelsMcCall’s quality modelreliability, usability, correctness, efficiency, interoperability, integrity, maintainability, testability, flexibility, portability, and reusability[34]
Boehm modelportability, utility, reliability, efficiency, maintainability, understandability and modifiability[35]
ISO 9126/IEC
model		functionality, usability, reliability, efficiency, maintainability, and portability[36]
Information system quality modelsThe updated DeLone and McLean modelsystem quality (usability, functionality, interface design, accessibility, ease of use, interactivity), information quality (content adequacy, content usefulness and content design) and service quality (availability, personalization, reliability, trust and responsiveness)[37]
Service quality
models		E-S-Qualefficiency, compliance, availability and
privacy		[38]
WebQual modelinformational fit-to-task, tailored communications, trust, response time,
ease of understanding, intuitive
operations, visual appeal, innovativeness,
emotional appeal, consistent image, on-
line completeness, and relative advantage		[39]
WebQual modelusability, information and interaction[39]

2.1. Software Quality Models

Regarding software quality dimensions, many frameworks were analyzed, including the McCall’s quality model, which included 11 quality dimensions (namely: reliability, usability, correctness, efficiency, interoperability, integrity, maintainability, testability, flexibility, portability, and reusability) developed by Joseph et al. [34]. The McCall’s model led to the development of Boehm model by Hamid et al. [35], which included seven dimensions (portability, utility, reliability, efficiency, maintainability, understandability, and modifiability). After that, Software and Systems Engineering of International Organization for Standardization developed ISO 9126/IEC model [36]. This model contains six quality dimensions, which are: functionality, usability, reliability, efficiency, maintainability, and portability. These models were employed as a foundation for many new methods and studies, as shown in Figure 1 and Figure 2.

2.2. Information System Quality Models

In the information system quality factors context, we analyzed the updated DeLone and McLean model, developed by Almaiah et al. [37], with a huge focus on three main dimensions (namely: system quality, information quality, and service quality). Each one of these dimensions is divided into sub dimensions as the following: system quality (usability, functionality, interface design, accessibility, ease of use, and interactivity), information quality (content adequacy, content usefulness, and content design), and service quality (availability, personalization, reliability, trust, and responsiveness). These became the indices to predict the quality and success of many types of information systems.

2.3. Service Quality Models

Finally, in the service quality dimensions context, we analyzed the E-S-Qual model, developed by [38], which contained 22 items for four dimensions (efficiency, compliance, availability, and privacy). Loiacono et al. [19] developed the WebQual model, which uses a basic scale of twelve dimensions. Subsequently, we studied an approach that shared the same name: the WebQual model by [19], with a main focus on three dimensions (usability, information, and interaction). We also reviewed the service quality attributes (availability, personalization, reliability, trust, and responsiveness) in the updated DeLone and McLean model developed by [37].

2.4. Research Contribution

After conducting a systematic review in the literature, as shown in Table 1, and presenting the comparison between different models of software quality factors, we proposed a preliminary list of technical quality dimensions for M-learning system. These dimensions could be utilized as a guideline to enhance the quality services of m-learning applications that meet users’ requirements. These dimensions can lead to the successful development of m-learning applications. Based on that, this study proposes a technical quality framework for M-learning systems, based on software and information system quality models such as McCall’s quality model, Boehm model, ISO 9126/IEC model, the updated DeLone and McLean model, E-S-Qual, and the WebQual model. The next section presents the method used in this study in order to achieve the research objectives.

3. Methodology

In order to achieve the research objectives of this study, we proposed a comprehensive framework to capture the most suitable technical quality requirements for mobile learning apps. A Delphi method was used to collect, evaluate, and analyze the data for this research. Based on that, the research methodology was divided into five main steps as the following:
  • Step One: a systematic review has been conducted of various types of quality models.
  • Step Two: we established an initial list of quality requirements from the different types of quality models.
  • Step three: we conducted the Delphi method in order to evaluate the list of quality requirements that have been identified in Step Two.
  • Step four: we identified the final technical quality dimensions and requirements from the Delphi analysis.
  • Step five: presentation of the framework of the technical quality dimensions and requirements for mobile learning applications.

3.1. Validity Analysis

In order to analyze the validity of the quality measurements for this study, two types of validity analyses were used. First, convergent validity analysis was performed using Average Variance Extracted (AVE), as presented in Table 2. The results of AVE analysis indicate that all values are greater than the minimum cut-off criteria of 0.5 [40]. This means that all convergent validity values, for all quality dimensions in this study, are suitable for the next analysis step.
Second, a discriminant validity analysis was conducted by evaluating the square root of AVE values for each quality dimension, as suggested by [41]. Based on the results in Table 3, all values of square root of AVE were greater than correlation values between two dimensions.

3.2. Delphi Study

In our study, we applied the Delphi study in order to reach to the consensus among experts about the most important quality requirements for ensuring the development of mobile learning applications successfully. According to [42,43,44,45], Delphi method consists of five main steps: (1) consult several experts in the field, (2) in an obscure way, (3) in number of rounds, (4) collect the data from experts and analyze the data, and finally, (5) give the experts more than one opportunity to review their answers in different rounds to reach the consensus [46]. However, the Delphi technique was selected due to several reasons: (1) it helps the researchers achieve the research objective through reaching the consensus among experts, and (2) it helps to develop the framework, which is considered one of the outcomes of the Delphi approach, according to [2,3]. Based on that, Delphi method is suitable for the study that helps researchers develop a comprehensive framework to capture the most suitable technical quality requirements for mobile learning apps, based on a consensus approach among experts.
For conducting the Delphi study, a quantitative method was employed to collect the data. The Delphi study included three different rounds in order to collect and analyze the data from experts. The data collection for the three rounds of the Delphi study was conducted for six months from June 2021 to January 2022. The results of the Delphi study will be discussed in the sections below.

3.3. Experts Selection

In the Delphi study, selection of the suitable experts in the field is considered one of the crucial criteria to success of the method [5]; this means that all participants in this study should be experts in the field of mobile learning, e-learning, or software engineering for the development of educational applications. In our study, we have selected 30 experts with high knowledge and experience in their fields, and they participated in different mobile learning development projects from different universities in Jordan and Saudi Arabia.
For conducting the Delphi study, 30 respondents, distributed across three groups, participated in the study. The first and second groups included 24 experts in the fields of Soft-ware Engineering and Information Systems, as well as six experts in Mobile Learning in the third group. All experts have participated in the three rounds of the Delphi study.

4. Conducting the Delphi Study and Analysis of the Results

4.1. First Round: Design of the Preliminary Quality Dimensions and Requirements Based on the Literature Review

To evaluate the initial list of technical quality dimensions requirements for mobile learning applications, we employed a Delphi study with 30 experts in Software Engineering, Information Systems, and the Mobile Learning field. For collecting the feedback from experts, we developed questionnaire items in the survey, based on quality dimensions that have been determined from the analysis of quality models from the literature. This step is very important in order to ensure the validity of items in the survey, and it is suitable for the development of mobile learning applications. Based on that, in the first round, we selected the open-ended questionnaire method that make experts have more important roles in determining the most suitable quality requirements, as compared with other survey methods, as mentioned by [6]. The first sections of the survey focused on the general question: ‘Which quality dimensions are important for the development of mobile learning applications?’ They were also asked to assign a domain to each dimension. Additionally, the respondents could indicate new dimensions and requirements.

Results of the First Round

After analyzing the data that have been collected from 30 respondents, a preliminary list of quality dimensions was formed. It comprised 21 technical quality requirements, divided into six quality dimensions (Table 4—preliminary list of dimensions). This list served as basis to the first round of the Delphi study, as shown in Figure 3.

4.2. Second Round: Identification of Important Technical Quality Dimensions and Requirements for Mobile Learning Applications

In the second round, a quantitative survey was used that included the results from round one (21 technical quality items) in order to reach expert consensus about which technical quality dimensions and requirements are regarded as most important. The experts were requested to evaluate and score each quality dimension by using a scale of six-point Likert, from necessary to none important. In addition, the survey included a section to allow experts to add more comments or feedback if they desired.
In the Delphi studies, there are several methods that can be used to compute the consensus value. One of the most crucial methods is the median and interquartile range (IQR), according to [7]. Based on that, in our study, we used the IQR method to compute the consensus values for each quality item in the quantitative survey in order to identify the importance of each quality item based on multiple experts’ opinions in the field. The median of IQR value indicates the middle points that fall between the highest scores and lower scores, as mentioned by [8]. Based on that, any median of IQR value greater than 0.5 indicates it to be necessary or very important. Nevertheless, when the median of IQR value is less than 0.5, it indicates that it is not important. These criterions have been used to consider which quality items can be moved to the next round or skipped.
As mentioned above, the IQR is the most important scale to measure the degree of consensus in a Delphi study [9]. The small values of IQR mean a high degree of consensus, while high values of IQR mean low degree of consensus [10]. In the analysis of IQR in our study, we used three different levels based on the IQR values. First level, when the values of IQR are lower than 0.5, means that the consensus is located in the high level. Second level is moderate when the values of IQR between 0.5 and 1. Third level, when the values of IQR are higher than 1, means that the consensus is in the low level. The results of the second round analysis are presented in Figure 4.

Results of the Second Round

Table 5 presents the analysis results of IQR values for the 21 technical quality requirements resulting from the second round. After this round, one out of five technical quality requirements items was left within the dimension of functionality. In addition, at the learning content quality dimension, one out of three items was removed. Notably, no technical quality requirement items were left within the quality dimensions interactivity, accessibility, interface design, and content design quality, suggesting that there is consensus regarding the importance of technical quality requirements related to these quality dimensions for the successful development of mobile learning applications. There is high consensus for 14 technical quality requirements and moderate consensus regarding the importance of 5 technical quality requirements. Based on the second round results, 19 technical quality requirements, with a median ≥ 5 and IQR ≤ 0.5, were retained because the majority of the respondents agreed that these dimensions are very important for the development of mobile learning applications.

4.3. Third Round: Consensus Confirmation of the Final Technical Quality Dimensions and Requirements

In round three of Delphi study, all experts had an opportunity to revise their feedback and scores, after analyzing the results in the second round, in a new round. In the third round, we sent 21 technical quality requirements, resulting from the second round, to the experts in order to reevaluate their previous responses. To do that, we sent the results of the IQR analysis that resulted in the second round. Experts were requested to score these 21 quality requirements another time for the final round. In this case, we gave the experts another opportunity to re-score the quality items in order to reach the consensus from all experts. After collecting the data, we analyzed the results using IQR measure through applying the same analysis steps that were conducted in the round two.

Results of the Third Round

Majority of the respondents agreed about the importance of nineteen technical quality requirements (median ≥ 5 and IQR ≤ 0.5) for the development of mobile learning applications after the third round (see Table 6). Interestingly, there was more consensus in the third round (a shift from moderate to high for two technical quality dimensions). The first is about the need for applications that enable learners to interact with instructors for the interactivity dimension, and the need for applications that provide learners different formats of learning content, such as text, audio, and video for the content design quality dimension is the second.
Based on that, there is high consensus for 16 technical quality requirements and moderate consensus regarding the importance of 3 technical quality requirements, compared with the second round. Finally, the results of our study allowed us to identify the final technical quality requirements for the development of mobile learning applications with a high consensus of respondents as shown in Figure 5.

5. Discussion

In this work, the current Delphi study developed a new framework of 19 technical quality requirements, divided into 6 quality dimensions, regarded as very important by majority of the respondent groups. Figure 6 summarizes the most important technical quality dimensions and requirements in the proposed framework. All these dimensions and requirements are discussed below. Finally, we present the limitations of this research and make recommendations for future work.

Technical Quality Dimensions and Requirements Framework for Mobile Learning Applications

Interactivity in mobile learning is defined as the quick and real interaction between learners and teachers, as well as among learners themselves, by using the mobile applications [2,3,4]. If learners can be able to interact and communicate effectively with teachers and peers via mobile applications, this will make these applications the better and more useful option for learning. Mobile learning applications should accommodate both synchronous and asynchronous communications. This communication through mobile learning applications can lead to creating an effective collaborative learning environment between learners and support the quick response in real time. Interactivity factor plays a critical role in the success of mobile learning applications because it improves instant response, collaboration, and interaction in real time, regardless of time and place. Mobile learning applications should accommodate all technical characteristics and mechanisms of interactivity, such as an online chat room, discussion room, online message board, and instant messenger. These requirements should be taken in the consideration of designers in order to develop high quality mobile learning applications. Therefore, designers should develop a mobile learning application that enable learners to interact with teachers, as well as create mobile learning application discussion boards that allow them to easily exchange and share learning content, and give them the chance to discuss their ideas with learners and faculty instructors, which leads to improved quality of mobile learning.
Functionality of mobile learning application refers to the necessary and effective features and functions of application that meet learners’ needs and perform their learning activities effectively [1]. The application should include all important features in order to provide useful and effective learning experiences for both learners and instructors using mobile devices. Functionality can be divided into a number of attributes, including operability, navigability, suitability, notificationability, and accuracy. These functionality requirements must be taken into account during the development of mobile learning applications. Operability refers to the capability of an application to adapt with different mobile application platforms and devices. Designers should develop mobile learning applications that support multiple mobile platforms such as Android, IOS, and Windows, which leads to improving the use of applications by all learners, regardless of the type of mobile devices. Navigability refers to the easy navigation and access of functions and tasks of application. Mobile learning applications should include all technical features and mechanisms of navigability, such as links, scroll bars, and buttons that enable learners to easily navigate between learning tasks and activities. Suitability refers to the ability of a mobile learning system to provide appropriate features and functionalities to fulfil learners’ requirements. Notificationability refers to the ability of mobile learning applications to give learners alerts for new notifications. This new feature enables learners to keep in touch with all updated actions from instructors, such as adding new courses, new announcements, new assignments, and others. Accuracy refers to the ability of mobile learning applications to provide appropriate information and results.
User interface design is a crucial factor for a successful application. Thus, designing and developing an efficient interface within a learning environment is still a challenge for most developers, facilitators, and designers [3]. Additionally, [3] stated that the interface for mobile must be more consistent and straightforward than those of e-learning. He believes that, if the mobile navigation must be learned to use, it is a failure [5]. Similarly, [4] stated that mobile learning applications must be simple and intuitive [7]. Furthermore, [6] urged developers of mobile learning applications to design attractive and easy to use interfaces, a pleasant visual design, and effective interaction styles [34]. In addition to instructional and interface design, the organization of visual elements and media on the mobile screen will influence the ease and quality of learning, and it has an important impact on learners’ cognitive load. It is also important to consider the number of pixels available on users’ devices to provide the best quality of images on users’ screens. Furthermore, designers should consider screen size and screen orientation (Horizontal and Vertical), knowing that learners sometimes need to be able to use both orientations.
In this study, mobile learning accessibility refers to the degree of ease of access for students to the learning content via mobile learning applications. Accessibility refers to the degree of ease of how a user can access and use the information extracted from the system [2]. Almaiah et al. [1] expressed that system accessibility refers to the degree of ease that enables students to access and use e-learning system. Practically, when mobile learning provides students online access, and downloads the learning materials when and where they need via mobile learning applications, they will perceive that the mobile learning is an easy to use and useful tool for learning.
Stakeholder considers the quality of learning content as a critical factor that motivates learners to use mobile learning applications. Mobile learning applications should enable learners to easy access to appropriate learning content. Content quality refers to the quality and accuracy of content, which is provided by the information system [22]. Learning content quality is very important in mobile learning applications in order to fully engage learners in the learning process. According to Almaiah et al. [1], learning content quality is a crucial factor in supporting the success of mobile learning applications because it demonstrates the actual use of mobile learning applications among learners. Mobile learning can make the learning process very interesting by giving learners the chance to access learning materials through their mobile devices, such as smartphones, which leads to enhanced flexibility of learning and participation, regardless of place and time. Learners expect mobile learning applications to offer the appropriate learning contents, such as lectures, courses, assignments, images, and quizzes. Various learning contents should be considered in the design of mobile learning applications. Mobile learning application designers should create various learning activities and styles to accommodate learners’ requirements to interact with learning content in different ways. If learners do not find the required learning contents in the mobile learning applications offered by universities, this may lead to failure in the use of mobile learning advantages. Mobile learning application designers should develop separate learning content, and it should not be the same learning content that is used in the computer-based applications [11].
The success of mobile learning applications basically depends on how the learning content is designed in order to meet learners’ perceptions, and this may lead to improving the learning process via mobile applications [5]. Content design quality in mobile learning is defined as the format and type of learning content that is presented by the application. The mobile learning application should accommodate multiple styles and formats of learning contents that are necessary to provide an effective learning experience. Practically, the design quality of learning content depends on the users’ perceptions, and therefore, the mobile learning application must be able to support different learners’ preferences of learning content styles and formats [6]. These styles and formats should be taken into account during the design phase of a mobile learning application. The design quality of mobile learning content should be acceptable by learners. To meet that, designers should develop mobile learning applications as collaborative learning platforms that enable learners to share and send learning content files to instructors, and they should design multiple learning styles such as multimedia learning contents (audio, video, and animation) and basic learning contents (text, graphics, and charts).

6. Conclusions

The present work tried to answer the proposed objectives, which involved the development of a framework to identify technical quality requirements for mobile learning applications and help those responsible for the development of mobile learning applications to identify and evaluate which technical quality requirements are necessary for mobile learning application development. A Delphi study was set up to identify technical quality dimensions and requirements, perceived to be important for the successful development of mobile learning applications, from the perspective of multiple stakeholders. After three rounds of data collection, there was consensus about the importance of nineteen technical quality requirements divided into six quality dimensions. We believe that the framework developed in the present research can support those responsible to develop mobile learning applications.

7. Limitations and Future Work

This research does not end with the present paper, as there are lines of development that can be followed in future investigation works. As the framework was developed in an academic context and under time restrictions, we intend to continue the study and carry out an in-depth validation of the framework. We want to use the developed framework in an extended case study. It is necessary to investigate the components of the proposed framework in different mobile learning applications empirically. In addition, it will be important to test and validate its applicability in a learning context.

Author Contributions

Conceptualization, M.A.A., F.H., A.L. and A.A.-K.; methodology, T.A.; software, validation, O.A. and R.S. Formal analysis, M.A.A.; investigation, M.A.A. and F.H. All authors have read and agreed to the published version of the manuscript.

Funding

This work was funded by King Faisal University and Nourah bint Abdulrahman University.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy.

Acknowledgments

This work was supported through the Annual Funding track by the Deanship of Scientific Research, Vice Presidency for Graduate Studies and Scientific Research, King Faisal University, Saudi Arabia [Project No. GRANT47] and Princess Nourah bint Abdulrahman University Researchers Supporting Project number (PNURSP2022R236), Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Lutfi, A.; Alsyouf, A.; Almaiah, M.A.; Alrawad, M.; Abdo, A.A.; Al-Khasawneh, A.L.; Ibrahim, N.; Saad, M. Factors Influencing the Adoption of Big Data Analytics in the Digital Transformation Era: Case Study of Jordanian SMEs. Sustainability 2022, 14, 1802. [Google Scholar] [CrossRef]
  2. Park, E.; Kim, K.J. An integrated adoption model of mobile cloud services: Exploration of key determinants and extension of technology acceptance model. Telemat. Inform. 2014, 31, 376–385. [Google Scholar] [CrossRef]
  3. Al Amri, M.; Almaiah, M.A. Sustainability Model for Predicting Smart Education Technology Adoption Based on Student Perspectives. Int. J. Adv. Soft Comput. Its Appl. 2021, 13, 60–77. [Google Scholar]
  4. Christensen, R.; Knezek, G. Validating a Mobile Learning Readiness Survey: Assessing Teachers’ Dispositions toward Adoption. J. Digit. Learn. Teach. Educ. 2017, 33, 148–159. [Google Scholar] [CrossRef]
  5. Pawar, S.H. A Study on Big Data Security and Data Storage Infrastructure. Int. J. 2016, 6, 539–542. [Google Scholar]
  6. Sarrab, M.; Elbasir, M.; Alnaeli, S. Towards a quality model of technical aspects for mobile learning services: An empirical investigation. Comput. Hum. Behav. 2016, 55, 100–112. [Google Scholar] [CrossRef]
  7. Cheon, J.; Lee, S.; Crooks, S.M.; Song, J. An investigation of mobile learning readiness in higher education based on the theory of planned behavior. Comput. Educ. 2012, 59, 1054–1064. [Google Scholar] [CrossRef]
  8. Al-Debei, M.M. The quality and acceptance of websites: An empirical investigation in the context of higher education. Int. J. Bus. Inf. Syst. 2014, 15, 170–188. [Google Scholar]
  9. Djouab, R.; Bari, M. An ISO 9126 Based Quality Model for the e- Learning Systems. Int. J. Inf. Educ. Technol. 2016, 6, 370. [Google Scholar] [CrossRef] [Green Version]
  10. Koole, M.L. The Framework for the Rational Analysis of Mobile Education (FRAME) Model: An Evaluation of Mobile Devices for Distance Education. Ph.D. Thesis, Athabasca University, Athabasca, AB, Canada, 2006. [Google Scholar]
  11. Ng, W.; Nicholas, H. A framework for sustainable mobile learning in schools. Br. J. Educ. Technol. 2013, 44, 695–715. [Google Scholar] [CrossRef]
  12. Abu-Al-Aish, A.; Love, S. Factors influencing students’ acceptance of m-learning: An investigation in higher education. Int. Rev. Res. Open Distrib. Learn. 2013, 14, 14. [Google Scholar] [CrossRef]
  13. ELdesouky, A.I.; Arafat, H.; Ramzey, H. Toward complex academic Web- Sites Quality evaluation method (QEM) framework: Quality requirements phase definition and specification. Computer and Systems Engineering Department. Comput. Educ. 2008, 59, 1054–1064. [Google Scholar]
  14. McCall, J.A.; Richards, P.K.; Walters, G.F. Factors in Software Quality. In Preliminary Handbook on Software Quality for an Acquisiton Manager; General Electric: Sunnyvale, CA, USA, 1977; Volume III. [Google Scholar]
  15. Boehm, B.W.; Brown, J.R.; Kaspar, H. Characteristics of Software Quality; North-Holland Publishing Company: Amsterdam, The Netherlands, 1978. [Google Scholar]
  16. ISO 9126; Software Engineering e Product Quality e Part 1: Quality Model. International Standards Organization: Geneva, Switzerland, 2001.
  17. Delone, W.H.; McLean, E.R. The DeLone and McLean model of information systems success: A ten-year update. J. Manag. Inf. Syst. 2003, 19, 9–30. [Google Scholar]
  18. Parasuraman, A.; Zeithaml, V.A.; Malhotra, A. ES-QUAL: A multiple-item scale for assessing electronic service quality. J. Serv. Res. 2005, 7, 213–233. [Google Scholar] [CrossRef]
  19. Loiacono, E.T.; Watson, R.T.; Goodhue, D.L. WebQual: A measure of website quality. Mark. Theory Appl. 2002, 13, 432–438. [Google Scholar]
  20. Almaiah, M.; Al-Khasawneh, A.; Althunibat, A.; Khawatreh, S. Mobile government adoption model based on combining GAM and UTAUT to explain factors according to adoption of mobile government services. Int. J. Interact. Mob. (iJIM) 2020, 14, 199–225. [Google Scholar] [CrossRef] [Green Version]
  21. Almaiah, M.A.; Nasereddin, Y. Factors influencing the adoption of e-government services among Jordanian citizens. Electronic Government. Int. J. 2020, 16, 236–259. [Google Scholar]
  22. Almaiah, M.A.; Al-Khasawneh, A.; Althunibat, A.; Almomani, O. Exploring the main determinants of mobile learning application usage during Covid-19 pandemic in Jordanian universities. In Emerging Technologies during the Era of COVID-19 Pandemic; Springer: Cham, Switzerland, 2021; pp. 275–290. [Google Scholar]
  23. Almaiah, M.A.; Al-Khasawneh, A. Investigating the main determinants of mobile cloud computing adoption in university campus. Educ. Inf. Technol. 2020, 25, 3087–3107. [Google Scholar] [CrossRef]
  24. Barnes, S.J.; Vidgen, R.T. An integrative approach to the assessment of e- commerce quality. J. Electron. Commerce Res. 2002, 3, 114–127. [Google Scholar]
  25. Hsu, C.C.; Sandford, B.A. The Delphi technique: Making sense of consensus. Pract. Assess. Res. Eval. 2007, 12, 1–8. [Google Scholar]
  26. Shaikh, Z.A.; Khoja, S.A. Personal learning environments and university teacher roles explored using Delphi. Australas. J. Educ. Technol. 2014, 30. [Google Scholar] [CrossRef] [Green Version]
  27. Okoli, C.; Pawlowski, S.D. The Delphi method as a research tool: An example, design considerations and applications. Inf. Manag. 2004, 42, 15–29. [Google Scholar] [CrossRef] [Green Version]
  28. Hasson, F.; Keeney, S.; McKenna, H. Research guidelines for the Delphi survey technique. J. Adv. Nurs. 2000, 32, 1008–1015. [Google Scholar] [PubMed] [Green Version]
  29. Heiko, A. Consensus measurement in Delphi studies: Review and implications for future quality assurance. Technol. Forecast. Soc. Change 2012, 79, 1525–1536. [Google Scholar]
  30. Al-Hunaiyyan, A.; Alhajri, R.A.; Al-Sharhan, S. Perceptions and challenges of mobile learning in Kuwait. J. King Saud Univ.-Comput. Inf. Sci. 2016, 30, 279–289. [Google Scholar] [CrossRef]
  31. Udell, C. Learning everywhere: How mobile content strategies are transforming training. In Float Mobile Learning, 1st ed.; RockBench Publishing Corp.: Nashville, TN, USA, 2012. [Google Scholar]
  32. Elias, T. Universal instructional design principles for mobile learning. Int. Rev. Res. Open Distrib. Learn. 2011, 12, 143–156. [Google Scholar] [CrossRef]
  33. Kukulska-Hulme, A.; Sharples, M.; Milrad, M.; Arnedillo-Sánchez, I.; Vavoula, G. Innovation in mobile learning: A European perspective. Int. J. Mob. Blended Learn. 2009, 1, 13–35. [Google Scholar] [CrossRef]
  34. Joseph, F.H.; Black, W.C.; Babin, B.J.; Anderson, R.E.; Tatham, R.L. Multivariate Data Analysis: A Global Perspective; Pearson Education: New York, NY, USA, 2010. [Google Scholar]
  35. Ab Hamid, M.R.; Sami, W.; Sidek, M.M. Discriminant validity assessment: Use of Fornell & Larcker criterion versus HTMT criterion. In Journal of Physics: Conference Series. IOP Publ. 2017, 890, 012163. [Google Scholar]
  36. Almaiah, M.A.; Alyoussef, I.Y. Analysis of the effect of course design, course content support, course assessment and instructor characteristics on the actual use of E-learning system. IEEE Access 2019, 7, 171907–171922. [Google Scholar] [CrossRef]
  37. Almaiah, M.A.; Alamri, M.M.; Al-Rahmi, W.M. Analysis the effect of different factors on the development of Mobile learning applications at different stages of usage. IEEE Access 2019, 8, 16139–16154. [Google Scholar] [CrossRef]
  38. Almaiah, M.A.; Al Mulhem, A. Analysis of the essential factors affecting of intention to use of mobile learning applications: A comparison between universities adopters and non-adopters. Educ. Inf. Technol. 2019, 24, 1433–1468. [Google Scholar] [CrossRef]
  39. Almaiah, M.A.; Al-Zahrani, A.; Almomani, O.; Alhwaitat, A.K. Classification of cyber security threats on mobile devices and applications. In Artificial Intelligence and Blockchain for Future Cybersecurity Applications; Springer: Cham, Switzerland, 2021; pp. 107–123. [Google Scholar]
  40. Lutfi, A.A.; Idris, K.M.; Mohamad, R. The influence of technological, organizational and environmental factors on accounting information system usage among Jordanian small and medium-sized enterprises. Int. J. Econ. Financ. Issues 2016, 6, 240–248. [Google Scholar]
  41. Almaiah, M.A.; Al-lozi, E.M.; Al-Khasawneh, A.; Shishakly, R.; Nachouki, M. Factors Affecting Students’ Acceptance of Mobile Learning Application in Higher Education during COVID-19 Using ANN-SEM Modelling Technique. Electronics 2021, 10, 3121. [Google Scholar] [CrossRef]
  42. Al Mulhem, A. Investigating the effects of quality factors and organizational factors on university students’ satisfaction of e-learning system quality. Cogent Educ. 2020, 7, 1787004. [Google Scholar] [CrossRef]
  43. Almaiah, M.A.; Alamri, M.M. Proposing a new technical quality requirements for mobile learning applications. J. Theor. Appl. Inf. Technol. 2018, 96, 6955–6968. [Google Scholar]
  44. Almaiah, M.A.; Al Mulhem, A. Thematic analysis for classifying the main challenges and factors influencing the successful implementation of e-learning system using NVivo. Int. J. Adv. Trends Comput. Sci. Eng. 2020, 9, 142–152. [Google Scholar] [CrossRef]
  45. Tatnall, A. Editorial for EAIT issue 2, 2020. Educ. Inf. Technol. 2020, 25, 647–657. [Google Scholar] [CrossRef] [Green Version]
  46. Tatnall, A. Editorial for EAIT issue 2, 2019. Educ. Inf. Technol. 2019, 24, 953–962. [Google Scholar] [CrossRef] [Green Version]
Electronics 11 00788 g001 550
Figure 1. Boehm Quality Model.
Figure 1. Boehm Quality Model.
Electronics 11 00788 g001
Electronics 11 00788 g002 550
Figure 2. ISO 9226 Quality Model.
Figure 2. ISO 9226 Quality Model.
Electronics 11 00788 g002
Electronics 11 00788 g003 550
Figure 3. Results of First Round of Delphi Analysis.
Figure 3. Results of First Round of Delphi Analysis.
Electronics 11 00788 g003
Electronics 11 00788 g004 550
Figure 4. Results of Second Round of Delphi Analysis.
Figure 4. Results of Second Round of Delphi Analysis.
Electronics 11 00788 g004
Electronics 11 00788 g005 550
Figure 5. Results of Third Round of Delphi Analysis.
Figure 5. Results of Third Round of Delphi Analysis.
Electronics 11 00788 g005
Electronics 11 00788 g006 550
Figure 6. Quality Requirements Framework for the Development of Mobile Learning Applications.
Figure 6. Quality Requirements Framework for the Development of Mobile Learning Applications.
Electronics 11 00788 g006
Table
Table 2. Results of convergent validity analysis.
Table 2. Results of convergent validity analysis.
Quality DimensionsAverage Variance Extracted (AVE > 0.5)
Interactivity (IN)0.752
Functionality (FN)0.779
Interface Design (ID)0.829
Accessibility (AC)0.801
Learning Content Quality (LCQ)0.750
Content Design Quality (CDQ)0.882
Table
Table 3. Results of Discriminant validity analysis.
Table 3. Results of Discriminant validity analysis.
INFNIDACLCQCDQ
IN0.936
FN0.7970.958
ID0.6300.7580.964
AC0.6460.6840.5450.978
LCQ0.7590.7690.5630.6890.963
CDQ0.7690.7920.6430.7070.7900.943
Table
Table 4. Preliminary List of Technical Quality Dimensions and Requirements.
Table 4. Preliminary List of Technical Quality Dimensions and Requirements.
Quality DimensionsTechnical Requirements Items
Interactivity (IN)1. The application enables learners to interact with instructors via online messages.
2. The application enables learners to exchange and share the learning content.
3. The application enables learners to discuss with learners and faculty by using
discussion board.
Functionality (FN)4. The learners can easily navigate between tasks.
5. The application gives learners alerts for new notifications.
6. Access to the application for both students and instructors.
7. The application gives learners sufficient features.
8. The application offers an interface with a good size and resolution.
Interface Design (ID)9. The application provides a simple and flexible user-interface with a good icons design.
10. The learners can easily identify the particular functions of the application.
11. The application offers good organization of course content and activities.
Accessibility (AC)12. Instructors can create courses and learning content items.
13. Instructors and learners can access the documents of learning content in
multiple formats.
14. Upload and download attachments.
15. Learners can submit assignments and home works.
Learning Content Quality (LCQ)16. The learners can find the complete learning content when using the application.
17. The learners can find the various activities of learning content when using the
application.
18. The learners can find the detailed contact information when using the
application.
Content Design Quality (CDQ)19. The application provides learners different formats of learning content such as text, audio and video.
20. The application provides learners up-to-date content.
21. The application provides learners accurate content.
Table
Table 5. Results of the Second Round.
Table 5. Results of the Second Round.
Quality DimensionsTechnical RequirementsIQR
Score		Degree of Consensus *
Interactivity1. The application enables learners to interact with
instructors via online messages.		0.6Moderate
2. The application enables learners to exchange and share the learning content.0.3High
3. The application enables learners to discuss with
learners and faculty by using discussion board.		0.8Moderate
Functionality4. The learners can easily navigate between tasks.0.2High
5. The application gives learners alerts for new
notifications.		0.3High
6. Access to the application for both students and instructors.0.1High
7. The application gives learners sufficient features.0.4High
8. The application offers an interface with a good size
and resolution.		1.3Low
Interface Design9. The application provides a simple and flexible user-
interface with a good icons design.		0.3High
10. The learners can easily identify the particular functions of the application.0.3High
11. The application offers good organization of course
content and activities.		0.5High
Accessibility12. Instructors can create courses and learning content
items.		0.4High
13. Instructors and learners can access the documents of
learning content in multiple formats.		0.7Moderate
14. Upload and download attachments.0.3High
15. Learners can submit assignments and home works.0.9Moderate
Learning
Content Quality		16. The learners can find the complete learning content
when using the application.		0.3High
17. The learners can find the various activities of learning content when using the application.0.3High
18. The learners can find the detailed contact
information when using the application.		1.2Low
Content Design
Quality		19. The application provides learners different formats
of learning content such as text, audio and video.		0.6Moderate
20. The application provides learners up-to-date content.0.2High
21. The application provides learners accurate content.0.4High
(* IQR < 0.5 = high consensus, 0.5 < IQR ≤ 1 = moderate consensus, IQR > 1 = low consensus).
Table
Table 6. Results of the Third Round (Final List of Technical Quality Dimensions and Requirements).
Table 6. Results of the Third Round (Final List of Technical Quality Dimensions and Requirements).
Quality DimensionsTechnical RequirementsIQR
Score		Degree of Consensus *
Interactivity1. The application enables learners to interact with
instructors via online messages.		0.3High
2. The application enables learners to exchange and share the learning content.0.3High
3. The application enables learners to discuss with
learners and faculty by using discussion board.		0.8Moderate
Functionality4. The learners can easily navigate between tasks.0.2High
5. The application gives learners alerts for new
notifications.		0.3High
6. Access to the application for both students and instructors.0.1High
7. The application gives learners sufficient features.0.4High
Interface Design8. The application provides a simple and flexible user-
interface with a good icons design.		0.3High
9. The learners can easily identify the particular functions of the application.0.3High
10. The application offers good organization of course
content and activities.		0.5High
Accessibility11. Instructors can create courses and learning content
items.		0.4High
12. Instructors and learners can access the documents of learning content in multiple formats.0.7Moderate
13. Upload and download attachments.0.3High
14. Learners can submit assignments and home works.0.9Moderate
Learning
Content Quality		15. The learners can find the complete learning content
when using the application.		0.3High
16. The learners can find the various activities of
learning content when using the application.		0.3High
Content Design Quality17. The application provides learners different formats of learning content such as text, audio and video.0.4High
18. The application provides learners up-to-date content.0.2High
19. The application provides learners accurate content.0.4High
(* IQR < 0.5 = high consensus, 0.5 < IQR ≤ 1 = moderate consensus, IQR > 1 = low consensus).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Almaiah, M.A.; Hajjej, F.; Lutfi, A.; Al-Khasawneh, A.; Alkhdour, T.; Almomani, O.; Shehab, R. A Conceptual Framework for Determining Quality Requirements for Mobile Learning Applications Using Delphi Method. Electronics 2022, 11, 788. https://doi.org/10.3390/electronics11050788

AMA Style

Almaiah MA, Hajjej F, Lutfi A, Al-Khasawneh A, Alkhdour T, Almomani O, Shehab R. A Conceptual Framework for Determining Quality Requirements for Mobile Learning Applications Using Delphi Method. Electronics. 2022; 11(5):788. https://doi.org/10.3390/electronics11050788

Chicago/Turabian Style

Almaiah, Mohammed Amin, Fahima Hajjej, Abdalwali Lutfi, Ahmad Al-Khasawneh, Tayseer Alkhdour, Omar Almomani, and Rami Shehab. 2022. "A Conceptual Framework for Determining Quality Requirements for Mobile Learning Applications Using Delphi Method" Electronics 11, no. 5: 788. https://doi.org/10.3390/electronics11050788

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop