Continuity of Use of Food Delivery Apps: An Integrated Approach to the Health Belief Model and the Technology Readiness and Acceptance Model

Abstract

The pandemic forced both organizations and consumers to make many adjustments to their daily lives. However, due the technological advances that have been seen in recent years, some tools have become much more widely used. Among them are the food delivery applications (FDAs) that experienced an exponential growth during the pandemic. This study proposes an integrated model based on the health belief model and the technology readiness and acceptance model to better understand the determinants of users’ continuance intention to use FDAs. Empirical data collected from 288 Portuguese users of FDAs during the pandemic was analyzed using partial least squares structural equation modeling. The results show that both the perceived susceptibility to and severity of COVID-19 infection positively influenced the perceived usefulness of food delivery applications. Technology readiness is also a predictor of perceived usefulness. Both self-efficacy and technology readiness predict users’ perceived ease of use. Users’ continuance intention to use food delivery applications is directly influenced by perceived usefulness and ease of use and indirectly by self-efficacy, technology readiness, perceived severity, and perceived susceptibility.

1. Introduction

The recent pandemic (COVID-19) erupted as a severe infectious disease in late 2019, progressively expanding to rapidly assume a worldwide expansion [1]. Several innovative measures have been presented and proposed to mitigate the situation, such as the use of a protective mask, social distancing and self-isolation, among others, all of them strongly recommended by the World Health Organization [2] and aimed at reducing the risk of disease transmission [1,3]. Given this situation, fewer consumers intend to use many services, such as the traditional restaurant industry, which suffered and suffers dramatically during this pandemic [4].

The negative influence of COVID-19 on supply and demand in the restaurant industry changed people’s consumption habits and accelerated the transformation of restaurant companies from traditional service to online services, to seek to survive the pandemic situation [4,5]. It is in this process that technology, based on a well-known growth of wireless communication technologies and high internet penetration rates, is seen by food service businesses as an important resource for innovation and competitiveness [6].

The rise of digital technologies has led to a reshaping of markets, and the convenience of being able to order food more easily, with vast options to choose from, has enabled consumers to shift to on-demand shopping through websites or apps [7]. In 2019, online food delivery services reached a value of USD 107.4 billion worldwide and are expected to be worth USD 182.3 billion by 2024 [8]. Food delivery services through online apps have become a global trend [9]. This type of application is among the fastest growing sectors of mobile applications [8].

During the COVID-19 pandemic, the use of food delivery applications (FDAs) has not only met the requirements of businesses but also the demands of customers for convenient food supplies and personal safety concerns since these applications allow customers to effectively and easily order and access their food from several restaurants at convenient times and locations [10]. Consequently, the factors that have motivated users to use these same applications continuously during this pandemic situation are essential to understanding online food delivery purchasing behavior and decision-making processes regarding FDA services.

Several theoretical perspectives have been applied to understand the usage behavior of a new technology and research focused on technology acceptance has been reported in the past two decades [11]. Among these, TAM (technology acceptance model) suggested by Davis [12], and TR (technology readiness) suggested by Parasuraman [13] have been popular models, used to study the factors that contribute to the acceptance of a new technology [14]. Perceived ease of use and perceived usefulness are considered the most important constructs of TAM [15] since users’ acceptance or rejection of a technology is mainly influenced by them [12]. However, these two constructs are both affected by external variables. Therefore, to better explain users’ technology adoption and continued usage of a technology (FDA in our study) it is important to understand the antecedents of perceived usefulness and perceived ease of use. TR has been recognized as an important antecedent of TAM constructs (e.g., [16,17]). TRAM (technology readiness and acceptance model), suggested by Lin et al. [17], is an extended model that combines TAM and TR. Despite the recognized importance of TR as an antecedent of TAM, the application of TRAM to explain the adoption and post-adoption of technologies in the mobile applications arena has been scarce. Some exceptions are the study of Aboelmaged et al. [18] in the context of mobile apps’ use for wellness and fitness applications, Ferreira et al. [19] in the context of mobile self-scanning applications, Jin (2020) in the context of brand applications, and Chiu and Cho [20] in the context of health and fitness applications. To the best of our knowledge, no study has applied TRAM in the context of FDAs.

Concurrently, active customer participation is an essential attribute of service in an e-service context and a crucial element for open innovation [21]. Thus, the implementation of TAMs in service contexts cannot be dissociated from high customer involvement to explain consumer adoption of technology [22]. It is therefore important to identify and qualify the psychological processes of perceived value of a technology and structure a model that incorporates individual difference variables such as technological readiness, self-efficacy, and perceived threat.

The health belief model (HBM) is used to directly explain perceived usefulness and indirectly the continued use of apps [23] from the individual perspective. HBM is used to predict health behavior more generally [24]. The basic assumption of HBM is that individuals will have a preventive attitude towards their health if they feel vulnerable to illness [25]. Wahyuni and Nurbojatmiko [26] in their study show that individuals’ concerns about their own health also influence their intentions to use e-services. Thus, it is appropriate to study the impact of individuals’ concerns about COVID-19 on their intention to continue using food delivery apps. Therefore, the present study aims to combine the TRAM with the HBM [27]. In other words, this study combines a technological perspective with the individual perspective [25]. The main objective of this study is to explain the intention to continue using food delivery apps.

The proposed conceptual model also analyzes the effect of the HBM on the perceived usefulness of these applications and the effect of technological readiness on the perceived usefulness, perceived ease of use, and continuity intention. Finally, it also examines user satisfaction. This study contributes to the literature as it examines the factors affecting users’ intention to continue using food delivery applications in the context of the COVID-19 pandemic. Furthermore, the study combines the TRAM model with the HBM model to explain continuance intent. To the best of our knowledge, this is the first study to combine these models to explain the intention to continue using food delivery apps.

2. Literature Review

2.1. Food Delivery Apps (FDAs)

Among the most popular mobile applications that have been recently developed by service organizations/companies are mobile food ordering applications [10]. These can be defined as mobile applications that smartphone users download and use as an innovative and convenient channel to access restaurants, view food menus, place orders, and make payments without any physical interaction with restaurant staff [5,28]. Technology has helped and driven food service businesses to keep up with the changes in the industry [6]. Smartphones allow for real-time connection/connectivity with mobile applications, and have greatly increased the popularity of food delivery applications, which has also led to much greater competition in these markets [29]. Mobile applications are seen as an additional means by companies to attract new customers and to influence the existing ones to continue and increase their loyalty [28]. The increasing use of smartphones has also led to many changes in people’s dining cultures and food delivery apps are among the most innovative changes in the contemporary restaurant market [30]. During the pandemic, many traditional food delivery services switched platforms and new companies entered the business and began using FDAs to maintain themselves or utilize the opportunity to transition to the digital platform [31].

2.2. Health Belief Model (HBM)

The HBM was initially developed by Hochbaum [32] to help predict individuals’ behavioral reactions to disease. This model is one of the most notable public health frameworks for understanding why individuals may or may not act upon a threat to either their personal or community health [33]. Like many public health behavior models, this model conceptualizes the determinants of behavior [34]. According to the HBM, the dimensions of perceived susceptibility, perceived severity, perceived benefits, perceived barriers, action cues, and self-efficacy can be used to explain whether a person takes action to prevent, track, or improve their health behaviors [35,36]. Beliefs are influenced by each person’s background and comprise their impression of perceived threat, perceived benefits and barriers to taking action, and their perceived ability to take action (i.e., perceived self-efficacy) [37]. Additionally, according to the HBM, the perception of the threat of disease is measured by the perception of susceptibility and severity; the perception of benefits and the perception of barriers, together with the perception of self-efficacy, promote the development of health behaviors among the population affected by a given disease [38]. The perception of susceptibility refers to the beliefs of being vulnerable to the disease, while the perception of severity refers to beliefs concerning the negative effects of disease contraction, i.e., the severity of the risk [39]. The perception of benefits refers to the existence of a way to reduce the incidence or severity of the disease, while the perception of barriers refers to the higher costs versus the benefits of the action [40].

The two dimensions of perceived threat, perceived susceptibility and perceived severity [41], have been widely adopted to explain different behaviors such as technology adoption (e.g., [27,42,43]), fear of travel (e.g., [44]), organic food choices [45], among others. Recent studies also adopted these dimensions to explain customer intention to use online food delivery services during COVID-19 [46,47].

2.2.1. Perceived Threat

Perceived threat has been recognized as a core component to understand a variety of preventive health behaviors, such as those related to COVID-19 [47]. The two dimensions of perceived threat (perceived severity and perceived susceptibility) are also among the various measurements that have been widely used to determine people’s perceptions of a disease [46]. The perception of susceptibility refers to the belief of being vulnerable to the disease, while the perception of severity refers to belief concerning the negative effects of disease contraction, i.e., the severity of the risk [39]. According to the HBM, an individual is considered more likely to take appropriate action if the perceived threat of disease is high. In turn, the perceived threat will be higher if the perceived severity is higher—that is, the disease is considered to be a serious problem.

2.2.2. Perceived Self-Efficacy

Perceived self-efficacy can be defined as the belief that one has the ability to overcome a given challenge [34].

In the health management literature, self-efficacy can be seen as a significant determinant of preventive health behaviors [48]. Venkatesh et al. [11] explain self-efficacy as the ability of individuals to perform a given task. In the context of technology adoption, self-efficacy thus refers to users’ confidence in their ability to use a technology and serves as a determinant of perceived ease of use [11]. Perceived self-efficacy is considered to be an important precursor to the adoption of new technologies [49], being especially relevant in the use of mobile devices and, although they offer advantages, they also increase challenges, compared to computers. Contemporary studies have shown that self-efficacy affects behavioral intention to adopt apps, e-government system, and e-portfolios, among other things, both directly and indirectly ([15,50]). In the present study self-efficacy was analyzed in relation to technology adoption, and not integrated into the HBM.

2.3. Models Related to Technology Acceptance

2.3.1. TAM

The literature has used several theoretical frameworks to explain the adoption and use of technologies. The technology acceptance model (TAM), developed by Davis [12], is now one of the most widely used models to explain the acceptance of new technologies [51], and is recognized as a valid and robust model [52]. TAM suggests that when a user encounters a new technology, there are several factors that affect how they accept and use it, and it has been used in both consumer and organizational contexts to explain the factors that affect the acceptance of a particular technology [53]. TAM has also been widely applied to examine individual technology adoption behaviors across different populations and types of innovative technologies [54], such as e-portfolios [10], and m-commerce [55], among others. This is also useful in explaining what influences an individual’s intention to use mobile technologies [56] and smartphones [22]. Fishbein and Ajzen [57] suggest that behavior can be predicted based on the intention to perform it and that this intention is driven, in part, by attitudes toward it. Some studies applied TAM to examine individuals’ usage and behavior in the context of applications (e.g., [58]). These studies demonstrated that TAM was an appropriate theoretical framework to explain individuals’ intentions to use apps.

Among the wide adoption in all fields of technology acceptance studies, TAM [12] has also been used to predict consumers’ acceptance of technology in relation to health ([59]). According to TAM, perceived usefulness and perceived ease of use are the two main determinants of technology use [60]. While TAM has proven useful [61], additional constructs believed to have enhanced TAM have resulted in a variety of extended models, such as TAM2 and TAM3 [11,62]. It is also important to note that while TAM is instrumental in the initial acceptance of the new technology, more and more researchers have emphasized that the success of the new technology should not be limited to that same initial acceptance, but supported by continued use [63]. For example, Bhattacherjee [64] suggests continuance intention as a variable of technology acceptance, and thus, in order to include continuance intention, research on technology acceptance has been expanded.

2.3.2. TRAM

Several studies have applied the technology acceptance model (TAM) as a theoretical basis to analyze individuals’ intentions to use applications (e.g., [30,58]). However, some have argued that this model may not be sufficient to explain individuals’ technology adoption behavior, as the main variables of TAM measure utilitarian aspects of technology use, i.e., ease of use and usefulness (e.g., [17]). Thus, several authors suggest an integration of additional factors in order to extend TAM to better explain individuals’ psychological processes in their behavior regarding technology adoption (e.g., [17,19]).

TRAM combines the general personality constructs of TR with the specific model of TAM, thus determining how individuals’ technology-related beliefs may affect their perceptions of interacting with, experiencing, and using new technologies [16]. The integration of TR and TAM can provide a deeper understanding of the psychological process involved in application adoption behavior [20]. Since Lin et al. [17] introduced TRAM, several researchers have conducted studies to examine users’ technology adoption behavior in a wide range of settings, such as m-services [65] and mobile self-scanning applications [19].

2.3.3. Technology Readiness (TR)

Technology readiness (TR) was defined by Parasuraman ([13], p. 308) as being “the propensity of people to embrace and use new technologies to achieve goals in home life and work”. The same author argues that technology readiness is divided into four components. The first two are related to positive feelings, i.e., optimism (belief that technology will bring efficiency, control, benefits, and flexibility) and innovation (being a pioneer in testing innovative technology-based services or products). The other two are related to negative feelings, i.e., discomfort (reflects the individual’s perception of lack of control and confidence in using the technology) and insecurity (fear that the technology-based service, product or process may not work in an accurate and reliable way).

The four dimensions of TR are independent of each other and are associated with an individual’s behavioral disposition and general thoughts and feelings toward technology [66]. TR can be considered as an overall state of mind arising from mental and inhibiting factors that jointly determine a person’s tendency to use new technologies [67]. If an individual has a higher level of TR then their rate of adoption of new technologies is higher. In addition, the individual exhibits more intensive use of technology and greater ease in using it [68].

2.3.4. Continuance Intention

The number of studies on the intention to continue using information systems (IS) has grown rapidly in recent years and now covers several contexts such as the intention to continue in m-services, in applications, and in m-commerce, among others [69]. Although most of the previous research on these systems is strongly focused on the initial acceptance, it is now sought to investigate the direct effects on the continuity intention of mobile applications, since it is considered essential for the long-term viability of an IS [64].

Kim and Kang [70] argue that ongoing IS usage may specifically reflect users’ behavioral patterns toward a target IS/m-service. Bhattacherjee et al. [64] also indicate that while the initial adoption of an IS/IT is an important advance for IS/IT success, users’ continued use, rather than initial acceptance, is the determining factor of the long-term sustainability and ultimate success of IS/IT. It becomes evident that the intention of continued use is strongly associated with user behaviors (i.e., a behavior that an individual can decide whether to perform or not) [71]. Bhattacherjee [72] was one of the first researchers to distinguish between technology acceptance and continuance of use behavior. Bhattacherjee [72] further defines continuance intention to use as an individual’s intention to continue to use an information system. In their literature review, Nabavi et al. [73] also described it as a user’s decision to continue using a specific IT that an individual has already used.

Designing strategies to continuously attract the user is one of the most critical phenomena in the IT world [74]. Similarly, other authors have postulated that continuous usage is more important than initial usage, as it is argued that the cost to develop a new customer can be up to five times more than the cost to maintain an existing customer (e.g., [72]).

2.4. Proposed Model and Development of Hypotheses

Due to COVID-19, people believe that their health is at risk and thus may formulate a higher perception of usefulness regarding applications, to prevent and thus reduce the likelihood of COVID-19 infection [27]. The adoption of technology was considered as a behavior to promote, protect, or maintain one’s own health [75]. Therefore, this technology adoption can be explained by the HBM, since it suggests that people’s beliefs about health problems, perceived benefits, and perceived barriers to action, as well as self-efficacy, explain the involvement or lack thereof in health promotion behavior by individuals [35]. The perception of health threat refers to people’s awareness and care, as well as the potential consequences. Previous studies developed in the health care context found contradictory results regarding the influence of perceived threat, which involves perceived susceptibility and severity, on perceived usefulness. For example, Dou et al. [76] found a strong relationship between perceived threat and perceived usefulness while Kim and Park [60] found lack of a significant relationship. However, more recent studies developed in the context of COVID-19 found a positive significant effect of perceived susceptibility and severity on perceived usefulness of mobile-based payments [27] and e-wallet systems [42]. Thus, the following hypotheses are proposed:

Hypothesis 1 (H1).

The susceptibility to COVID-19 positively affects perceived usefulness of the FDA.

Hypothesis 2 (H2).

The severity of COVID-19 positively affects perceived usefulness of the FDA.

Technological self-efficacy is the personal belief that a person has the adequate and accurate skills and abilities to succeed when dealing with a technology-related task [77]. Based on Luarn and Lin’s [78] study on mobile services, the current research focuses on whether individuals believe that they have the necessary knowledge, skills or ability to use food delivery applications (FDAs). Thus, perceived self-efficacy is defined as the judgment of one’s ability to use food delivery applications. Self-efficacy has been adapted for the purpose of being incorporated into technology adoption models (e.g., [15,49]). This implies that consumers of mobile services are more likely to pursue activities within their perceived areas of competence, self-efficacy being an important factor in understanding individual responses to new technologies [79]. This variable has figured in studies developed in different contexts such as e-shopping [80], mobile banking [10], use of e-portfolios [15], food delivery services [46], use of electronic wallets [42], and mHealth services [44], among others.

Self-efficacy plays an important role in the context of technology and IS use (Ahmed et al. 2010) and internet self-efficacy (ISE) in the context of internet technology [4]. Self-efficacy affects user behavior towards using a technology, as individuals with high levels of self-efficacy will be confident in their capability to overcome any difficulties when using the technology [15]. Regarding computer usage, “the higher the individual’s computer self-efficacy, the higher his/her use of computers” ([49], p. 196). A sense of self-efficacy may increase the likelihood that users will evaluate the technology as easy to use [76]. Previous studies developed in different contexts such as mobile commerce, mobile banking, e-portfolios, smartphone health apps, among others, associate higher levels of self-efficacy and perceived ease of use (e.g., [42,71,78]).

Regarding the relationship between self-efficacy and perceived usefulness, the literature presents more contradictory results. Although some studies have found a non-significant effect between these two variables (e.g., [60]) or a negative significant effect (e.g., [15]), several studies in fact found a positive significant effect (e.g., [42,60,71,80]). A recent study developed in the context of mobile technologies’ usage, more specifically, the usage of mobile wallets while dining out in a restaurant, also found a strong association between mobile self-efficacy and mobile usefulness and ease of use [81].

Therefore, the following hypotheses are proposed:

Hypothesis 3 (H3).

Self-efficacy positively affects perceived ease of use of the FDA.

Hypothesis 4 (H4).

Self-efficacy positively effects perceived usefulness of the FDA.

There are few studies assessing the link between TR and TAM, compared to the number of studies applying the TAM model. A high TR may result from previous experience with the same technology which, in turn, may increase ease of use and perceived usefulness [82]. It is expected that technology readiness has a direct positive effect on perceived usefulness, since individuals with higher innovativeness and higher optimism towards technological innovations should be more able to see the utility related to their adoption [17]. Previous studies that linked technology readiness to TAM constructs in various technology adoption contexts, for example self-service technologies [83], online stock trading systems [17], mobile self-scanning applications [19], and m-commerce [84], among others, found a positive and significant relationship between it and perceived usefulness and perceived ease of use. Moreover, Jin [85] also confirmed a positive and a negative effect of positive technology readiness and negative technology readiness, respectively, on perceived usefulness and perceived ease of use.

Previous studies have also linked technology readiness to users’ behavioral intentions in various technology adoption contexts such as self-service technologies [86], online stock trading systems [17], and self-checkout services using smartphones [87], among others. Regarding the relationship between these two variables, the literature reports several results. Some studies found a positive direct effect (e.g., [86]), others support indirect effects through other variables such as perceived usefulness and ease of use [17], and others indicated lack of significant relationship (e.g., [87]). Blut and Wang [16] in their meta-analysis about TR constructs and its impact on technology usage found an indirect effect of technology readiness on usage intention via TAM mediators (ease of use and usefulness).

In view of the above, the following set of hypotheses are presented:

Hypothesis 5 (H5).

Technological readiness positively affects perceived ease of use of the FDA.

Hypothesis 6 (H6).

Technological readiness positively affects perceived usefulness of the FDA.

Hypothesis 7 (H7).

Technological readiness positively affects continuance intention to use the FDA.

TAM is a representative model used to explain and predict individuals’ adoption of information technology. Several studies have used this model as well its extensions to explain the process of information technology acceptance, such as studies of e-service, service mobile apps, information technology systems, and internet-based services, among others (e.g., [11,15,19]), further indicating that behavioral intentions to use a given technology are determined, in part, by users’ perceived ease of use (PEOU) and perceived usefulness (PU). According to TAM, PEOU is a determinant of PU [11,12]. When individuals have perceived ease of use of technology, they are more likely to believe that the technology is useful and helpful for a specific purpose. Venkatesh ([11], p. 343) stated that “the easier a technology is to use, the more useful it may be”. Once individuals perceive ease in using a technology and it has perceived usefulness, individuals will adopt and accept it for a specific purpose [20].

The literature further indicates that PEOU and PU appear to be particularly vital measures of users’ intention to use a particular system [12]. A great deal of research on TAM demonstrates that these two factors have a joint impact on the use and acceptance of a wide variety of technologies (e.g., [65,86]). Users will always want to continue using a particular application that can help them improve their productivity [64,72].

Users need to feel that a particular application (e.g., FDAs) is easy enough to use to motivate them to use it [39]. The theory of reasoned action (TRA) [57], a theory that gave rise to the development of TAM by [12], states that perceived usefulness and perceived ease of use can influence user’s attitudes and intention to use. Thus, PEOU and PU are expected to be positively related to the intention to continue using applications. Moreover, a recent study developed in the context of online food delivery services confirms a strong positive effect of both perceived ease of use and usefulness on continuance intention [46].

According to TAM, perceived ease of use is hypothesized to be a determinant of perceived usefulness. Several empirical studies have also supported this relationship for a wide variety of technologies (e.g., [15,39]). A recent study developed by Roh and Park [88] in the context of O2O food delivery services also found a strong effect of perceived ease of use on usefulness. When an individual realizes that few resources are needed to learn a new mobile technology, he/she may perceive the technology as being useful, which leads to its continued use. In view of the above, the following set of hypotheses is presented:

Hypothesis 8 (H8).

Perceived ease of use positively affects perceived usefulness of the FDA.

Hypothesis 9 (H9).

Perceived ease of use affects continuance intention to use the FDA.

Hypothesis 10 (H10).

Perceived usefulness affects continuance intention to use the FDA.

Figure 1 presents the conceptual and hypotheses.

3. Methodology

3.1. Data Collection and Demographics

The target population of this study consisted of Portuguese smartphone users who have used food delivery apps (FDAs), at least once, during the ongoing COVID-19 virus period. Due to confidentiality issues it was not possible to obtain a sampling frame for the target population. Thus, this study used a non-probability convenience sampling (cf. [5,19]). We adopted a web-based survey questionnaire approach to collect the data. The link of the web-based survey, which included a brief explanation of the study purpose, was available on online channels such as social networks (e.g., Facebook, Instagram, WhatsApp) and online groups since offline data collection was not possible during the pandemic. Data collection occurred over a period of two months during the pandemic, from March to April of 2021. A total of 618 responses was obtained, of which 288 were usable.

As suggested by Churchill [89], a literature review was initially conducted to understand how the variables present in the model have been measured in the literature. Then, the first version of the survey questionnaire was revised by three academic experts in information systems to access content validity of the scales. After that, the survey was pre-tested with users of FDAs. Based on the feedback and comments from both users and academic experts, the final survey was developed.

The details of demographic characteristics are presented in

Table 1. Sample characteristics.

Characteristics	Frequency	Percentage
Gender
Male	100	35.8
Female	185	64.2
Age
Less than 24	136	47.2
25–34	100	34.7
35–44	16	5.6
45–54	22	7.6
Over 54	14	4.9
Education
High school or below	15	5.2
Intermediate	7	2.4
Undergraduate and postgraduate	155	53.8
Master or above	111	38.5
Net monthly income
Less than EUR 2.001	179	62.1
EUR 2.001–4.000	31	10.8
More than EUR 4000	5	1.7
Did not answer	73	25.3

. A total of 64.2% respondents were female, 53.8% were graduates or post-graduates and 38.5% had master degrees or above as educational background. Regarding the respondents’ ages, it was found that almost half were aged between 18 and 24 years (47.2%) and that 34.7% of respondents were aged between 25 and 34 years. In relation to household monthly income, 62% of respondents had a monthly income of less than EUR 2000.

Most of the respondents reported Uber Eats as the FDA that they used more often (71.2%), followed by Glovo (17.4%), and Bolt Food (6.9%). Respondents were asked when they started using FDAs. It was found that 67% of respondents started using these types of apps in 2019 or before. We further observed that approximately 29% of the respondents started using these apps in the year 2020 and 3.8% of the respondents started using these types of apps in 2021. Out of the 95 respondents that start using FDAs in 2020 or 2021, 59 respondents stated that they started using these apps as a consequence of the COVID-19 pandemic.

3.2. Measures

In the present study, the scales used to measure each of the variables of the proposed conceptual model were adapted from the literature.

Perceived usefulness was measured with four items adapted from Thong et al. [90] and Hsiao et al. [91]. The four items used to measure perceived ease of use were adapted from Thong et al. [90]. Continuance intention was measured with four items adapted from Wang et al. [28] and Hsiao et al. [91]. Self-efficacy was measured using four items adapted from Luarn and Lin [78] and Abdullah et al. [15]. Perceived susceptibility and perceived severity were measured with three items each adapted from Walrave et al. [43]

A sixteen-item scale was used to measure optimism (4 items), innovativeness (4 items), discomfort (4 items), and insecurity (4 items). All items were adopted from Parasuraman and Colby [67] after obtaining permission from Professor Parasuraman and Rockbridge Associates. Following the suggestions of Parasuraman and Colby [67], technology readiness was computed as the average of the four dimensions. Before computing the average, the discomfort and insecurity dimensions were re-coded by subtracting the value indicated in each item by the respondent from 6. All variables were assessed using Likert scales anchored by one (strongly disagree) and five (strongly agree). Items are shown in Appendix A.

Two control variables were also included in our model to ensure the robustness of findings. Following recent studies developed in the context of mobile applications, we used demographic variables as control variables (e.g., [5,19]). More specifically, the variables gender and age were used.

4. Results

The proposed model was tested using used partial least squares structural equations modeling (PLS-SEM), a variance-based structural equation modeling (SEM) technique [92]. The analysis was conducted using Smart PLS 3.0 software (cf. [93]).

4.1. Measurement Model

Before testing the hypotheses, we analyzed the measurement model to check the reliability and validity of the constructs. The standardized indicator loadings and their t statistic for all constructs are presented in

Table 2. Measurement model evaluation: individual item reliability, construct reliability, and convergent validity.

Construct	Item	Standardized Loading	t-Value	CR	CA	AVE
Perceived susceptibility	PS1	0.876	12.957	0.771	0.865	0.682
	PS2	0.832	10.139
	PS3	0.765	6.840
Perceived severity	PSEV1	0.850	4.005	0.719	0.875	0.778
	PSEV2	0.913	4.601
Self-efficacy
	SE1	0.784	18.945	0.795	0.867	0.619
	SE2	0.828	25.965
	SE3	0.734	16.308
	SE4	0.799	23.225
Perceived usefulness
	PU1	0.930	86.507	0.931	0.951	0.829
	PU2	0.922	73.866
	PU3	0.930	64.534
	PU4	0.858	32.617
Perceived ease of use	PEOU1	0.878	46.413	0.889	0.922	0.747
	PEOU2	0.814	24.135
	PEOU3	0.888	39.517
	PEOU4	0.876	46.269
Continuance intention	CI1	0.845	37.109	0.768	0.851	0.589
	CI2	0.771	20.416
	CI3	0.740	14.693
	CI4	0.705	14.093

. The values show that all loadings are higher than or equal to the threshold of 0.70 and p < 0.001 (Hair et al. [92]), which supports individual indicator reliability. Internal reliability and convergent validity were examined by means of composite reliabilities (CRs) and average variance extracted (AVE). The values of CR and AVE for all constructs (see also Table 2) are greater than the acceptable cutoff of 0.7 and 0.5, respectively [92,94,95]. Likewise, we confirm that our constructs exhibited adequate construct reliability and convergent validity.

To establish discriminant validity, we apply the Fornell−Larcker and heterotrait-monotrait ratio (HTMT) criteria [92,95]. The Fornell−Larcker criterion compares the square root of the AVE of each construct (shown on the diagonal with bold values in

Table 3. Measurement model evaluation: discriminant validity.

Discriminant validity: Fornell−Larcker criterion.
Variables	PS	PSEV	SE	PEOU	PU	CI	TR	Age	Gender
Perceived susceptibility (PS)	0.826
Perceived severity (PSEV)	0.107	0.882
Self-efficacy (SE)	0.018	−0.061	0.787
Perceived ease of use (PEOU)	0.001	−0.095	0.393	0.865
Perceived usefulness (PU	0.196	0.093	0.148	0.026	0.911
Continuance intention (CI)	0.129	0.027	0.335	0.405	0.354	0.767
Technology readiness (TR)	−0.101	−0.54	0.194	0.2	0.171	0.196	NA
Age	0.047	−0.049	0.052	−0.1	−0.073	0.013	0.068	NA
Gender	0.024	0.061	−0.008	−0.032	−0.024	−0.054	0.101	0.038	NA
Discriminant validity: heterotrait−monotrait ratio (HTMT)
	PS	PSEV	SE	PEOU	PU	CI	TR	Age	Gender
Perceived susceptibility (PS)
Perceived severity (PSEV)	0.156
Self-efficacy (SE)	0.057	0.113
Perceived ease of use (PEOU)	0.070	0.107	0.442
Perceived usefulness (PU	0.220	0.113	0.173	0.071
Continuance intention (CI)	0.158	0.084	0.424	0.454	0.408
Technology readiness (TR)	0.114	0.627	0.215	0.203	0.174	0.217
Age	0.054	0.063	0.063	0.112	0.076	0.093	0.068
Gender	0.024	0.081	0.068	0.046	0.029	0.092	0.101	0.038

Note: Bolded numbers are the square roots of AVE. Below the diagonal elements are the correlations between the constructs.

) with the paired inter-correlation between the construct and any other construct (shown on the left of the diagonal). The results presented in Table 3 confirm that this criterion was meet for all constructs. According to the heterotrait−monotrait ratio (HTMT) criterion, discriminant validity is confirmed when the HTMT ratios are below 0.9 [92,96]. As can be seen in Table 3, this criterion was fulfilled since all HTMT ratios are below 0.9. Taken together, these results provide evidence of the discriminant validity of all constructs.

4.2. Structural Model

The structural model was evaluated using the sign, magnitude, and significance of the structural path coefficients, the variance explained (R²), and the predictive relevance (Q²). To assess the statistical significance of the loadings and path coefficients, a boostrapping procedure using 5000 subsamples was employed as suggested by Hair et al. [97]. Prior to assessing the structural model, the model collinearity was assessed. The VIF values ranged from 1.016 to 1.489, which was below the indicative critical value of 5 [92], showing multicollinearity is not an issue in the model.

To evaluate the predictive relevance of the model, a blindfolding procedure with an omission distance of 7 was applied. The Stone–Geisser’s Q² obtained for all endogenous constructs (continuance intention: 0.157; perceived usefulness: 0.100; perceived ease of use: 0.112) was well above zero. Thus, the model’s predictive relevance was confirmed. The coefficient of determination (R²) measures the variance of the endogenous constructs explained by the exogenous constructs. According to Falk and Miller [98], the R² of each endogenous construct in the model should be equal to or greater than 10%. This criterion was accomplished for all endogenous variables in the model (continuance intention: R² = 30%; perceived usefulness: R² = 13%; perceived ease of use: R² = 17%).

5. Discussion: Open Innovation in Food Industry after Using of FDA

Most of the hypothesis were supported by the results. A summary of the hypotheses and their results is presented in

Table 4. Results of the hypothesis testing.

Hypothesis	Path Coefficient	t-Value	Decision
(H1) Perceived susceptibility→perceived usefulness	0.199	3.713 ***	Supported
(H2) Perceived severity→perceived usefulness	0.241	2.928 **	Supported
(H3) Self-efficacy→perceived ease of use	0.368	7.043 ***	Supported
(H4) Self-efficacy→perceived usefulness	0.123	1.731 n.s.	Not supported
(H5) Technology readiness→perceived ease of use	0.129	1.989 *	Supported
(H6) Technology readiness→perceived usefulness	0.310	4.393 ***	Supported
(H7) Technology readiness→continuance intention	0.059	0.8961 n.s.	Not supported
(H8) Perceived ease of use→perceived usefulness	−0.061	0.944 n.s.	Not supported
(H9) Perceived ease of use→continuance intention	0.391	5.478 ***	Supported
(H10) Perceived usefulness→continuance intention	0.338	5.155 ***	Supported
Control variables
Age→continuance intention	0.074	1.441	NA
Gender→continuance intention	−0.042	0.831 n.s.	NA

Note: * p < 0.05, ** p < 0.01, *** p < 0.001; n.s.—not significant; NA—not applicable.

. The importance of these findings is an alert to the importance of developing specific organizational competences to innovate in FDA, a key element to enhance open innovation, as suggested by Rufat-Latre, et al. [99], and represents an important path to enhance competitive advantage [100]. Hypotheses H1 and H2 postulate that perceived usefulness positively relates to perceived susceptibility and perceived severity to COVID-19, respectively. Both hypotheses were supported with (β = 0.199; p < 0.001) and (β = 0.241; p < 0.01). These results are consistent with previous studies also developed during the COVID-19 pandemic in the context of mobile-based payments applications [27], and digital wallets [42]. In other contexts, Wei et al. [101] also concluded that perceived threat of weight loss induces a higher level of usefulness among users of fitness mobile apps. This finding reveals the importance of apps not only in an open innovation context but also for achieving competitive advantage, promoting a more engaged innovation with a wider variety of participants [102,103]. As Jain [104] posits, “mobile app usage will put considerable pressure on mobile access networks, requiring extensive R&D to improve spectral efficiency and expensive investments to increase network capacity” (p. 4).

The results also indicated a strong positive significant effect of self-efficacy on perceived ease of use (β = 0.368; p < 0.001). Thus, H3 was supported. This finding is consistent with previous studies addressing different technologies (e.g., [15,71,76,78]). This means users who are confident in using food delivery apps found their usage easy. Contradicting our expectations, the effect of self-efficacy on perceived usefulness was not significant (β = 0.123; ns). This finding is similar to those of previous studies [56] that also found lack of significant association between self-efficacy and perceived usefulness. However, it contradicts several studies that found a positive strong effect of self-efficacy on perceived usefulness (e.g., [10,42,81]. As such, further development of research in this field is required, especially in studying the predictors of self-efficacy and how they can relate direct and indirectly with perceived usefulness. In this field, cooperation among competing firms can lead to interesting results, because of the complementarity between platforms’ use and functionality, as suggested by Ghazawneh and Henfridsson [105]. For example, in the field of open innovation, the development of delivery apps can represent another form of open innovation for new product development through increased cooperation between competing firms by participating in the development and commercialization of complementary products [106,107].

Hypotheses H5 and H6 postulate that technology readiness positively relates to perceived ease of use and usefulness, respectively. Both hypotheses were supported (β = 0.129; p < 0.05) and (β = 0.310; p < 0.001), respectively. These results are consistent with many previous studies that also found support for these relationships (e.g., [17,19,84]). Hypothesis H7, which postulated a positive effect of TR on continuance intention, was not supported (β = 0.059; not significant). This result is in line with the meta-analysis developed by Blut and Wang [16] in which the authors found lack of significant direct effect of TR motivators (innovativeness, optimism) and inhibitors (insecurity and discomfort) on intention to use a technology.

The proposed link between perceived ease of use and perceived usefulness was not significant (β = −0.061; not significant). Thus, hypothesis H8 was not supported. This is contrary to many previous studies developed in the context of service mobile apps (e.g., [14,39]) as well as other technologies (e.g., [15,17]).

This study also found that perceived ease of use (β = 0.391; p < 0.001), and perceived usefulness (β = 0.382; p < 0.001) are strongly associated with continuance intention. Thus, H9 and H10 were supported. These results are in line with numerous previous studies that also found a strong positive effect of perceived ease of use and usefulness on customers’ usage intention or continuance intention toward a wide variety of technologies (e.g., [15,65,86]). The same effects have been reported in recent studies that analyze the continuance intention to use online food delivery services (e.g., [46,88]). A recent study developed in the context of FDAs also found a strong positive effect of perceived ease of use on intention to use [108]. In the context of a mobile taxi booking application, Weng et al. [109] found a strong relationship between usefulness and continuance intention. In the open innovation field, this is consistent with equity theory that posits that customer satisfaction is highly correlated with the perception of service quality [110]. In this vein, Alzoubi et al. [111] also found that the use of the technologies associated with delivery services increases customer satisfaction and leads to enhanced loyalty levels.

In addition to the hypothesized associations, we also tested indirect effects as these could provide further insights into understanding the continuance usage of FDAs. Although the direct effect of TR on continuance intention was not significant, we found that TR indirectly affects continuance intention through perceived usefulness (β = 0.105; p < 0.001). This result is in line with the conclusions of Blut and Wang [16] who found that TR motivators not directly affect usage behavior, but instead affect it indirectly through other variables such as TAM mediators (ease of use and usefulness). The results also show an indirect effect of HBM constructs, perceived susceptibility (β = 0.067; p < 0.01) and perceived severity (β = 0.081; p < 0.05), on continuance intention through perceived usefulness. A strong indirect effect of self-efficacy on continuance intention through perceived ease of use was also found (β = 0.144; p < 0.001).

Finally, the results showed that both control variables (age and gender) have no significant effect on the continuance intention related to age (β = 0.074; ns) and gender (β = −0.042; ns) (see Table 4). These findings are in line with the study of Kumar and Shah [5] developed in the context of FDAs during the COVID-19 pandemic, as well as other studies developed in the context of mobile applications (e.g., [19]). Although the strong growth of FDA usage was influenced by the pandemic lockdown, the rapid development of this technology and consequent user adoption points to a ‘technology scouting’ approach (instead of technology sourcing), which is more likely to be found in an incremental innovation situation, as found by Parida et al. [112].

6. Conclusions

This study developed a theoretical framework, augmenting the technology acceptance model (TAM) by integrating the health belief model (HBM), and technology readiness model (TRM), to enrich the current understanding of the continued use of FDAs within a broader context of open innovation. According to the study’ findings, seven out of the ten proposed hypotheses were supported.

The study findings show that perceived susceptibility and severity positively influence the perceived usefulness of FDAs. As for perceived self-efficacy, it positively influences perceived ease of use but does not influence perceived usefulness. The results also show that consumers’ technological readiness positively and directly affects both perceived usefulness and perceived ease of use. The direct effect of TR on continuance intention was not confirmed. However, a significant indirect effect through perceived usefulness was found. Both perceived ease of use and perceived usefulness have a strong direct effect on continuance intention. It should be noted that, contrary to previous studies, the effect of perceived ease of use on perceived usefulness was not significant. This fact may be related to the final sample obtained. In other words, most respondents were young people for whom ease of use issues were not so critical.

6.1. Theoretical Contributions

This study offers key contributions for IS continuance usage literature. First, by investigating the factors that contribute to the continuance usage of FDAs during the COVID-19 pandemic, this study contributes to the literature of technology use in emergency situations [5]. Perceived ease of use and usefulness are recognized as the most important constructs of TAM. However, in order to be able to explain users’ technology continuance intention behavior, it is important to understand how perceived usefulness and ease of use are affected by external variables [11,15]. Thus, our study also contributes to the literature by validating the integration of HBM and TAM in the context of FDAs. More specifically, our study highlights the importance of perceived susceptibility and severity as antecedents of perceived usefulness and self-efficacy as antecedent of perceived ease of use. This represents an important link to research in open innovation. For example, Barlatier and Josserand [113] found that social media can enhance these perceptions and create a better connection to key stakeholders for innovation. Finally, in line with previous studies that highlight the importance of the integration of TR with TAM to better understand technology usage [16], and continuance intention to use mobile applications [19], our study also validates this integration by demonstrating that TR is an important antecedent of both perceived usefulness and perceived ease of use.

6.2. Practical Implications

This study has short-term and long-term practical implications. First, the findings indicate that perceived severity and susceptibility to COVID-19 infection influences the perceived usefulness of FDAs and indirectly the continuance intention to use them. This suggest that in the short term, policymakers and marketers should promote the adoption of this type of application as a means of social distancing and consequently prevent the spread of COVID-19. These results also have long-term implications because of future threats of the same kind. Moreover, similarly to Uber Eats and Deliveroo, other FDAs should incentivize customers and drivers adhere to social distancing by launching a “leave at your door service” [46].

The long-term implications also include better understanding of how to ensure the continuance usage of FDAs since the digital transformation that has occurred needs to be sustained and FDAs are the interface between restaurants, suppliers, and consumers. Thus, it is relevant that companies ensure their continued use during and after the COVID-19 pandemic [5].

Based on the empirical results of the study, managers and FDAs designers should also focus on perceived usefulness and perceived ease of use since both are important in promoting users’ continuance intention to use FDAs. Thus, FDA designers should make things easier for users to ensure the continued usage of the apps. For instance, they may simplify registration and data recording procedures, and make use of the information from previous orders to make new orders easier, among other things. In addition, FDA marketers should develop campaigns highlighting the usefulness of FDA services.

The results of the study demonstrate that TR directly influences both perceived ease of use and usefulness, and indirectly affects continuance intention to use a FDA. Managers should take into consideration the fact that the usefulness and ease of use perceived by consumers with low TR when encountering a new FDA will be well below the ones by consumers with high TR. Thus, it would be more profitable for managers to concentrate marketing efforts on consumers with low TR since this kind of consumer is resistant to new technologies. In this way, if a firm can get these consumers to see its innovative FDAs as beneficial and to continue using them, it is expected that consumers with high TR who are more open to new technologies should take the same path [19].

6.3. Limitations

As with any research, this study presents some limitations. First, this study used data collected through a web-based convenience sampling procedure, which can lead to self-selection bias and is prone to obtain samples including users with lower levels of internet phobia. Second, the study focused on users of FDAs in Portugal, and the results cannot be generalized to other countries with different cultures. Therefore, the use of probabilistic sampling and comparations across countries are encouraged.

The variables in the model are all based on consumers’ perceptions. It would be interesting to use objective data. Thirdly, this study raises only a short-term reflection of the users’ perception of the intention to continue using food delivery apps, especially in a particular situation (pandemic context of COVID-19). Having said this, it would be interesting to conduct other studies exploring users’ perceptions in different situations and investigate causality over time, for example, in the case of the COVID-19 pandemic, exploring its effects after it was overcome.

According to Humbani and Wiese [114], the relationship between ease of use and continuance intention warrants further research to determine if indeed the importance of ease of use decreases with experience. Thus, this aspect should also be considered in future studies.