Exploring the Factors in the Discontinuation of a Talent Pool Information System: A Case Study of an EduTech Startup in Indonesia

Abstract

This research was conducted to determine the reasons behind users’ discontinuation of talent pool information system use. A qualitative approach was chosen to explore these factors in depth. Respondents were selected using purposive sampling techniques, and the data collection process was carried out through semi-structured interviews. The thematic analysis method was then applied to the transcripts of the interviews with the users. Based on the qualitative methodology employed, we found seven factors behind users’ discontinuation of the use of the studied information system. The seven factors were grouped based on two dimensions, namely, experiential factors and external factors. Poor system quality, informational issues, interface issues, and unfamiliarity with the system influenced the experiential factors. On the other hand, the external factors were influenced by workforce needs, talent mismatches, and a lack of socialization. This research offers a novel, in-depth analysis of the factors that cause users to stop using information systems based on direct experience from users. In addition, the results of this study will be used as feedback companies can use to improve their systems.

1. Introduction

The information system life cycle is a cycle that includes various stages that users undergo when utilizing information systems. It starts from the development phase and proceeds to the stage of continuous usage or the discontinuation of usage [1]. In the discontinuation phase, the information system is abandoned, and expensive IT investments go to waste. Unfortunately, there is still little research addressing why an information system is discontinued; most researchers analyze this topic solely from the perspective of information system adoption [2]. The phenomenon of discontinuing use of information systems is very biased, a fact that stems from a lack of understanding of the interrelationship between the adoption of information technology innovations at the individual and organizational levels [3]. Moreover, according to Bhattacherjee, the phenomenon of information system discontinuance often occurs because a system does not meet user expectations, which ultimately affects a user’s decision to stop using the information system [4].

Information systems are generally designed for and tailored to the specific needs of a given company to support certain business processes [5]. One of Indonesia’s startups in the Education Technology sector uses an information system to support its business processes. One example of an information system used to support a company’s existing business processes is the talent pool system. This system is designed to allow a company’s partner organizations to search for IT staff who have completed their training in the company’s bootcamp program and ultimately recruit them into the organization. There are significant problems related to the low utilization of a system by a company’s partner organizations. In the study, out of a total of 15 users, 12 decided to stop using the system in question. The users tended to ignore this system and preferred to manually request IT talent directly from the company’s administrators. This situation posed operational challenges, wherein the administrative teams were forced to manually collect and review IT talent data according to the job descriptions provided by the company’s partners. Eventually, due to the considerable workload of the administrative team, the distribution of talent was well below the target of 80% from the total number of students, with less than 60% of the potential 3645 candidates people distributed to the company’s partners. These problems could have been avoided if the information system had been optimally utilized.

A prior study examined the factors that influence an individual’s decision not to use technology from the perspective of user trust in system performance [6]. The findings of this study show that the choice to use or not use technology is influenced by two perspectives of user beliefs: positive and negative [6]. The ease of use and the usability of technology influence positive optimism. Meanwhile, negative beliefs are influenced by the costs incurred if complaints and technology hinder individual work.

Meanwhile, another study discussed the user perspective on deciding whether to continue or abandon use of a specific piece of technology in a case study on cloud computing [7]. The output of this study revealed four factors that affect decision making, namely, system reliability, system support costs, information security, and system compatibility [7]. The downside of both studies is that they only focus on technologies typically purchased by an organization, such as SaaS (System as a Service) and cloud computing. Besides having a company’s internal system as a primary object of study, the current paper addresses the reason behind the discontinuation of a talent pool system use, which has not been addressed by other works in the literature. Another shortcoming of the two earlier studies is that they only focus on the factors that drive the internal decision to terminate usage from the perspective of individuals or organizations only, without considering external factors that can also affect system discontinuance. Finally, the two previous studies conducted analyses based on quantitative methods, focusing on the results of statistical tests without deeply describing the experiences and backgrounds that affect the perceptions of individuals regarding their discontinuation of information system use.

Therefore, the novelty of this study is that it explores the factors that affected the implementation failure of a talent pool system from the users’ point of view. Moreover, this study also delves deeper into the internal and external factors that affected the decision to stop information system usage. This research will contribute to practice by explaining the failure factors that caused users to want to stop using this system. In addition, this research will also contribute theoretically by identifying the factors that provoked users to stop using information systems, especially the talent pool system, which was an internal system in the analyzed company.

2. Literature Review

This section includes information about fundamental theories. In addition, the research object is described, including a brief overview of the company. Finally, an explanation of the methodology chosen for this study is provided.

2.1. Information Systems’ Life Cycle

In companies, an information system has a life cycle, spanning from pre-adoption to adoption and post-adoption. Some studies define the phenomena experienced by information systems in terms of several phases of life. The pre-adoption phase is defined as the phase occurring before an information system is officially used by a company, as defined by Seran et al. [8]. Their study stated that before adopting an information system, there is a process of defining and negotiating the business process context as well as planning for its usage. Then, there is the adoption phase, which is when information systems are designed and built for an enterprise [8]. At this stage, a company uses an information system in accordance with their business processes and needs that have been previously defined. According to research conducted by Kai Wu et al. (2022), the analysis of individual needs for a system in the early phase of development plays a very important role in determining the success of an ERP project, which is a type of information system [9]. Their study revealed that one of the main causes of ERP project failures is a mismatch between the information technology (IT), capacity needs, and overall project needs [9].

After a company has used an information system, the system enters a post-adoption phase. Saeed and Abdinnour (2013) stated that at this stage, the system will go through routinization, infusion, and extension [10]. Routinization is when an information system’s features are explored and used by users. After that, a system will enter the infusion stage, in which the company will fully integrate the information system into its operations. Finally, the extension phase is when users explore the benefits of the information system in a new context. In this stage, the exploration of the benefits will vary depending on each individual’s absorptive capacity, which is defined as an individual’s capacity to recognize new information value and implement it for a specific purpose [9]. This capacity plays a crucial role in determining whether or not an individual will continue to use an information system [11] and is therefore an important factor in the continuation of an information system.

The existence of a sustainable information system is beneficial for facilitating a positive change in an organization and a community [12]. Examples include resource optimization in software development, using big data to improve healthcare treatments, and facilitating information management improvement through social media analysis. The focus of this implementable strategy highlights the pragmatic implementation of SIS principles [13]. However, the sustainability of information systems needs to be supported by High-Performance Work Systems (HPWSs) [14]. High-Performance Work Systems (HPWSs) concern the capacity of human resource management to improve the skills and productivity of their employees in the workplace to produce competitive benefits [14]. According to the study by Uddin et al. (2023), it is known that with the influence of good human resource management of user skills or the suitability of features for individual needs, businesses can easily adapt to changes [12].

However, the above stages apply in the case of the successful use of information systems. The use of information systems does not necessarily need to be continued by a company. An information system can also enter a discontinuation stage. This phase is described in more detail in the next subchapter.

2.2. Discontinuance of Information System

The research on the reason why individuals choose not to use an information system focuses on situations wherein users or organizations are faced with new information systems and intend to refuse to use them. Furneauz and Wade were the first to examine decisions regarding information system use discontinuation at the organizational level [15]. They argue that the shortcomings of existing systems, degrees of integration between systems, and organizations’ strengths and environments can influence the decision to stop using a system. The decision to stop using a system is related to a person’s intention to continue or stop using the system. The intention to stop or continue using information systems at the individual level has important consequences for management with regard to deciding whether to make changes in order to stick to a plan.

The existing research significantly improves our understanding of behaviors relevant to most of information systems’ life cycles, but there are still significant gaps that limit our ability to comprehensively explain an individual’s decision to stop using an information system [16]. When individual users decide to continue using an existing system but management decides to replace it with a new system, then there will be rejection and interference behavior that can damage the performance of the new system and cause a decrease in motivation, performance, and work ethic [6].

2.3. Theories Relating to Decisions to Use an Information System

In this section, theories related to users’ decisions regarding the use of information systems will be explained, including the theory of reasoned action, the theory of planned behavior (TPB), and expectation–confirmation theory (ECT).

2.3.1. Theory of Reasoned Action (TRA)

The decision to adopt or reject a piece of technology is based on a series of considerations that are in turn based on users’ perceived usefulness of the system in question [17]. This phenomenon is supported by the theory of reasoned action (TRA) developed by Ajzen and Fishbein (1980). Based on the TRA, the behavior of a user is the result of a decision-making process involving careful consideration and planning [18]. In this context, the intent to act (behavioral intention) is formed by two components, namely, attitudes towards behavior and subjective norms. Attitudes toward behavior refer to an individual’s positive or negative evaluation of an action, while subjective norms refer to the perception of the social pressure to perform or refuse to perform the action. The advantage of the TRA is that it has a simple structure and provides a clear insight into how users’ behaviors (attitude) and subjective norms influence a person’s decision to act [19]. The TRA models, created by Ajzen and Fishbein (1980), have formed the basis of various other theories, one of which is the theory of planned behavior (TPB), which introduces a construct that adds to the TRA models [19,20]. The theoretical framework of the theory of reasoned action (TRA) can be seen in Figure 1.

2.3.2. Theory of Planned Behavior (TPB)

The theory of planned behavior (TPB) is an extension of the theory of reasoned action [20]. The TPB was developed by Icek Ajzen (1991) [18] by adding one construct to the TRA, namely, perceived behavioral control [19]. The TPB proposes that these three factors influence behavioral intention, which then influences actual behavior. Perceived control can also have a direct influence on behavior, especially in situations where individuals feel they have greater or less control over their actions [21]. In this theory, perceived control is influenced by factors such as the ability to control beliefs and perceived power [20]. Perceived control is generally an assessment of how easy or difficult it is for an individual to engage in a desired behavior under both external and internal factors. The theoretical framework of the TPB can be seen in Figure 2.

2.3.3. Expectation-Confirmation Model (ECM) Theory

The expectation–confirmation model (ECM), developed by Richard Oliver (1980), is a theory originating in the field of marketing. This theory explains the causal relationship between customer satisfaction and two factors: initial expectations and confirmation/disconfirmation [22]. This theory states that user satisfaction affects the intensity of the consumption of goods/products. In the context of information systems, this theory is often used to understand how users become satisfied with information systems and how this affects their intention to continue using such systems. A diagram of the ECM can be seen in Figure 3.

Anol Bhattacherjee (2011) redeveloped this theoretical framework into the expectation–confirmation model information system (ECM-IS). The ECM-IS is a framework used to explain the reasons why users will continue to use an information system after its adoption [4]. This model focuses on the factors that influence a user’s decision to continue using an information system after its initial use (making use of post-acceptance variables), i.e., confirmation, perceived usefulness, satisfaction, and discontinuance intention. Pictures of ECM-IS theory can be seen in Figure 4.

The similarity between the two types of ECMs developed by Oliver (1980) [22] and Bhattacherjee (2011) is that both conceptualize the initial idea of user satisfaction as result of a process in which initial expectations are compared to actual experience [23]. Though rooted in similar principles, Bhattacherjee’s adaptations provide a more specific framework for understanding how expectations and confirmations influence sustainability behavior in regard to the use of information systems.

2.4. Hedonic and Utilitarian Benefits of Information Systems

Utilitarian and hedonic values influence users’ decisions in using information systems [24]. Utilitarian benefits are benefits that users derive from a system, with a focus on system efficiency, productivity, and performance [25]. Hedonic benefits refer to the benefits related to the emotional experience of using information systems [25]. According to Alnawas et al. (2023), utilitarian benefits have a strong influence on cognitive processes, while hedonic benefits tend to affect affective (emotional) processes [26]. According to motivation theory, utilitarian benefits represent extrinsic motivation, while hedonic benefits represent intrinsic motivation [27].

Utilitarian benefits are extrinsic motivations because they emphasize the behavior a user engages in to obtain a specific goal or reward [24]. Utilitarian benefits are generally influenced by a user’s assessment of system performance, such as speed, information quality, and the suitability of features to the user’s needs [26,28]. Utilitarian benefits are also more objectively based on the assessment of the experience of using information systems, which contrasts with hedonic benefits, which are based on the user’s pleasurable experience [24]. Hedonic benefits are more likely to come from internal motivation because they are related to satisfaction obtained from the use of information systems. Hedonic values can be transmitted through aesthetics, gamification, interactivity, perceived enjoyment, perceived ease of use, entertainment, and information personalization [25,29]. Users will tend to feel happy when an information system also focuses on visual value or user experience [30]. The research by Cheng (2023) states that utilitarian and hedonic benefits affect users’ decisions regarding whether to continue using an information system [29]. The conclusion they came to is that it is crucial for those involved in the information systems field to pay attention to users’ utilitarian and hedonic benefits because both affect users’ decisions to use information systems.

2.5. Company Profile

PT XYZ is an Indonesian Edutech startup that focuses on information technology competency training in fields such as programming, data science, UI UX design, and product management. PT XYZ was founded in 2020 and now has 200 employees. One of its core business processes involves providing IT talents to partner companies. To support business processes, the company created a talent pool system with the aim of channeling IT talent according to the needs of partner companies, thereby increasing the number of successful job placements for students who are part of PT XYZ. In the talent pool system, activation of company partner accounts was carried out by the internal team based on the duration of the partnership contract. Meanwhile, students could use their e-learning accounts during training to access the talent pool system.

The interface on the partner company’s side offers features such as talent search, talent profile details, job posting, and talent recruitment. Partners can search for talent based on name preferences, age, and IT skills. Additionally, partners can view detailed talent profiles including personal information, educational background, work experience, CVs, and portfolios. They can also schedule interviews with talent and update interview statuses. Unfortunately, the system is not integrated with Google Calendar yet, so all communication occurs through the system and email correspondence. Moreover, partners can post new job openings and view interested talent applying for said positions. The student account interface includes features for job applications, job application status details, and profile updates. Students can update their profiles, CVs, and portfolios to attract partner companies. Furthermore, students can apply for jobs posted by partner companies.

There are, however, significant problems related to the low intensity of use by the company’s partners. They tend to ignore this system and prefer to manually request IT talent directly from the company’s administrators. Out of a total of 15 users, 12 decided to stop using the system. This situation posed operational challenges, wherein the administrative teams were forced to manually collect and review IT talent data, which then had to be adjusted to the job descriptions provided by the company’s partners. In addition, due to the considerable workload of the administrative team, the number of work distribution programs is still below target, with less than 60% of 3645 potential hires distributed. Therefore, this study was conducted to identify the factors preventing an information system from being reused via a case study of the talent pool information system.

2.6. Research Method

This research was conducted using a qualitative descriptive method. This method was chosen because the purpose of the research is to understand a phenomenon by providing an in-depth description of events that occurred in the field [19]. Previous research, such as the study conducted by Recker and Bian, who used quantitative analysis methods to ascertain the factors that cause discontinuity of IS use, focused only on measuring and analyzing the factors statistically [20]. In contrast, we of this study used a qualitative method to explore and understand the complexity of a phenomenon in depth [31] The processes carried out were data collection/preparation and descriptive interpretation of the data. The descriptive qualitative methods were not intended to be used to create new theories but instead to allow for a focus on understanding new phenomena. In this study, we obtained the consent of all interview respondents regarding their involvement in the research while maintaining the confidentiality of the respondents’ personal data.

2.7. Research Instruments

The questions asked were semi-structured, with each question having been created based on previous research by J. Recker and Y. Bian [6,7]. All research instruments have been tested for readability, using purposive sampling in selecting respondents for the readability test. The sampling criteria used are individuals who have a strong understanding of the talent recruitment process or practitioners with at least 3 years of experience in information system development. The respondents for the readability test were four people from the internal HR team and three analysts from the IT division of PT XYZ. The results of the readability test showed improvements in difficult-to-understand questions, including revisions of ambiguous word choices. The list of research instruments can be seen in

Table 1. Research instrument and interview question list.

No	Questionnaire Questions	Theme Construct	References	Reason
1	Do you use a PT XYZ’s talent pool system for talent search?	Usage Continuance	[4,6,7]	To identify the relationship between individuals’ use of the talent pool system and their intention to continue using it.
2	How often do you access the PT XYZ’s talent pool system?	Usage Continuance	[4,6,7]
3	In your opinion, can PT XYZ’s talent pool system help in the search for talent data?	Perceived usefulness	[6]	To analyze individual perceptions regarding the level of usefulness of the talent pool system.
4	Does the use of the PT XYZ’s system help you to do your job?	Perceived Usefulness	[6]
5	Is the talent search process using the PT XYZ’s talent pool system easier for your work when compared to using spreadsheets?	Perceived Usefulness	[6]
7	Can you easily search for talent that matches the job requirements through the talent search feature in the PT XYZ’s talent pool system?	Perceived Ease of Use	[6]	To understand users’ opinions about how easy it is to operate the features of the talent pool system.
6	Do you find it difficult to use the PT XYZ’s talent system?	Perceived Ease of Use	[6]
8	Can you easily use the interview invitation feature for talents?	Perceived Ease of Use	[6]
9	Are you having trouble using the interview invitation feature for talent?	Perceived Ease of Use	[6]
10	Do you experience confusion when you first access/use PT XYZ’s talent pool system?	Perceived Ease of Use	[6]
11	Have you ever experienced difficulties or encountered errors in using the PT XYZ’s talent pool system?	System Reliability	[7]	To analyze individual perceptions of trust in the reliability of the talent pool system, including regarding the speed of information processing and information that meets user needs.
12	Do you think this system can be trusted to conduct accurate talent searches according to job descriptions?	System Reliability	[7]
13	Have you ever encountered errors related to system performance during the use of PT XYZ’s talent pool? (Example:system downs, failed logins, and the searched talent data did not appear.)	System Reliability	[7]
14	Do you think the appearance of the PT XYZ’s talent pool system is now comfortable to use, whether accessed via desktop or mobile devices?	System Compatibility	[7]	To ensure how well the talent pool system suits the devices and applications that users usually use, because this affects user satisfaction and the usability of the system.
15	Is the output data from PT XYZ’s system directly and quickly generated without delay when you use it?	Confirmation	[4]	To ensure functional aspects of the talent pool system.
16	How compatible is the system’s ability to display talent data with the job requirements needed by the company (company partner)?	Confirmation	[4]
17	Do you have any insecurity related to data privacy when accessing PT XYZ’s talent pool system?	Information Security Threats	[6,7]	To identify individual trust in the talent pool system in relation to possible risks regarding data security.
18	Do you change your company partner account’s password regularly?	Information Security Threats	[6,7]
19	Do you give company partner account access to HR/Other Co-workers?	Information Security Threats	[6,7]
20	Do you have any input related to PT XYZ’s talent pool system?	(Open Ended Question)	-	To collect suggestions and feedback from users about the talent pool system.
21	If you could provide one feature that could help your team’s work, what would it be and why?	(Open Ended Question)	-

.

2.8. Research Participants

The participants were selected using purposive sampling. Purposive sampling is a sampling technique that is carried out by deliberately selecting samples based on certain criteria and characteristics [32]. The respondent selection criteria were as follows: individuals who served as persons in charge (PICs) working for company partners who were in active partnership regarding recruiting IT talent from PT XYZ but no longer used the talent pool system. The selection of respondents was focused on partner companies that stopped using the talent pool system as it helps in identifying the barriers that caused them to stop using the system, as well as providing broader insight into the factors that affect the acceptance or rejection of the system. Out of the 12 respondents who met the interview criteria, only 7 were willing to participate in an interview. The interviews with the respondents were conducted online using Zoom conferencing tools. We used the semi-structured in-depth interview method, which is a recommended qualitative descriptive method, to directly obtain information in harmony with the facts, events, activities, or opinions from each participant [31]. The interviews were conducted in Indonesian. In addition, the participants were IT recruiters or IT talent acquisition professionals with experience sourcing IT talent. A list of all the respondents can be seen in

Table 2. Interview participant data.

No	Participant Code	Role	Age	Gender	Company Field
1	R1	IT Talent Recruiter and Acquirer	24 y.o	Female	IT Outsource and Service
2	R2	IT Talent Acquirer	25 y.o	Female	IT Outsource and Service
3	R3	IT Talent Recruiter	27 y.o	Male	Property Management
4	R4	IT Talent Recruiter	24 y.o	Male	IT Outsource and Service
5	R5	IT Talent Acquirer	37 y.o	Female	Clinics and Beauty Products
6	R6	IT Talent Acquirer	40 y.o	Male	IT Service SaaS HR and Finance
7	R7	IT Talent Acquirer	20 y.o	Female	IT Software House

.

2.9. Participant Interviews

Data collection (interviews) was conducted from 25 April 2023 to 5 May 2023. The duration of the interview for each participant was 25–45 min. Before the interview, each participant was given information regarding the purpose of the interview. The results of the interview were recorded in one document. In addition, after the interviews were over, the interviewer confirmed whether the interview results were in accordance with the answers obtained from the participants.

2.10. Data Analysis

Thematic analysis was used for data analysis in this research. According to Bogdan and Taylor, thematic analysis is the process of finding themes, formulating working hypotheses, and sorting data into basic patterns, categories, and units of description so that specific themes can be found [33]. The definition of the theme itself is core information that can stand on its own. In this study, we conducted data analysis with the assistance of Atlas.ti software version 23. In the first step, we created codes based on the identification of the issues that appeared in the interview results. Then, we read and identified patterns and themes in the data. After acquiring the results of the analysis, we verified the results with an expert in the field of information systems regarding the suitability of the predetermined theme. The output of this stage was the factors that caused the participants to choose to stop using the talent pool information system.

3. Results

The data were collected using in-depth semi-structured interviews with seven participants. The participants had the same duties and work responsibilities in the human resource division but different role names, namely, IT Recruiter (n = 2), IT Talent Acquisition and Recruiter (n = 1), and IT Talent Acquisition (n = 4). The participants were predominantly female (n = 4), with fewer males (n = 3). This study found twelve factors that caused company partners to decide not to use the talent pool information system. The research results can be seen in

Table 3. The research results.

Dimension	Construct	Description
Experiential Factors	Poor System Quality	Conditions wherein an information system does not meet expected or desired standards regarding performance, reliability, ease of use, or other relevant aspects.
	Information Issues	Conditions in which the information provided by an information system does not meet the expected or desired standards.
	Interface Issues	Problems or challenges related to the user interface of a system or product.
	Unfamiliarity	Conditions wherein someone is unfamiliar with the workings, interface, or features of the system.
External Factors	Talent Mismatch	Situations where an individual’s skills, knowledge, or levels of experience do not match the job’s requirements.
	Workforce Needs	The organization’s need for a certain number of workers
	Lack of Socializations	Lack of interaction or involvement of organizations or individuals related to training in using available systems or technologies.

.

3.1. Experiential Factors

Experiential factors are variables related to an individual’s direct experience when interacting with a system, product, or service. User experience can relate to the user interface, aesthetics, and interactivity [34]. User experience plays an important role in the use of information systems, impacting the emotional aspects of the user and thus influencing their decision regarding whether to use a system [26]. According to DeLone and McLean, the successful implementation of technology also influences an individual’s decision to use information systems that impact organizations [35]. The successful implementation of technology plays a crucial role in facilitating and optimizing a system’s overall functioning, which affects the user experience. This covers all technical and functional aspects of the system infrastructure, including performance, reliability, ease of use, and integration with existing operational needs [36]. In the analysis conducted this study, it was found that there were two main problem factors, one of which consisted of experiential factors, encompassing information issues, poor system quality, interface issues, and unfamiliarity with the system.

3.1.1. Poor System Quality

In Primiero’s (2014) study, errors in information systems refer to the condition wherein a system produces outcomes that are incorrect or unexpected [37]. The effect of system errors on users may vary depending on the specific error and the context in which it occurred. In some cases, system errors can cause discomfort for users, such as when an application cannot be used. System errors can also result in data loss or corruption, leading to more serious consequences. Therefore, it is important to address and correct system errors as soon as possible to minimize their impact on users and prevent further problems from arising. On the talent pool system, system errors that occurred included users failing to log in, the appearance of a blank page, and unexpected filtered data. The following are quotes from each respondent regarding some of the problems pertaining to the quality of the talent pool information system:

“Then it is still often found that the system case displays a blank page (just blank white without any information).”

(R1 and R2)

“I’ve failed to log in once, even though I logged in with the same username and password.”

(R4)

“In my opinion, it is still inaccurately displaying the data because I have found cases where I was looking for digital marketing talent, but what appeared was talent from product management”

(R1 and R2)

“The obstacle I experienced while using this system was that the talent filtering feature often errs”

(R1)

3.1.2. Information Issues

Information is a set of data that has meaning [34]. In the context of this study, information related to talent data was stored electronically and managed using a database management system. Databases were structured such that they would facilitate various data-processing operations. The accuracy and reliability of a database can be gleaned from the true reflection of real-time changes in the data it represents. When a database does not meet its accuracy and reliability standards, problems that may disrupt the end-users can occur [38]. In PT XYZ’s talent pool information system, there are several information problems, including double keywords for the data-filtering feature, a lack of detailed information on talent data, and talent data that are less varied and not up to date. The following are excerpts from each respondent regarding problems with information in the talent pool system:

“in the skills dropdown on the filtering feature, there is a skill name category that appears double. Examples of Javascript and Javascript Programming.”

(R3 and R4)

“but I think it can be improved again in terms of providing more talent data”

(R2 and R5)

“I checked the talent database but it was not updated; all the old talents are still there.”

(R2 and R4)

“and found many incomplete talent data (e.g.: CVs and resumes are not filled by talent)”

(R1, R2, R4, and R5)

“But it’s unfortunate that for the skill section, there is less information provided on how deep the talent has experience for the skill. Example: 4/5 Javascript Skill”

(R6)

3.1.3. Interface Issues

Every digital product, be it a system or application, has an interface design, which is often referred to as a user interface or user experience [39]. User experience usually refers to challenges or problems that arise in the design of the interface of an application or system, such as feature layout, color selection, and the ease of operating an information system [39]. User interfaces play a big role in affecting users’ comfort and, in turn, will affect the decision regarding continuing to use an information system [34]. Some of the problems with the interface of the talent pool system included a less attractive mobile display, components that were too packed with writing, font sizes that were too small, and the selection of less appealing color palettes. The following are some quotes from each respondent regarding problems with the display of the talent pool information system:

“Honestly, in terms of UI/UX Design, I think it’s still lacking; it looks too plain and less attractive”

(R1)

“You see, I feel that the current design is indeed better than the talent pool system version 1, but there is still a lot of information displayed quite densely, and the font size is also too small”

(R4)

“Then the appearance and color selection are less attractive”

(R4 and R6)

“I prefer to use websites (in desktops) over mobile because I think the mobile display feels less neat when compared to desktops”

(R7)

“Just put it from me in the talent explanation card; there are too many writings displayed.”

(R3 and R6)

3.1.4. Unfamiliarity

User familiarity refers to the level of knowledge, experience, and comfort of a user with regard to a product or piece of technology [40]. It reflects a user’s ability to properly use all features and functions of a product according to its purpose. In this study, it was found that some users were still unfamiliar with using the talent pool system; for example, they experienced confusion when they first used the system and encountered difficulties when accessing features without guidance. Based on these problems, it can be seen that the current talent pool system is still not user-friendly because users are still confused with some of the available features. The following are quotes from each respondent regarding the problem:

“I’m a little confused when I see the job posting and talent interview features, maybe I’m not familiar with how it looks”

(R5)

“At the beginning of use, I need to try the feature first, my fault is that I also didn’t open the guidebook.”

(R6)

3.2. External Factors

External factors include influences from the organization as well as the social environment of individuals that influence their decision to use an information system [41]. In the context of this study, external factors are related to demand from stakeholders or the organizational drive for IT talent searches. In addition, external factors are also related to the extent to which the expertise of IT talent is in accordance with the job requirements.

3.2.1. Talent Mismatch

The talent pool system aims to assist in the process of distributing talent work. In this study, it was known that one of the reasons behind the failure of the studied system is the mismatch of talent capabilities with partner companies’ needs (job vacancies). This condition can be described as a talent mismatch. Talent mismatch can be interpreted as a condition wherein there is a mismatch of the abilities possessed by the talent with respect to the needs or demands of available jobs [42]. Based on this, it is known that several things cause talent mismatch, including the unsuitability of a prospective employee’s educational background to the minimum requirements of job vacancies, abilities that are not what the company sought for, and finally companies’ preference for looking for experienced talent in a given field, posting extra requirements such as a minimum of one to two years of work experience. The following are quotes from each respondent regarding the problem:

“Just now, there are still many talent databases, which are not in accordance with the requirements I need at this time.”

(R1 and R5)

“Only a few in the talent databases fit the current needs.”

(R1)

“Sometimes the talent is ok, but the background is lacking, for example, there are currently many job requirements with a minimum requirement of bachelor’s degree (S1), but in the system, it is more common to find high school/vocational graduates.”

(R1, R2, R4, and R6)

3.2.2. Workforce Needs

Another external factor is workforce needs, which, in this context, are defined as the demands of IT talents in the workforce. Some of the problems related to workforce needs include low demand for IT staff, the existence of a ’hiring freeze’ in the company, or the need for a fast IT talent search that leads to a lack of time for company partners to look for talent in the talent pool system. The following are quotes from each respondent regarding the problem:

“But unfortunately these few months the office has been in hiring freeze”

(R7)

“Currently there is no open opportunity and the HR team also has other tasks so, yes, they did not access PT XYZ’s talent pool system.”

(R6)

“But honestly, we rarely access the talent pool system because the demand for IT outsourcing is currently quite low.”

(R1)

“…only sometimes there are external factors that requires fast talent data to fill positions in urgent companies, so it’s more convenient to ask for talent through admin talent support.”

(R3, R4, R5, and R6)

3.2.3. Lack of Socialization

In the practice of implementing information systems, socialization is needed to explain the function and usefulness of information systems. Socialization is needed to increase user awareness and knowledge of certain information systems [34]. Based on the results of this study, the participants were not aware that PT XYZ’s talent pool system was running because there was no information and socialization from the company’s management side. The following are quotes from each respondent regarding the problem:

“To be honest, I rarely use PT XYZ’s talent pool system because I am not aware that this system was running as one of the work distribution programs from PT XYZ.”

(R2)

“But it’s a pity that I just found out that there is information about the improvement of this talent pool system.”

(R7)

“I was just confused because the talent support team of PT XYZ was often more responsive to give talent recommendations via spreadsheet compared to updating talent data in the system.”

(R2)

4. Discussion

The results of the research model of the factors found can be seen in Figure 5, while the frequency with which constructs emerged during the thematic analysis can be seen in

Table 4. Number of themes in the constructs.

No	Construct	Frequency
1	Poor System Quality	4
2	Information Issues	5
3	Interface Issues	8
4	Unfamiliarity	2
5	Talent Mismatch	3
6	Workforce Needs	6
7	Lack of Socialization	2

.

Based on the results of the analysis, we found that we could group the factors into two dimensions of factors: experiential factors and external factors. Experiential factors are influenced by poor system quality, informational issues, interface issues, and unfamiliarity with the system, while external factors are influenced by workforce needs, talent mismatches, and lack of socialization.

The dimension of experiential factors is relevant to users’ expectations in regard to exploiting the usefulness of an information system, namely, their desire to obtain utilitarian and hedonic benefits [43]. Utilitarian benefits are related to cognitive assessments, namely, system quality factors and information quality factors [25], while hedonic benefits are related to subjective experiences, such as convenience perception factors that affect familiarity and interface display factors [44].

The first factor in the experiential factor dimension is system quality. The quality of information systems is known to influence user decisions when using such systems. This is in accordance with research by Rongbin et al. (2023), who stated that system quality influences decisions on using information systems from a technical point of view [45]. System quality is also related to processing speed and a system reliability with respect to meeting user needs [46]. Finally, the quality of an information system also has a significant effect on user satisfaction and is a factor that influences the success of information system implementation [47]. Another factor is the information quality factor, which refers to the quality or accuracy of the information presented by a system or application [48]. Information quality can also be a benchmark for the success of an information system. A good information system must also have high credibility. These qualities can be seen from the relevance of the information provided and the source’s reliability [49]. This is in line with the study by Yang et al. (2023), who states that information quality influences user decisions regarding satisfaction with information system performance [45]. The third factor, namely, user interface, refers to the visual appearance of an information system, which subjectively influences the user experience [26]. The consistency of the visual elements of an information system has been proven to influence the user’s ease of use of the information system [30]. Apart from that, the aesthetic value of a system can be seen from the combination of elements such as images, colors, font selection, and so on [26]. Visual appearance also has a significant effect on satisfaction with and the ease of use of information systems [50]. The fourth factor is the user’s familiarity with the system, as seen by the perception of ease of use of the information system [25]. One of the things that reduces users’ familiarity with a system is their low intensity of use of the system [51]. To improve users’ sense of familiarity with an information system, organizations need to encourage users to adopt and use it [51].

The second dimension of factors, the external factor dimension, proves the influence of organizational motivation and socialization with respect to a system on the successful implementation of information systems. According to Roca et al. (2006), it is known that organizational motivation has a significant effect on satisfaction in information system implementations [46]. The main motivation for partner organizations, which is to match IT talents to their demands, is generally hard to predict for PT XYZ. One possible way for the company to predict demand is by evaluating and communicating with the partner companies regarding IT talent needs. The three external factors must be managed well because this forms a part of the user’s expectations in using the information system [52]. It is known that external factors can influence the perceived usefulness and perceived ease of use of information systems [52]. To better understand all the factors found in this research, we created a theoretical model based on these findings, connecting the seven factors to the discontinuation of the use of the talent pool management system. The theoretical model for the discontinuation of the use of the talent pool management system can be seen in Figure 5.

In Figure 5, it is shown that each factor directly influences the discontinuance of the talent pool IS; this is based on the results of the analysis of the findings. When relating these findings to the ECM-IS theory proposed by Bhattarcharjee, it can be surmised that before users decide to continue using an information system, they undergo several stages, namely, perceived usefulness and satisfaction [4]. The factors found in this research can be categorized into the expectation construct in the ECM-IS theory. However, the model made in this study is based solely on the findings regarding the identified factors, so the result was purely based on the thematic analysis of the interview transcripts.

5. Conclusions, Implications, and Limitations

This study evaluated the factors that affect the discontinuation of the use of a talent pool information system used by the edtech startup company PT XYZ. The results of this study show that there were two dimensions of factors affecting a user’s decision to stop using the talent pool system, namely, experiential factors and external factors. This finding is in line with the success theory of information systems developed by Delone McLean (2003), which states that the quality of the system and the quality of the information provided affect the successful implementation of the information system [35]. On the other hand, the ease of operation of information systems also has an impact on user satisfaction and familiarity, thus influencing the decision to use information systems [26]. Moreover, the consistency and harmony of the user interface of the information system create a memorable visual experience of the interface [30], thus influencing the decision to use the information system [25].

Finally, there are external factors that affected the decision to use the application, namely, talent suitability, workforce needs, and the socialization process carried out by the company’s internal team. Talent mismatches and workforce needs reflect the drive of corporate partners to seek talent according to their specific needs, which is difficult to predict. Furthermore, the lack of socialization is also a determining factor because encouragement from the providing organization has an important influence on the successful use of information systems.

In terms of practical implications, this study’s results suggest that organizations should pay attention to the quality of their systems and the information they present. An evaluation of the interface of an information system should also be carried out periodically by paying attention to user expectations so that users feel comfortable and satisfied while using the information system. Organizations should also analyze the needs of company partners and their talent and encourage users to use talent pool information systems to address the issues related to the external factors.

In terms of theoretical contributions, this research adds to the understanding of the factors that influence a user’s decision to stop using an information system. The research results show the influence of internal and external factors. Internal factors are driven by users’ experience of using an information system, where utilitarian benefits tend to come from functional aspects of the system, such as system speed and information quality, while hedonic benefits are driven by aesthetic value and comfort felt by users. External factors that encourage users to stop using information systems are driven by conditions of workforce needs, talent suitability, and socialization regarding information system use.

The weakness of this research lies in the limited sample size of organizations, as the number of information system users was less than 50 people, thus possibly creating differences in research results with respect to information systems used by organizations that have a larger number of system users. In addition, this research only presents the findings of factors that influence information system discontinuity without analyzing in depth the factors that are most significant in influencing the user’s decision-making process. Therefore, we hope that in future research, a more in-depth analysis of the influence of each of these factors can be carried out. Finally, the results of this research are specific to this case study on a talent pool system, so the results are not necessarily the same as those obtained in studies on other information systems. Therefore, we highly recommend that researchers evaluate the factors influencing the decision to stop using other information systems by considering the factors found in this research.