*3.6. The Importance–Performance Map Analysis (IPMA)*

To obtain further insights, IPMA was used, by merging the importance (I) and performance (P) dimensions analysis [20,38]. IPMA allows identifying areas where the action is required. Namely, one may identify parts of the process with relatively high importance yet relatively low performance to implement the corresponding management tools leading to improvements. Table 12 and Figure 6 show both dimensions of the constructs influencing the dependent variable—ExU.

**Table 12.** Data of the importance–performance map for extended use of ERP system (ExU).


IPMA results are presented by the two-dimensional graph, where the horizontal axis describes the "importance" (total effect) of influential factors using a scale from 0 to 1, and the vertical axis describes their performance, using a scale from 0 to 100. The graphs in Figure 6 and Table 12 reveal that the most important construct was WC, followed by STC, AT, OPC, PCIL, PU, and PEOU. However, the construct with the best performance was PCIL, followed by the AT, STC, WC, PU, OPC, and PEOU. The most important finding is that the performance of WC does not match its importance. Consequently, for managerial activities to increase the ExU, the emphasis should be on the construct of WC, which can be obtained by emphasizing the predecessors of the second-order construct STC and second-order construct OPC, where performance still has the possibility for significant improvements.

**Figure 6.** 3 IPMA for the endogenous variable extended use of the ERP system (ExU).

#### **4. Discussion**

The main objective of this paper is to show the effectiveness of the proposed methodological process, which consists of five phases and are aimed at determining the important information provided by the results of each implemented methodological phase: (i) PLS-SEM—assessment of measurement model; (ii) PLS-SEM—assessment of structural model; (iii) PLS-SEM—the blindfolding procedure; (vi) ANN analysis based on PLS-SEM results; (v) IPMA procedure based on PLS-SEM results.

We presented a research case and a model of constructs influencing the adoption and in-depth use of important information technology in companies by employees. We emphasize that the information technology we are considering is crucial for maintaining the company's competitive position and for creating its competitive advantages; moreover, in the conditions of the digitalization of business, which are especially present in the automotive industry (which we considered in our research case), the in-depth use of information systems in the company is the basis of and condition for the operation (existence) of the company. The information technology in question is the ERP, which is a core information system in modern companies.

The importance of successful ERP implementation in all phases of the life cycle is extremely important for companies [46,64,69,81]. Nevertheless, this implementation often suffers from failure, reflecting the fact that the success of the advanced use of ERPs in companies is influenced by several factors, in turn influencing the level of acceptance of these systems by employees. In the research case presented in this paper, we analyzed the factors that influence the acceptance of ERPs by employees in companies based on the attitude they develop towards this information technology and their in-depth use of it. The importance of this topic for companies is also evidenced by extensive research work and the search for solutions to reduce the failure rate through process modeling with a series of theoretical models. In doing so, the TAM has been confirmed to be appropriate in a number of different cases, with SEM proving to be an appropriate methodology used to test the model [11–14,109–113].

Although business information solutions and systems themselves are becoming increasingly complex, the research methodology for testing theoretical models in this field

relatively rarely uses advanced, new approaches within SEM or their combinations with advanced artificial intelligence methods despite the high level of complexity associated with human decisions (which is, of course, the case in the field addressed in this study). Results show that the proposed methodological process enriches results of PLS-SEM; this is achieved using the advanced data analysis methods of the ANN and IPMA, thus creating the basis for evidence-based, grounded business decisions to support the development of the mature use of ERPs in companies. We explored a combined methodological approach involving PLS-SEM, advanced new procedures in this framework, and ANN analysis of artificial intelligence that can intervene in the linearity of the SEM model. We wanted to supplement the PLS-SEM results in terms of the assumption of nonlinear relationships in the model on one hand and, on the other hand, to establish the ranking of the factors obtained with PLS-SEM according to their relative importance as predictors.

The research results obtained in the first three phases (i) PLS-SEM—assessment of measurement model; (ii) PLS-SEM—assessment of structural model; and (iii) PLS-SEM the blindfolding procedure) follow.

The outcomes endorse the existence and significance of most of the expected relationships foreseen in the structural model using the PLS-SEM technique (Figures 3 and 4), except relationships for construct PEOU (hypotheses H1 and H2), which are two relationships proposed by Davis [75,76]. As Figure 4 shows, construct PEOU had no significant impact on the construct PU and/or the construct AT. This finding corresponds with findings of research studies by other authors, which argue that PEOU seems to be more meaningful during implementation phases of the ERP system and becomes less important in the latter stages of the ERP system life cycle, when the system is in use for a longer time [30,76].

Data analysis shows that construct PU had a direct positive impact on the construct AT, which confirms H3 and prior conducted research studies [33,75,76]. Construct WC was introduced as the level to which an ERP user is able to implement almost all of his/her work duties using implemented ERP system. Our investigation shows that construct WC influenced construct PU, which confirms hypotheses H4 and prior findings [34,35,81,97]. Construct WC also directly and indirectly (through construct PU) influenced construct AT, which confirms hypotheses H5 and prior research studies [30,34,35]. Construct WC also directly and indirectly (through construct AT) influenced construct ExU, which confirms hypothesis H6 [35]. Construct AT did not have as strong a direct impact on construct ExU as construct WC, but it was a significant one (which confirms H7).

The research included external factors by grouping them into second-order factors (which confirms hypotheses H8a, H9a, H9b, H9c and H10a). Results (Figure 4, Table 8) show that several important external factors were identified.

Research results of the fourth phase ((vi) ANN analysis based on PLS-SEM results) that enrich the results of the first three phases follow.

The results show that some relationships show a significant deviation from linearity, which was expected given the content characteristics of the variables in the model. There were some, albeit minor, differences between the findings of the traditional PLS-SEM technique in addition to the ANN analysis, which can represent important added value and useful information for the informed decision-maker and the basics for business decisionmaking. For example, such a result is a different order of importance for factors or predictors of WC values: the most important predictor was OPC, followed by STC and PCIL, which is in contrast to the PLS-SEM results, in which STC had a stronger influence than OPC. Similarly, the ANN model predicted that WC had a stronger impact on ExU as compared to AT, which is consistent with the PLS-SEM findings, but the relative difference between their importance in ANN is slightly smaller. The results of the ANN analysis and the differences with the SEM results reflect the higher prediction accuracy of the ANN models, which consider existing non-linear effects among variables [62].

Research results of the fifth phase ((v) IPMA procedure based on PLS-SEM results), add the following important information to what has already been gained.

The last step in the research was the importance–performance analysis to identify the gap between the levels of importance and the levels of performance of factors in the model. Based on IPMA results, the researched company can improve ExU through construct WC and its second-order antecedents, as well as via the STC, where the most effort should be focused on data quality and accuracy, higher system performance, better user manuals, and improved ERP system functionalities.

By implementing this methodological approach, this research gives important insights on how to increase the recognition of the impact of several constructs that can expand the level of the ExU in the maturity stage. Knowing the structure of the individual important factors that we have identified, but which have not yet been sufficiently developed in the company, it is possible to form direct instructions for the implementation of managerial decisions based on the results of the research. The implementation of these decisions then affects the success of the acceptance of the ERP system in the mature phase of the use of these systems in the company and results in more in-depth use of ERP solutions by ERP users. This contributes to improvements in their productivity.

This study was limited to quantitative research and was conducted using the TAM research method for the described sample. We also believe that the described findings can be extended by implementing the qualitative approach, which could further enrich the understanding in the field of acceptance and perception of information technology by employees in companies. As we have already noted, human perception and assessment are complex. Therefore, additional qualitative research, with in-depth interviews with carefully selected focus groups (IS/IT users, managers, developers of business information systems) could represent important value-added content from the conducted quantitative research. In addition to that, it would make sense to upgrade this research by examining the differences regarding acceptance of an ERP system by two groups (management and employees) as the scope of functionality that one group of users (employees) has to use varies greatly as compared to the other group (management). This study was also limited to the mature stage of ERP usage in the company; therefore, future research may also investigate the factors that influence the acceptance of ERP solutions in different stages of the life-cycle of ERP usage in companies. Furthermore, it would be also worth studying and testing the model for other business information systems such as CRM, HRM, DMS, etc.

#### **5. Conclusions**

The results of our research are important from two perspectives: (i) from a methodological perspective and (ii) from a business practice perspective, to provide a basis for evidence-based decision-making.

From a methodological point of view, we showed how to upgrade the traditional PLS-SEM method results with the artificial intelligence method of ANN analysis and with new advanced techniques within PLS-SEM. We showed how to enrich the results of the PLS-SEM model by identifying nonlinear relationships in the model and by analyzing the relative importance of factors in the model, as well as by IPMA, which in some way shows a bottleneck in the process—the possible gap between the levels of importance and performance for individual factors in the model.

On this basis, business managerial decisions can be more in-depth and reasoned. We used this approach in the example of the model of the acceptance of business information systems by users in organizations, where we studied the mature stages of the use of ERP systems in a company. The presented methodological process is useful in various areas of the business decision-making process in organizations.

**Author Contributions:** Conceptualization, U.Z., S.B. and P.T.; methodology, S.B., P.T., Z.K. and S.S.Z.; software, U.Z., P.T., Z.K. and S.S.Z.; validation, U.Z. and S.B.; formal analysis, U.Z., S.B., P.T., Z.K. and S.S.Z.; investigation, U.Z.; resources, S.B., P.T. and S.S.Z.; data curation, U.Z. and S.S.Z.; writing original draft preparation, U.Z.; writing—review and editing, P.T. and S.S.Z.; visualization, U.Z., P.T., Z.K. and S.S.Z.; supervision, S.B. and P.T.; project administration, S.S.Z.; funding acquisition, P.T. and S.S.Z. All authors have read and agreed to the published version of the manuscript.

**Funding:** Authors acknowledge financial support from the Slovenian Research Agency (research core funding No. P5–0023, 'Entrepreneurship for Innovative Society) and the Erasmus+ programmes (grant No. 2019-1-CZ01-KA203-061374 "Spationomy 2.0" and grant No. 2019–1-PL01-KA203-065050 "Economics of Sustainability").

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.
