Measuring Efficiency and Satisfaction in the Context of Digital Transformation

Abstract

Currently, much attention is paid to digital transformation in all areas, including the public sphere. The latest studies show that it is necessary for the public sector to monitor the efficiency and satisfaction with the services provided. However, there are significant gaps in research in this area, including in Slovakia. This research proposes and applies the measurement of efficiency using the DEA method in the context of e-Government, provides a comparison of the roles of states in the use of public electronic services in the EU, and applies the method of measuring satisfaction using the American Customer Satisfaction Index, focused on the central state portal in Slovakia. The main methods that were used to fulfil the objectives of the work were data envelopment analysis, “DEA”, and the American Customer Satisfaction Index, “ACSI”. Other methods used include the Mann–Whitney U test, the chi-squared test, and Sperm correlation analysis. From the results of the work, it is possible to conclude that ACSI can be applied within Slovakia. Furthermore, the results show a strong correlation between perceived quality and satisfaction, which is 0.855. Overall satisfaction with the central state portal of public electronic services reached 61.7%. We conclude that it would be appropriate and possible to use ACSI as part of DEA measurement.

1. Introduction

Currently, a period of digital transformation is underway, in which almost all information and knowledge are transformed into a digital form, which allows us to store a large amount of data. In addition, it allows us to efficiently process, search, archive and categorize data. Humans are developing ever more advanced systems that extend into the field of artificial intelligence, which is based only on binary computer notation (Harari 2015). Citizens of countries constantly communicate with public administration institutions. Therefore, it is important to improve and simplify communication using information and communication technologies (ICT) that contribute to more effective interaction between the given subjects. Knowledge is becoming increasingly important in our society, and its proper use is a key aspect of success in today’s digital world.

Efficiency represents the use of economic resources that bring the maximum level of satisfaction achievable with given inputs and technologies (Beňová et al. 2006).

The efficiency of any phenomenon or process is the result of the relationship between the size of the inputs put into the realization of this phenomenon or process and the size of the outputs that result from the realization of this phenomenon. It follows that the effective unit reaches the value 1 and the other units reach values in the interval (0.1) (Beňová et al. 2006). The goal is to approach the value of 1 in efficiency. In this context, an important question is how to measure inputs and outputs when it comes to the public sector and the field of e-Government (Beňová et al. 2006). Economy, efficiency and effectiveness represent the 3Es. The 3E principles refer to the following:

• Economy: performing activities at the lowest possible cost.
• Efficiency: maximizing output with the minimum input.
• Effectiveness: achieving the desired objectives or producing the intended effect (Greenlaw et al. 2022).

To ensure the principle of purchasing value for money is applied, it is optimal for the fulfillment of objectives not to assess compliance with the 3E principles individually, but as a whole (see Figure 1) (Sborník Dobré Praxe-Mpvs.Cz 2020).

The public sector provides services and brings some new possibilities, so it is important to design new input and output factors. Djellal and Gallouj (2008) divide measurement methods into two categories: index-based methods and threshold value methods. Index-based methods are based on pointers (Djellal and Gallouj 2008).

Digitization can be described as an important technological trend that is transforming society, business and the functioning of the state. Although there are many benefits of digitization, it requires investment and associated costs (What Is Digitalization 2018). Digitization is the technical process of transforming analogue information flows into digital data that have discontinuous values, which are based on two separate states (Feldman 2003). Digital technology takes information and decomposes it into its smallest components. By transforming an analog signal into distinct pieces, digitization enables the manipulation of information, text, graphics, software code, sound and video, i.e., its informing, transforming ability (Gorenšek and Kohont 2018; What Is Digitalization 2018). The extent and impact of the digital economy are determined by the ways in which individuals adopt information and communication technologies (Feliciano-Cestero et al. 2023).

As a result of the ongoing globalization and informatization projects, information and communication technologies are gradually being integrated into the processes of e-government. It is increasingly recognized that e-government is moving towards a holistic approach and that governance in the field of sustainable development requires strategic national planning (UN E-Government Survey 2022). The authors Fine and Johnson pointed out that public sector companies face many similar challenges to private sector companies, especially in the area of digitization (Fine and Johnson 2005). Breaugh and colleagues argue that public administration digitization projects require collaborative approaches for successful development and implementation (Breaugh et al. 2023). In the business sphere, measuring the efficiency of processes is essential, as well as measuring customer satisfaction. Therefore, this article focuses on the importance of measuring efficiency and satisfaction in the context of digital transformation within the public sector.

It is also necessary to mention that in the Slovak Republic, there is no regular measurement of satisfaction with new or existing services. The paper fills this gap and suggests a way in which it could be implemented and possibly adjusted. Similarly, this paper fills a gap in measuring the efficiency of data envelopment analysis (DEA analysis), since similar measurements using several available indicators were not identified within the European Union (EU). Overall, this work presents an overview for satisfaction assessment in e-government services, measuring the efficiency of the state of e-Government and providing a basis for future research in this area.

Citizens outside big cities benefit from using e-Government services. E-Government services can be much more useful for residents who live in rural areas (Roy et al. 2015; Seo and Bernsen 2016). Urban residents use the Internet twice as often as rural residents (Local E-Government Development 2022). The benefits of online communication in rural areas may be greater than in urban areas due to transportation costs, time constraints, or traffic congestion (Roy et al. 2015). The usefulness of online services is especially perceived in terms of flexibility, speed, availability, mobility and education. It is important that the content is expressed in a comprehensible and readable form, in addition to the correct organization of information and security of communication, i.e., perceived trust, which ensures more reliable information when using e-Government services. Government information provided by government institutions on websites can generally be considered and perceived as trustworthy; some requests or information may be misunderstood if not provided verbally or personally (Roy et al. 2015).

E-Government is divided into several levels—e-Government 1.0 to 3.0. E-Government services are constantly evolving, becoming more sophisticated and complex and, thus, increasing their costs for operation and development (Štandardy Vlastností Elektronických Služieb Verejnej Správy 2014). E-Government 1.0 focuses on the provision of transactional public administration services for citizens and businesses. E-Government 2.0 is aimed at increasing citizen participation, as well as openness and accountability. E-Government 3.0 is considered as a strategic response to the growing problems and challenges that modern society has to cope with, i.e., the flood of data from the second generation of e-Government. The goal is to provide support in policy making and solving social problems for the well-being of citizens using new technologies (Vrabie 2023; Charalabidis et al. 2019). The National Agency for the Information Society considers the concept of e-Government 3.0 as a set of measures that provides low-cost, high-quality services designed to ensure the satisfaction of citizens. In addition, it adapts to the needs of citizens, promotes entrepreneurship and increases efficiency and better access to information and services. Consequently, it adapts services to citizens using new technologies (Nam 2013). With the arrival of new technologies, new levels of e-Government services can be defined. It is important to ensure that web portals do not comprise technical problems that can negatively affect the opinions of citizens. In addition, it is important to secure functionality at all levels, so that there are no malfunctions, page slowdowns, etc. (Muhammad and Hromada 2023; Paul and Paul 2023). The development of the levels of e-Government is shown in

Table 1. Development stages of e-Government.

	e-Government 1.0	e-Government 2.0	e-Government 3.0
Main intention	Better service	Openness and cooperation	Solving societal problems, ensuring citizen welfare, optimizing resources
Main method	Online public administration	Open and coordinated governance	Intelligent management
Level of use	National	National and local	From local to international
Tools used and service delivery	Web portal (personal visit—one-way communication)	Web portal, social media (two-way communication)	Ubiquitous smart services, smartphones, apps
ICT area	Infrastructure and organization	People and data	Artificial intelligence (AI) technologies and Internet of Things (IoT) infrastructure

.

The efficiency of e-Government is considered by several authors to be one of the most important drivers of progress in e-Government, and the efficiency of electronic public administration includes various disciplinary perspectives (Chan et al. 2008; Purón-Cid 2014). The efficiency of e-Government can also be measured by measuring user satisfaction, process efficiency, security and trust, innovation and adaptability, etc. In addition, it is possible to identify other areas that are closely related to e-Government (Purón-Cid 2014). The improvement of quality in public services is an important issue. The OECD has identified that improving service quality is also a measure of success. In addition, several IT tools were found to improve services, such as online portals, targeted customization, email communication, authentication, corruption reduction, transparency, etc. (West 2004; von Haldenwang 2004; Purón-Cid 2014).

An overview of efficiency measurement models was carried out in the area of e-Government. DEA has proven to be a useful tool for measuring performance, as well as the efficiency of electronic public administration and digital development in various countries, such as China, South Korea, Mexico, Taiwan and the European Union (see

Table 2. DEA models for measuring efficiency in the field of e-Government.

	Authors	Input Data	Output Data	State and Level	Models
e-Government	(Wu and Guo 2015)	GDP index, Internet penetration index, operational maintenance index	Information disclosure index, interaction index, education services index, employment services index	Country Regional	DEA— J-SBM, S-SBM
	(Afonso et al. 2010)	Technical quality of governance, democratic quality of governance, government expenditure	Higher education attainment, child survival rate (HDI), life expectancy at birth, income equality (Gini index), median income, deflation, GDP/capita, employment rate, highway density, share of renewable energy	European Union States	DEA— BCC
	(Luna et al. 2013)	Number of Internet users, number of computer users, number of smartphone users, efficiency of public administration, infrastructure index	Level of information, interactions, transactions, level of integration, citizen participation	Mexico	DEA— CCR, BCC
	(Seo et al. 2018)	IT budget (in 42 central administrations and demand for information systems from 2014 to 2017), number of employees	Number of public services (korea.go.kr), number of open public data (public portal for open data (data.go.kr), Government 3.0 activities (e.g., achievement reports)	States	DEA—CCR, BCC
	(Liu and Tang 2009)	Number of personal computers per hundred inhabitants, ratio of computers relative to the Internet, number of Internet servers per ten thousand inhabitants. share of population on websites, GDP per capita, level of education in the region, level of security of web services.	Information provision, interaction (communication) and information processing on the authorities’ websites	South Korea Resorts	DEA—C2R
	(Hsieh et al. 2013)	Number of computers, number of servers., systems development budgets, budgets for maintenance of systems, payments to IT staff.	Annual revenue of government units, number of completed transactions from e-commerce, number of malicious attacks by outsiders, website capacity, time taken to upload/download information.	China Regional	DEA— CCR, BCC
	(Yalçın 2021)	Capital investment (share of public expenditure), investment in human resources (share of civil servants).	Government websites and services (level of online services and their integration in public administration), impact of new media channels in public administration (Government Weibo, competitiveness index)	China Regional	DEA— BCC a Malmquist index

).

The factors that are most commonly used in the construction of input and output models are the economic factor, the technological factor and the human factor. The main variables include, for example, the level of digital services provided, gross domestic product (GDP), the employment rate in public administration, the information and communication infrastructure variables themselves, etc. A fundamental element in measuring efficiency is data homogeneity, which means that each datum has to be the same within the comparison of DMU units. The main measurement model is DEA, using CCR (Charnes–Cooper–Rhodes) and BCC (Banker–Charnes–Cooper) models. The authors Wu and Guo (2015) consider the measurement of e-Government performance within provincial governments in China to be an important topic and conclude that the authorities are quick to respond to citizens’ demands. Afonso et al. (2010) recommend that government officials focus on improving efficiency in new EU member states and that they meet the EU’s goals for raising living standards. Seo et al. (2018) state that the Korean government’s 3.0 initiative has improved the efficiency of public service delivery (Wu and Guo 2015; Seo et al. 2018).

There are various indices that are relevant and evaluate the levels of digital society and e-Government. These indicators are important in assessing the level of digitization, efficiency and involvement of government digital services institutions in different countries. These indices and assessments are policy-making tools that help countries and regions to identify areas where they can improve their digital level. Individual authors used various international indicators from databases such as Eurostat, The Organisation for Economic Co-operation and Development (OECD), the European Commission, the World Bank, etc., when measuring efficiency.

The United Nations created a database in which data on the development of e-Government in 193 countries of the world are recorded and stored. An important indicator in the development and progress of e-Government is the e-Government Development Index (EGDI). The E-Government development index measures the willingness and ability of certain governments to use information and communication technologies to provide public services. The index can reach values in the range from 0 to 1, where 1 represents the maximum value and 0 the minimum value (E-Government Development Index 2022).

Table 3. Values of the e-Government Development Index (EDGI).

Ranking 2022	Country	EGDI 2022	EGDI 2020	EGDI 2018	Comparison in 2022
					in [%] k 2020	in (%) k 2018	Order to 2018	Numbers to 2018
TOP 10	Denmark	0.9717	0.9758	0.9150	−0.41	5.67	/	57 × 10−3
	Finland	0.9533	0.9452	0.8815	0.81	7.18	+4	72 × 10−3
	J. Republic of Korea	0.9529	0.956	0.9010	−0.31	5.19	/	52 × 10−3
	New Zealand	0.9432	0.9339	0.8806	0.93	6.26	+4	63 × 10−3
	Iceland	0.9410	0.9101	0.8316	3.09	10.94	+14	109 × 10−3
	Sweden	0.9410	0.9365	0.8882	0.45	5.28	/	53 × 10−3
	Australia	0.9405	0.9432	0.9053	−0.27	3.52	−5	35 × 10−3
	Estonia	0.9393	0.9473	0.8486	−0.80	9.07	+8	91 × 10−3
	Netherlands	0.9384	0.9228	0.8757	1.56	6.27	+4	63 × 10−3
	United States of America	0.9151	0.9297	0.8769	−1.46	3.82	+1	38 × 10−3
12.	Singapore	0.9133	0.9762	0.9663	−0.17	3.21	−5	32 × 10−3
20.	Austria	0.8801	0.8914	0.8301	−1.13	5.00	/	50 × 10−3
29.	Latvia	0.8599	0.7798	0.6996	8.01	16.03	+28	160 × 10−3
34.	Poland	0.8437	0.8255	0.7926	0.18	5.11	−1	51 × 10−3
45.	Czech Republic	0.8088	0.8531	0.7084	−0.94	10.04	+9	100 × 10−3
47.	Slovakia	0.8008	0.8135	0.7155	−0.47	8.53	+2	85 × 10−3

presents the results.

The table shows that e-governance is a dynamic process that is influenced to a certain extent by various factors such as policy, the level of investment, implementation and technological development.

The biggest improvement compared to 2018 was achieved by Iceland and Latvia, which means that these countries have recently emphasized the modernization of e-Government. Denmark maintained its leadership compared to 2018, which means stability in the area of e-Government, with constant improvement of services. Even a country with a lower level of economic power can have a high level of e-Government. Other commonly offered online services include applying for government vacancies and business licenses, applying for certificates and paying utility bills.

The evaluation of e-Government is carried out by the European Commission and compares the levels of e-Government in individual countries. The comparison is made in four areas, according to which the indicators for the government are measured (see Figure 2) (E-Government Benchmark 2021). Individual areas are focus on the user, transparency, login and eID and cross-border services. The results achieved by Malta and Estonia indicate that their e-Governments are most focused on users, transparency, technological equipment and services, and they are open to users from other European countries as well. These countries were followed by Luxembourg (87%), Iceland (86%), the Netherlands (85%), Finland (85%), Denmark (84%), Lithuania (83%), Latvia (80%), Norway (79%), Spain (79%) and Portugal (81%).

From Figure 2, it can be concluded that Denmark and Estonia are significant leaders in the digitalization of public administration and that they achieved above-standard results for most indicators. By contrast, Slovakia achieved lower values in selected indicators, or decreases in some areas. Slovakia should pay more attention to building e-Government and target areas such as education, the better participation of citizens in public sector and involvement in collaborations in the area of e-Government, such as with Estonia and Finland.

When carrying out the analysis in Slovakia and abroad, we also focused on the evaluation of the central portals of the public administrations of individual countries. The central portals in various countries are different. Denmark, Estonia and Finland are also leaders in terms of the results achieved within the EU27 (27 states of the European Union). In some areas, Singapore achieves a better or equal level. In Denmark, Finland and Singapore, citizens mainly access e-Government services via smartphones. In Singapore, this adoption of this approach is as high as 83%. In Slovakia, there is still a large share of citizens who do not communicate with institutions electronically, at approximately 30% less than in other countries. From the performed analysis, we identified a gap in research on the measurement of efficiency and satisfaction in the context of digital transformation in Slovakia. Efficiency measurement models within the European Union are focused more on EU digitization and not on EU e-Government, which forms part of digitization. For this reason, we decided to measure and identify efficiency within the European Union using available data and indices.

2. Results

The Results section is divided into two parts. The first part identifies the level of portal usage. The testing of hypotheses and comparisons between urban and rural residents were carried out. Furthermore, the second part is the calculation of the American Customer Satisfaction Index (ACSI). The second part is the results of the DEA measurement.

• Identification of the use of electronic public administration services and measurement of satisfaction with the state portal (Slovensko.sk)—ACSI.
• Identification of the effectiveness of e-Government in Slovakia—DEA.

2.1. The First Part—Primary Research in Slovakia

The rate of the use of public services through the state portal in Slovakia is 74%, and 26% of citizens do not use this portal. Rural portals are used by 88% and other services are used by 87% of the respondents (see

Table 4. Descriptive analysis of satisfaction questions from the primary questionnaire.

	Other Electronic Services		State Portal (Slovensko.sk)		Rural Portal
Rate of use of basic access points in Slovakia number of respondents in %	yes	no	yes	no	yes	no
	87%	13%	74%	26%	88%	12%
Min	1		1		1
Max	10		10		10
Average	6.986		6.621		6.741
Modus	8		8		5
Median	7		7		7
Standard deviation	1.900		1.976		2.355
Peakiness	−0.050		−0.414		−0.392
Skewness	−0.512		−0.325		−0.543
N	368		272		174

). Other services include electronic vignettes, e-prescriptions, ePN, the online land registry, etc. The basic access points include the central state portal, Slovensko.sk, the rural portals of municipalities and other electronic services. The skewness may indicate that respondents are less satisfied or have a different experience, which is reflected in the lower-scale values.

Based on the analysis, hypotheses were established. The hypotheses were tested using the chi-squared test between the city and rural areas. The aim was to find out the difference in use between citizens living in cities and those in rural areas.

The research question (RQ) was as follows: Does the type of permanent residence, i.e., city, influence the central portal for public electronic services or electronic state services?

Hypothesis 0AB (H0A). 

There is no significant difference in the level of use of the central state portal for public electronic services (Slovensko.sk) between citizens living in cities and in rural areas.

Hypothesis 1A (H1A). 

There is a significant difference in the level of use of the central state portal (Slovensko.sk) between citizens living in cities and in rural areas.

Research question 1 (RQ 1): Do you currently live in a city or a rural area?

Research question 2 (RQ 2): How often do you visit the portal (slovensko.sk) (see Figure 3)?

The positive answers that were used to represent a “yes” answer were always, often, sometimes, and occasionally. A negative answer was represented by “no”—we do not use it. If Chí Χ2-test < Chí Χ2-critical, we accepted hypothesis H0B. In this example, based on calculations, with Chí Χ2-test > Chí Χ2-critical, we accept H0B (See

Table 5. Chi-squared test results for cities and municipalities in rural areas.

Observed Data	City		Rural Area	Total
Yes	134		138	272
Well	57		39	96
Total	191		177	N = 368
Expected data	City		Rural area	Total
Yes	141.2		130.8	272
Well	49.8		46.2	96
Total	191		177	N = 368
Indicator	Degree of freedom	Chi X2-test	Chi X2-Critical	Significance level	p-value	Decision rule
Values	2	2.905	5.991	5 × 10−2	8.8 × 10−2	Accepted H0B

).

From the results, it can be concluded that there are no significant differences in the use of public electronic services (slovensko.sk) in between the two areas, nor within regions or between cities and municipalities. Respondents from rural areas use the portal to a greater extent and more frequently than respondents from cities.

2.2. Evaluation of the American Customer Satisfaction Index

The American Customer Satisfaction Index questionnaire contains 15 questions from the Government model. The individual questions are focused on information, process, customer service and websites. The mentioned areas represent perceived quality. Furthermore, there are questions about the customer’s expectations of the service. In addition to these areas, there are questions on overall customer satisfaction, customer complaints and user trust.

The individual questions on the questionnaire use a Likert scale from 1 to 10, where, for example, 1 is a negative attitude—“do not recommend, difficult, etc.”, and 10 is a positive answer—“recommend, easy, etc.” By calculating satisfaction and using the American Customer Satisfaction Index, it is possible to assess the applicability of the consumer satisfaction indicator and the measurement of satisfaction with various services.

The data for determining satisfaction using the American Customer Satisfaction Index were successfully collected (see

Table 6. Descriptive analysis of the results of questions from the American Customer Satisfaction Index.

The Areas of the ACSI		Questions	Average	Standard Deviation	Median	Pointedness	Skewness
Perceived Quality	Information	OT2	6.03	2.11	6	−0.25	−0.22
		OT3	6.03	2.26	6	−0.47	−0.22
	Process	OT4	6.61	1.97	7	−0.38	−0.33
		OT5	6.06	2.09	6	−0.52	−0.15
	Customer service	OT6	7.17	1.95	7	−0.79	−0.19
		OT7	7.07	2.02	7	−0.65	−0.24
	Website	OT8	6.38	2.11	6	−0.65	−0.17
		OT9	6.88	1.87	7	−0.41	-0.20
Customer expectation		OT1	6.83	2.32	7	-0.65	−0.35
		OT12	6.09	2.11	6	-0.46	−0.43
Customer Satisfaction (ACSI)		OT10	7.03	2.06	7	-0.34	−0.46
		OT11	6.62	1.98	7	-0.41	−0.32
Customer complaints		OT13	Almost no complaints
		OT13A
User confidence	Fidelity	OT14	7.04	2.38	7	−0.49	−0.58
	Recommendation	OT15	6.78	2.34	7	−0.28	−0.56

).

In addition, the correlations between individual ACSI areas were determined (see

Table 7. Correlations between ACSI elements.

	Correlations between ACSI Core Areas
	Customer Expectations	Customer Satisfaction	User Confidence	Quality
Customer expectations	1.000
Customer satisfaction	0.676	1.000
User confidence	0.594	0.755	1.000
Quality	0.663	0.855	0.682	1.000
	Correlations between Areas of Perceived Quality
	Information	Trial	Customer Service	Web Page
Information	1.000
Process	0.794	1.000
Customer service	0.489	0.558	1.000
Website	0.696	0.744	0.605	1.000

).

The greatest correlation was achieved between quality, customer satisfaction and user trust and customer satisfaction. The domains of perceived quality, process and information were the most strongly correlated, followed by website and process.

The first step in the calculation was to assign weights to the certain questions and areas. The weights are determined using the entropic method. Subsequently, we multiplied the corresponding weight $v_j$ with the corresponding measurable transformation $M{P}_{ij}$. It is stated that $I_{ij }{=(\mathrm{MP}}_{\mathrm{ij}}{\mathrm{v}}_{\mathrm{j}}•)$. In this way, all the answers from the respondents were multiplied with the corresponding weight for the relevant question. Next the values were added up for one respondent $I_i={{\displaystyle \sum }}_{j=1}^n{I}_{ij}$. Subsequently, this value could be divided by 10, and we obtained partial satisfaction for one respondent. E represents the overall mean value of the American Customer Satisfaction Index for all respondents. Satisfaction scores for individual respondents for all areas (see

Table 8. Satisfaction scores for individual respondents for all areas.

Respondent	Partial Respondent Satisfaction
I1	69.17%
I2	81.60%
I3	43.10%
I272	67.92%
E—total value	1783

).

$$I_i={\displaystyle \frac{{{\displaystyle \sum }}_{j=1}^n{I}_{ij}}{10}}={\displaystyle \frac{{{\displaystyle \sum }}_{j=1}^{1}7.5}{10}}\times 100=70.45\%$$

$$\mathrm{E}={\displaystyle \frac{{{\displaystyle \sum }}_{\mathrm{n}=1}^{\mathrm{N}}{\mathrm{I}}_{\mathrm{i}}}{\mathrm{N}}}={\displaystyle \frac{7.05+8.25+\dots +6.93}{272}}={\displaystyle \frac{1783}{272}}=6.557$$

$$ACSI=\left({\displaystyle \frac{E-MIN}{MAX-MIN}}\right)\times 100=\left({\displaystyle \frac{ 6.557-1 }{10-1}}\right)\times 100=61.74\%$$

This result represents the level of customer satisfaction on a scale from 0 to 100, while the value of 61.7% indicates slightly higher satisfaction with the use of the state portal, Slovensko.sk, and the assumptions of RQ 1–2 are accepted. We proceeded further in terms of satisfaction for individual areas. The results are presented below:

$${\mathrm{I}}_{\mathrm{k}-1}={\displaystyle \frac{{{\displaystyle \sum }}_{\mathrm{n}=1}^{\mathrm{N}}{(\mathrm{R}}_{\mathrm{ij}}\times {\mathrm{v}}_{\mathrm{j}})}{10 \times {{\displaystyle \sum }}_{\mathrm{n}=1}^{\mathrm{N}}{\mathrm{v}}_{\mathrm{j}}}}={\displaystyle \frac{(7 \times 0.084)+(8 \times 0.098)+\dots +(6 \times 0.048)}{10 \times (0.084+0.098+\dots +0.048)}}=0.712$$

$${ \mathrm{I}}_{\mathrm{K}1}={\displaystyle \frac{{{\displaystyle \sum }}_{\mathrm{n}=1}^{\mathrm{N}}{\mathrm{I}}_{\mathrm{k}-\mathrm{n}}}{\mathrm{N}}}={\displaystyle \frac{0.712+\dots +00.615}{272}}=64.2{ \times 10}^{-2}$$

Additional results were obtained for satisfaction for individual areas of perceived quality (64.2%), customer expectation (64.5%), customer satisfaction (68.2%) and user trust (69.1%).

The total calculated satisfaction was at the level of 61.7%. Among the individual areas, user trust achieved the best rating, at the level of 69.1%, and customer expectation reached the lowest value, i.e., slightly above average. It is necessary to update the central state portal for public electronic services and, thus, increase the level of satisfaction of users with e-Government services.

2.3. The Second Part of the DEA Efficiency Measurement

The next part was focused on the DEA method. The following table (see

Table 9. The results measures of e-Government efficiency for the selected period.

	2022		2020		2018		2016		2014		Efficiency (Max 1)
	Y	X	Y	X	Y	X	Y	X	Y	X
Finland	7	0.981	5	0.959	6	0.958	7	0.904	1	1	0.960
Denmark	9	0.955	6	0.950	4	0.989	4	0.986	1	1.177	0.976
Sweden	8	0.970	7	0.928	1	1.036	11	0.832	11	0.983	0.943
Netherlands	1	1.009	3	0.980	8	0.934	10	0.845	14	0.936	0.939
Luxembourg	17	0.828	22	0.706	21	0.724	12	0.811	18	0.844	0.782
Malta	10	0.942	13	0.824	19	0.754	20	0.681	15	0.879	0.816
Estonia	1	1.159	1	1.159	1	1.253	1	1.647	16	0.858	0.972
Ireland	1	1.043	16	0.773	10	0.904	1	1.116	1	1.077	0.936
Spain	18	0.809	20	0.745	16	0.794	15	0.734	21	0.789	0.774
Latvia	1	1.903	1	1.621	1	1.475	1	1.928	1	1.440	1.000
Lithuania	14	0.863	11	0.856	13	0.850	9	0.881	1	1.332	0.890
Austria	13	0.892	8	0.927	11	0.887	13	0.790	17	0.857	0.870
Belgium	16	0.841	18	0.753	20	0.737	19	0.697	20	0.825	0.771
Slovenia	11	0.938	9	0.902	12	0.875	16	0.728	1	1.056	0.889
Portugal	21	0.730	21	0.744	17	0.781	8	0.884	1	1.191	0.828
France	1	1.013	4	0.966	7	0.938	6	0.926	19	0.841	0.934
Germany	25	0.651	17	0.761	22	0.669	23	0.620	23	0.697	0.680
Czech Republic	12	0.902	19	0.745	18	0.777	25	0.528	24	0.675	0.725
Croatia	22	0.709	23	0.673	23	0.660	22	0.631	22	0.737	0.682
Slovakia	19	0.772	15	0.794	15	0.794	14	0.775	1	1.060	0.827
Cyprus	20	0.769	24	0.664	24	0.627	18	0.703	13	0.947	0.742
Hungary	1	1.039	12	0.849	9	0.908	17	0.722	1	1.042	0.896
Italy	26	0.508	26	0.458	26	0.436	26	0.452	26	0.520	0.475
Poland	24	0.672	25	0.603	25	0.611	24	0.555	25	0.671	0.622
Bulgaria	23	0.702	14	0.794	14	0.816	21	0.653	12	0.956	0.784
Greece	15	0.844	10	0.868	5	0.958	5	0.957	1	1.139	0.926
Romania	27	0.417	27	0.269	27	0.213	27	0.231	27	0.358	0.297
efficiency in years of years [max 1]		0.841	-	0.796	-	0.800	-	0.760	-	0.866	-

Where Y—ranking and X—super efficiency. The overall efficiency for individual years reached the highest score in the year 2014, followed by the year 2022. The results can also be seen in the overview in Figure 4.

) shows the output values of the DEA measurement for the years 2014, 2016, 2018, 2020 and 2022.

A comparison of the efficiency levels of the electronic services provided by EU countries is displayed in Figure 4. Western EU countries did not reach efficiency in 2014, but in 2022, two countries managed to achieve efficiency, i.e., France and the Netherlands. Their average score is the second highest for all the areas. The development of most countries is growing, except for Germany and Austria.

Based on the results of the DEA analysis and the obtained efficiency values for 27 EU countries, the countries were divided into four areas. Moreover, the results were verified through hypothesis testing using the Mann–Whitney U test, which is primarily used to verify two independent sets due to low numbers of elements. In addition, we tried to answer the question of whether the difference was statistically significant or just random. The following hypotheses were tested for verification:

Hypothesis 0B (H0B). 

There is no difference between the efficiency levels of e-Government in selected areas of the European Union.

Hypothesis 1B (H1B). 

There is a difference between the efficiency levels of e-Government in selected areas of the European Union.

Through the hypotheses H0C and H1C, we verified whether there were differences in the effectiveness of e-Government among four areas of the European Union. The values for the average effectiveness of e-Government for individual years were used for the testing. In addition, when testing the hypotheses, we chose the northern part of the European Union as a control group, as these countries achieved the highest average value for 2022 and 2014 among the other groups. In this example, the alpha value was 5 × 10⁻². However, the file did not contain more than 25 elements per group, that is, U_crit was taken from the table for the Mann–Whitney U test. Based on the comparison of U_min and U_crit, the hypothesis H0C was accepted or rejected. The Mann–Whitney test results are displayed below (see

Table 10. Mann–Whitney U test results for EU regions 2014–2022.

Areas Tested	Number of Elements	Umin	Ucirt	Sign	Level Significance Level	Decision Rule
Northern EU—Eastern EU	n1 = 7 n2 = 6	1	6	≤	5 × 10−2	H0C reject
Northern EU—Southern EU	n1 = 7 n2 = 6	3	6	≤		H0C reject
Northern EU—Western EU	n1 = 7 n2 = 8	1	10	≤		H0C reject

).

The calculation was carried out using the Excel program. Based on the results of the testing, it was confirmed that there are differences between the effectiveness of e-Government in countries from the northern EU and the countries from the southern, eastern and western EU. Thus, we confirmed H1C and rejected H0C in all the selected combinations.

The results show that there are EU areas that have a higher level of e-Government. In these countries, the interaction between citizens and public administration is more intensive through e-Government services. We can also observe countries with a lower level of economic performance that offer effective e-Government services.

3. Discussion

It is possible to measure the impact of digitization on the economy, society, management or the environment (Maximizing the Impact of Digitization 2022). Within this research, the impact was measured only in the European Union. The results show that the selected indicators confirmed several authors’ conclusions, and that this influence persists and is visible in the EU region (Countries’ Performance in Digitisation 2022). Increasing digitization significantly increases societal well-being in an advanced economy. Analyses carried out on 24 EU countries show that a certain increase in digitization score results in an increase in life quality. In less developed economies, other factors than digitization are critical for quality of life (basic needs take precedence, followed by housing, clothing, water and energy, and then health and, finally, transport and communication). Digitization has an impact on quality of life only when citizens have their basic needs satisfied. Increasing levels of digitization also promote better access to basic services, as measured by the HDI, which tracks global access to health and education, as well as overall living standards. The analysis shows that if countries are more digitized, all these aspects improve. The impact of digitization on indicators of health, education and standards of living is more pronounced in developing economies (Maximizing the Impact of Digitization 2022; Digital Around the World 2022; GSMA Intelligence 2022). Future studies should take into greater consideration the sorting of electrical waste, recycling and returning used resources back into the production cycle (UNEP-UNITAR 2023).

Another interesting result is that the rate of use of the portal is high, but satisfaction levels are different for local and other public administration services, as well as the central portal for public services. These levels are alo to some extent influenced by the difficulty of using the services. The results demonstrate that the respondents show a negative attitude towards the development of e-Government if it is compared with the ideal. Therefore, in the future, it will be necessary to carry out further research to determine the ease of use of individual services in the central state portals of all EU countries using the Index Calculation and Maintenance Methodology (CES).

We answered the following question: Is there a difference in the use of e-Government services between citizens living in cities and in rural areas? (Roy et al. 2015). According to Hypothesis 0B, there is no significant difference in the level of use of the portal Slovensko.sk between citizens living in cities and in rural areas. From the results achieved, it is possible to accept the 0B hypothesis and to state that there is no significant difference between the two groups studied. This result can be perceived positively, as it suggests that citizens use e-Government services even outside urban areas. Moreover, a similar conclusion applies to regions. Roy et al., Seo and Bernsen perceive use by this group as a positive due to transportation costs, time constraints, or traffic congestion (Roy et al. 2015; Seo and Bernsen 2016). However, there are several services that are underutilized, such as mID or app-in-picture. Therefore, it would be advisable to increase awareness of these and other available services for citizens outside urban areas. However, one difference can be perceived in choosing the type of service, since residents in rural areas are in favor of the state portal for public electronic services rather than the Singapore solution, in which an application is used more frequently, but the portal is also available. The respondents see the major advantages of e-Government as saving time, the minimal physical burden and 24/7 availability. The results indicate that among the disadvantages and weaknesses of e-Government, citizens perceive digital literacy and a low level of information. The government should focus on these two aspects by integrating the information portal within Slovensko.sk, as other countries have.

In practice, e-Government should enable citizens to interact with authorities from the comfort of their homes and not force citizens or entrepreneurs to appear in person at an office. This could also reduce costs (the automation of processes could save some human resources) and, last but not least, save the environment (for example, by reducing paper consumption). The prerequisite for the successful implementation of this project is also, above all, the expansion of the Internet to the majority of households and companies in Slovakia. In the case of e-Government in Slovakia, this is already a real and legally expected project. Law no. 275/2006 Coll. about information systems for public administration is about the implementation of this idea in life. The strategy for the informatization of public administration states, among other things, that the benefits at which the strategy, strategic goals and specific steps are aimed are focused on client-oriented public administration (for citizens, entrepreneurs and the rest of the public). The performance of public administration will be possible to implement electronically, using information and communication technologies (through the Internet, mobile phones or other means of communication) and others (Kupka 2008; Marcinčin et al. 2023).

It is possible to measure satisfaction using the American Customer Satisfaction Index within Slovakia, but the results show that it would be necessary to reevaluate the area of complaints, as a relevant number of responses were not obtained in this area. The spplication of the ACSI model and the result show that the overall satisfaction with Slovensko.sk is 61.7%. This contributes to a clearer demonstration that satisfaction with the portal is not sufficient. The portal has been running for more than 10 years, and the state has not been able to finetune it so that citizens are satisfied to a greater extent. A larger group of respondents complained about the reliability and design of the portal. The data contribute to a clearer understanding of the the functioning of ACSI. It is possible to state that the applicability of ACSI is possible to use for several services or solutions within the framework of e-Government in Slovakia. Future research could take into account the modification of ACSI, whether based on the determination of the level of e-Government 3.0 or on the reevaluation and addition of questions to the model (Nam 2013). Since e-Government is also used through mobile applications, such as in Singapore, or the implementation of artificial intelligence in public administration (National Digital Identity 2024; National AI Strategy 2024), if the questionnaire could be used in another country, such as Singapore, it would be appropriate to create a multilevel questionnaire that takes into account the level of service through e-Government in those countries. The results are based on the existing evidence, established by E.Welch, on the recursive interaction between trust and satisfaction: trust leads to satisfaction and vice versa (Welch et al. 2005). The results suggest that citizens who trust the government more are also more likely to be satisfied with e-Government. The results provide a new perspective on the relationship between satisfaction and quality, according to which this relationship is somewhat stronger. As part of quality, it is affected by processes, website and information.

When determining the weights in the ACSI, the entropic method was used, but the primary intention was to also use the Analytic Hierarchy Process (AHP) method. Non-tropical weighing provides us with a more objective view when determining weights and is more suitable for determining the weight of a complex system. Each method has its disadvantages, so it would be advisable to use the AHP method as well, in order to take advantage of the advantages of the methods of objectivity and subjectivity. For this reason, when calculating the complex weight, the proportion of AHP could be set at 50% and the weight of the entropy method at 50% in future research (Yang and He 2022).

Furthermore, the time aspect of the service was not taken into account during the primary research. How likely is it that there would be a difference in satisfaction if the service were available 1 year or 10 years after release, or if it were a newly launched service, or a portal? It is possible to assume that a certain group of users will adopt the service and that there may be a difference between a user who is using the service for the first time and one who has used it several times already. Future research could also address the question of whether there is a difference in ACSI satisfaction between citizens living in a city and in a rural area. In addition, a suitable solution for when to use the service is to use it at a regular time interval (1 per year), with major system changes or minor updates that could affect the customer experience. It would also be possible to compare (Morgeson and Mithas 2009) who measured the levels of satisfaction with central state portals and private sectors. In one study, the authors found that federal portals do not provide the same level of quality as other electronic services (Morgeson and Mithas 2009). When modernizing and updating the portal, Slovensko.sk, it would be appropriate to carry out such a comparison, as it would be possible to find out whether the updated central portal for public services is comparable in terms of quality with the most modern portals in the field of the electricity trade. The ACSI questionnaire can be used at several levels, whether at the local level or with other portals for public services.

Tan et al. (2008) state in their research that electronic public administration is increasingly becoming a well-known part of virtual countries. However, the lack of public trust caused by the novelty and uncertainty of online transactions hinders the widespread adoption of public electronic services. Taking into account the perspective of technology as a social actor with whom the customer communicates and transacts, we propose a research model that emphasizes the key role of e-Government service quality as the main driver of citizens’ beliefs about the trustworthiness of e-government websites, which, in turn, promotes the corresponding adoption of e-Government website services. The findings of this study demonstrate that the quality of e-government websites is important in building citizens’ trust in public electronic services (Tan et al. 2008). The results of the study by Li and Shang (2023) reveal that the use of e-government by citizens revives their trust in government indirectly, through the influence on citizens’ evaluation of the integrity of public administration, as well as its performance and responsiveness based on the experience of using e-Government, while the mediating effects of citizens’ evaluations of the government on the relationship between the use of electronic public administration and public trust are reduced by citizens’ expectations of their government. The findings highlight multifaceted strategies to accelerate the adoption of e-Government. Increasing citizens’ trust in e-Government systems through increased reliability, security and transparency remains essential. At the same time, it is necessary to take initiatives to cultivate digital access, skills and abilities within segments of the population (AbdulKareem and Oladimeji 2024). The quality of electronic services has a significant impact on creating public trust. The quality of electronic services also significantly affects public satisfaction. Public trust directly contributes to public satisfaction with public services, but other unmeasured factors also play a role in shaping public perception and satisfaction (Taufiqurokhman et al. 2024).

DEA is the measure of efficiency in the European Union. It follows from the results of the work that the countries in the northern part of the European Union are more effective in the field of digitization than the others. The resources they spend on building e-Government infrastructure and services effectively correspond to the outputs. Moreover, when measuring the efficiency of the Mann–Whitney U test, we statistically confirmed that there is a difference between the levels of efficiency of e-Government in selected areas of the European Union. In addition, the ACSI results could be added to the model (in model B) if such a measurement were implemented across the European Union, and the two output indicators could be the level of online communication with the public administration and satisfaction (ACSI) with the public administration‘s central state portal or another service. If it were possible to increase satisfaction by “10%”, individual states could take a closer look at this index and, thus, increase the required parameters of perceived quality.

When measuring efficiency, countries that are leaders in the field of efficiency were included. These countries include Estonia, Latvia, Finland and Denmark. The inputs of these countries are spent efficiently in relation to the outputs. Even countries with a low budget can achieve a high level of e-Government if the spending of resources is efficient in relation to their outputs, as in Estonia or Latvia. The cooperation of some countries with others, such as Estonia, Finland and others, can also be considered a key factor (Finland and Estonia Deepen Cross-Border Digital Partnerships 2023).

The Recovery and Resilience Plan is intended to assist selected or all countries in their digital transformation (Recovery and Resilience Facility (Country Pages) 2023). These investments and their results will only be visible in a few years. Therefore, it is necessary to monitor how the efficiency of the use of these financial resources was affected by the results of e-Government, either by the increased interaction between the citizen and the public administration, or by the increased number of online services. Digital technologies increasingly contribute to increasing productivity, efficiency, accessibility and, above all, the overall well-being of the population.

Among the limitations we encountered while writing the research, for example, is the low number of articles using the DEA method in the field of e-Government. Other restrictions may influence the cultural and economic nature of the development of this part of Europe. Education in digital skills among the population (digital culture), economic opportunities and investments are needed for the development of not only e-Government, but also the economic and knowledge availability of these services (Taipale 2013). Other limitations include the limited comparability of data for measuring efficiency across the world and the necessary normalization of data. This would make it possible to compare EU countries with Asian countries. Furthermore, countries do not record investments or resources used in e-Government for a certain period to a uniform standard so that these data can be examined for use in the DEA method. The determination of the weights using the AHP method when determining the ACSI area was also an important issue. There was a problem with finding relevant experts who understand indexes, the ACSI index and the AHP method.

Next, a summary of further suggestions for future research is presented. The use of the ACSI, mainly when introducing new services, is recommended. The creation of several ACSI questionnaires that take into account the levels of given countries or measure satisfaction with the local portals of cities and municipalities is also suggested. Furthermore, the use ACSI for new services, such as “mID” and others, is proposed. To use the analytic hierarchy process (AHP method) and the entropy method when determining the weights, in order to use the advantages of both methods of objectivity and subjectivity, is suggested. The use of the DEA method when comparing local regions in Slovakia is indicated. A use for cities and municipalities that provides and most efficiently manages particular budgets to provide the best services to citizens is suggested. When measuring efficiency, it is also important to take into account education levels in mathematics. Future research could address AI solutions within AI countries.

The European Union is becoming increasingly interconnected, with individual countries’ economies becoming integrated into a unified digital market. This fact should be taken into account in European Union politics, with a focus on the unification of services provided at the level of citizen identification and the creation of shared services that can be utilized by any EU member state. It would be beneficial for individual nations to engage in more collaborative efforts, both within their own borders and with countries outside the European Union.

The practical implications of e-Government, particularly in terms of policy, warrant further discussion to ensure effective implementation and optimization. Moreover, the search for long-term strategic solutions that are viable for a period exceeding five years is of paramount importance. The 3E principle was described in this article. The efficiency principle is the main part of the 3E principle, which is the focus of this paper. The DEA method was employed to ascertain the efficiency of each country. The outcome of the DEA model is directly correlated with e-Government efficiency. The cost-effectiveness of the countries in the DEA model is represented by their respective GDPs. The measurement results assist in identifying which countries utilize resources to produce quality e-Government services. Examples of countries that have achieved high levels of e-Government despite low economic strength (GDP) include Estonia, Hungary and Latvia. The DEA model incorporates a temporal dimension, enabling the identification of cost-effectiveness over time.

The efficacy of the countries in question can be represented by the ACSI score, which can be measured in the European Union. It is important to determine whether the individual results have met the expectations of the population, which should be reflected in their satisfaction (Krejnus et al. 2023; Kumar et al. 2021).

4. Materials and Methods

This article contains both qualitative and quantitative data collection. Within the article, mathematical–statistical methods, such as multicorrelation methods and entropic method of determining weights were used. The mathematical–statistical methods of DEA and the American Customer Satisfaction index (ACSI) were empleyed as the primary analytical tools. The procedure for using DEA is presented below. In the framework of primary research, quantitative method in the form of questionnaire survey was used.

A research gap was found in identifying the efficiency of e-Government within EU countries using various indices. This gap was identified by analyzing articles in an international context; the specific countries are mentioned in Table 2. For measuring efficiency within public administration, or e-Government, the DEA method is used using the CCR model. Subsequently, the primary research, aimed at measuring satisfaction in the selected area, was designed using the American Customer Satisfaction Index, Government model, and other questions.

Within the research, following research questions were posed:

• Is it possible to measure the impact of digitization on the economy, society, management or the environment?
• Is it possible to apply ACSI for central state portal, Slovensko.sk? What will be the overall satisfaction with central state portal?
• Are there indicators that can be used to measure efficiency in the European Union, and if so, can they be used in the measurement?
• What models were used to measure the efficiency of e-Government?
• Is there a difference between the use of e-Government services among citizens living in the city and in the countryside?

In order to address the research questions, it was necessary to employ a number of different methods. Methods of analysis, collection and processing of information, excerption and compilation methods were used. In addition, the CCR efficiency measurement model, which is output-oriented, was employed in the measurement process. Within this method, the query method and chi-squared method were used. The research questions are answered in the Discussion and in the Conclusion (see

Table 11. Research methods used.

American Customer Satisfaction Index—ACSI	Efficiency Measurements of e-Government—DEA
Within the framework of ACSI, the following main methods were used: the inquery method, the calculation of the ACSI index itself, the correlation analysis between elements, descriptive analysis, the entropic method of determining the weights and the chi-squared test.	Within DEA, the following methods were used: output-oriented CCR measurement model, super-efficiency, correlation analysis, descriptive analysis and Mann–Whitney U test.

).

Customer Satisfaction Index

The most widely used customer satisfaction indexes in the world are The American Customer Satisfaction Index—Government and the European Customer Satisfaction Index (The American Customer Satisfaction Index—Government 2024). The ACSI government model is a critical control point for evaluating the success of a government’s projects and online initiatives. It is one of the more complex and representative reflections of citizens’ experiences with government websites. ACSI is used to track user satisfaction with product quality over time. The results can be compared with the results of organizations in both the private and public sectors. The ACSI government model was first used in 1994 (The American Customer Satisfaction Index Science and Methodology 2019). ACSI provides cause-and-effect analysis to better target resources to where improvement will have the greatest impact on product improvement. The government’s ACSI model consists of satisfaction factors, satisfaction outcomes and overall satisfaction. Satisfaction factors are grouped into four broad categories that are used as inputs to measure quality (information, process, customer service and website) on the left, overall satisfaction (ACSI) in the middle and satisfaction outcomes on the right (see Figure 5). As one of the solutions, the ACSI model—Government questionnaire was utilized, and it was subsequently used to evaluate satisfaction with e-Government services aimed at the central state portal of the Slovak Republic.

The process of selecting participants was carried out by random selection among citizens of the Slovak Republic. The questionnaire was created in electronic form using the Google Docs tool. Electronic inquiry took place in the time interval from 19 February 2022 to 21 February 2022. The target group was represented by the citizens of Slovak Republic aged 18 to 65 years, with residence in the Slovak Republic and experience using e-Government services. The criteria for inclusion in the sample of respondents were age (18–65 years), residence (Slovakia) and use of e-Government services. Participants under the age of 18, older than 66, respondents residing outside the Slovak Republic and participants who do not use e-Government services were excluded from the survey. The survey involved 423 respondents. The target sample for filling out the questionnaire was 386, which was calculated on the basis of the base set, which is greater than 100,000. The variability of the base set represents a value of 0.5. The confidence interval is 95% and the maximum accessible margin of error is 5%.

Of the participants, 301 were women and 122 were men. There were 225 respondents with current permanent residence in cities and 198 in rural areas. Of the total number, 162 respondents live in the West Slovakia region (38.3%), 106 respondents (25.1%) in East Slovakia and 155 (36.6%) in Central Slovakia. The largest group of respondents was between 41 and 50 years old, with a 2nd degree of higher education and in employment. The methodology of the article is described in Table 8.

The DEA method ranks among the most important management methods. It allows users to evaluate efficiency based on selected inputs and outputs. DEA was first used in 1978 by Charnes, Cooper and Rhodes as a CCR model. In 1984, Bunker, Charnes and Cooper introduced a variant of BCC that evaluates the efficiency of decision-making units under the assumption of variable returns to scale (Omrani et al. 2020; Charnes et al. 1978).

The basic goal of the DEA method is to compare organizational units, which are also referred to as decision-making units (DMUs). Each decision-making unit uses a certain number of inputs for its activity, and the activity results in certain outputs. Input quantities are those units that are consumed in the given activity, and outputs represent the resulting products. In general, smaller input values and larger output values are preferred. The meaning and purpose of the analysis depends on the selected inputs and outputs in the model. Inputs and outputs should be logically linked, as they are in a production process. In order for inputs and outputs to be chosen correctly, we use correlation analysis. With the help of correlation analysis, we select the relationship between the variables and thus eliminate the variables with very strong and very weak correlation. In addition to a suitable correlation coefficient between the indicators, the inputs and outputs must be adapted to the number of decision-making units (DMUs). The rule used is the sum of the number of inputs and outputs $\le$1/3 or 1/5 of the total number of decision-making units. The CCR model calculates the weights of inputs and outputs, the so-called optimization calculation, so that they are as accurate as possible for the DMU in terms of its efficiency while observing the conditions of the maximum efficiency units of all other units (DEA Modely a Meranie Eko-Efektívnosti 2022).

The model evaluates the electronic public services in a given time period (see Table 9). As in the previous measurement, the maximum number of inputs is calculated based on $m+s<{\displaystyle \frac{n}{5}}$, where n represents the number of EU countries. The model has a maximum number of indicators for measurement, which are divided into 4 input and 2 output indicators. The indicators meet the criterion that their data are available from 2014 to 2022. Each input indicator was chosen on the basis that a larger input value in a given country would be more likely to provide more effective e-Government services. Interactions with public administration online, which is one of the most important indicators, were chosen as the output indicators. The output indicators provide us with a picture of the use of e-Government services for input taxes, which take into account the performance of countries in key areas of the current economy and the level of e-Government (see

Table 12. Input and output DMUs for time model—measurement model B.

Index	Used Attributes from the Index	Index Attribute	Index Direction
Use of the Internet	Human factor (number of Internet users)	Access	-	Time availability of data From 2014 to 2022
EGDI	Technological factor (level of e-Government)	Access	-
GDP per capita	Economic factor	Access	-
GBARD share of total government expenditure	Economic factor (share of R&D expenditure)	Access	-
Submission of completed forms when communicating with the public administration	Human factor (level of communication)	Output	+
Interaction with public administration online	Human factor (e-Government—e-Government user communication)	Output	+

).

The statistical relationship between individual quantities can be determined using the Spearman correlation coefficient. The correlation coefficient r is defined by the following relation:

$$r=1-{\displaystyle \frac{6\times {{\displaystyle \sum }}_{n=1}^N{D}^2}{N\times \left(N^2-1\right)}}$$

(1)

• N = number of elements;
• D = difference between $x_n$ and $y_n$ i.e., in two rows;
• r = correlation coefficient.

The correlation coefficient can take the values ⟨−1; 1⟩. The correlation coefficient takes values from the interval ⟨−1; 1⟩ and expresses the degree of linear correlation between variables. Minus 1 means absolute indirect dependence, 0 means non-existent linear dependence, and 1 means absolute direct dependence between two variables. Within DEA analysis, it is advisable that the correlation coefficient is not higher than 0.8, otherwise the efficiency result may be distorted. The ideal correlation coefficient is between 0.3 and 0.8 (Schober et al. 2018).

A research gap was found in identifying the efficiency of e-Government within the EU countries using various indices. For measuring efficiency within public administration, or e-Government, the DEA method is applied using the CCR model. Subsequently, the primary research, aimed at measuring satisfaction in the selected are, a was designed using the American Customer Satisfaction Index, Government model, and other questions. The correlation between individual indicators was determined for the year 2022. The average value of all correlations for the year 2022 was 0.570, which is in the range <0.3 to 0.8>. Thus, we also confirmed the appropriateness of our inputs and outputs. Correlations between individual elements are presented in

Table 13. Identification of the baseline population of interest.

Year 2022	A	B	C	D	E	F
A	1
B	0.669	1
C	0.446	0.717	1
D	0.654	0.425	0.376	1
E	0.798	0.591	0.311	0.420	1
F	0.714	0.677	0.473	0.442	0.845	1

.

Correlation analysis was performed for all years, but we report only one. The appropriate values of the correlation for individual years were (2020—0.541), (2018—0.596), (2016—0.613), and (2014—0.629).

The methodology and procedure of the paper are presented in

Table 14. Methodology of the article.

Paper	Information	Method	Research Phase
Main data collection	The data were focused on digitization, e-Government, efficiency	Analysis, method determination	I.
Analysis	Books, scholarly articles on digitization, e-Government, DEA models that have been used to measure e-Government within Asia. Key findings from DEA measurement models.	Analysis, method of collecting and processing information, extraction and compilation methods, method of abstraction	II.
User identification and satisfaction measurement using ACSI	Data were obtained from respondents.	Primary results of entropic data measurement method, chi-squared test, multi-correlation method	III.
DEA method	Output-oriented CCR model	-	IV.
	From the analysis, none of the indices that can be used have been filled in. The article was targeted at e-Government, meaning that it was necessary to establish input and output data.	-
	Determination of the number of inputs and outputs, verification of appropriate inputs and outputs.	Multi-correlation method, Mann–Whitney U test
	Determination of input and output values. Descriptive data analysis. Dividing selected countries into regions and EU areas and determination of efficiency in 2014, 2016, 2018, 2020, 2022. Identification of effective and ineffective countries.
Conclusion, Discussion and results	The Conclusion determines the level of satisfaction determined by ACSI within Slovakia and the efficiency levels of countries for individual years.	Synthesis method, deduction method, induction method, generalization method, DEA method, comparisons with other studies	V.

Source: own processing.

.

The article comprises both qualitative and quantitative data collection. In addition, the article uses several methods, such as multi-correlation methods, entropic methods for determining weights, etc. The procedure for using DEA is presented in Figure 6.

The graph presents a diagram showing the calculation procedure for the DEA method. The graph can be divided into several parts.

The diagram begins with the identification of existing efficiency measurement models in the examined area and, subsequently, it presents the selection of appropriate indicators and data, with the help of which the goal of DEA efficiency measurement can be achieved. These indicators and data were used for the calculation from which the efficiency ratings were obtained. If there were insufficient data on the examined issue, it was not possible to carry out such a measurement, or it was necessary to look for other indicators and data. This step is one of the most important, as a wider spectrum of information gives us greater possibilities when combining data during measurement.

After this step, it was necessary to move from A1-A12-A2 to select a suitable DEA model. There are several models, which differ in how they calculate efficiency. Therefore, it is necessary to choose the correct direction of the input- or output-oriented model, to know what the number of decision-making units (DMUs) will be and, based on this, to calculate the maximum number of inputs and outputs. Subsequently, it is important to verify, if necessary, the time point of data availability, whether the data are available for a longer period or only one year, etc. In this step, it is necessary to divide the indicators and data.

Furthermore, it is necessary to test the correctness of inputs and outputs based on correlation analysis. A moderate correlation between indicators and data is most appropriate. If the input and output data have inappropriate correlations, it is necessary to find other indicators and try to repeat the correlation analysis. This is also necessary in the case of a longer period of time. If the data are suitable, we can perform descriptive statistics for the data after the correlation analysis.

Subsequently, it is necessary to calculate the efficiency itself using the selected model, determine the order of efficiency, etc. For measuring efficiency over time, it is suitable, for example, to calculate the average efficiency for a certain period. In our case, it is appropriate to divide the countries into regions, or EU regions. In individual steps, it is possible to divide this diagram into 10 steps. In addition, this diagram also serves as a tool for understanding the DEA procedure (Krejnus et al. 2023).

5. Conclusions

This research was aimed at verifying the use of the American Customer Satisfaction Index of the e-Government model in Slovakia and the use of the efficiency measurement method (DEA). For summary of answers to the research questions, see

Table 15. Answers to the research questions.

Type of Question	Research Question	Short Answer
First Main	Is it possible to apply ACSI for central state portal, Slovensko.sk? What will be the overall satisfaction with central state portal?	Yes, it is possible. Satisfaction is 67.8%.
Second Main	Are there indicators that can be used to measure efficiency in the European Union, and if so, can they be used in the measurement?	Yes, there are several indicators and they can be used in the DEA method.
-	Which models were used to measure the efficiency of e-Government?	The most commonly used models are CCR and BCC.
-	Is there a difference between the use of e-Government services between citizens living in the city and in the countryside?	Within Slovakia, the difference is not statistically significant.

.

Both methods were applied, and their applicability was verified. During primary research, the verification of the use of the ACSI model was carried out. From the results, it can be concluded that it is possible to use this measurement model. The research revealed that the overall results of satisfaction with the central state portal for public services was 61.7% (Slovensko.sk). Furthermore, the results of the model showed that quality is the most important, as it has the greatest impact on customer satisfaction and, subsequently, this satisfaction affects customer trust. Of the qualitative elements of central portal for public services, it is necessary to focus on optimization and improvement of processes. Central portal for public services has several shortcomings, such as outdated design, reliability problems, etc.

In the case of DEA, it was necessary to propose a measurement procedure and then indicators and measurement models were used in the field of e-Government. The conclusions can be drawn from the output that measuring efficiency for the European Union area is possible, and two measurement proposals were introduced, focused on the technological area. This research showed Estonia, Denmark, the Netherlands and Hungary as the effective countries. The above results show that governments should focus on service system connectivity, as well as processes and quality, to achieve utility for users. In addition, of the European Union areas, the western European Union is the most effective compared to the other areas, which was also confirmed by the Mann–Whitney U test. The article showed the applicability of the methods. When measuring DEA, further research would be appropriate to determine the effects between inputs on the resulting efficiency. At ACSI, based on these conclusions, experts could consider e-Government 3.0, reworking questions focusing on the local area of e-Government, or new services.

It is also necessary to mention that within Slovakia, there is no regular measurement of satisfaction with new or existing e-Government services. Thus, the research filled this gap and suggested a way in which it could be implemented and, possibly, adjusted. A similar gap was identified in measuring the efficiency of DEA analysis, in which a similar measurement using several available indicators was not identified within the European Union. Overall, this article provides an understanding for satisfaction assessments, measuring the efficiency of the state of e-Government and a basis for future research in this area. The government should focus on service system connectivity, as well as processes and quality, to achieve utility for users. The combination of the ACSI and DEA methods would provide a more comprehensive view of ACSI areas with the necessity for improvement. The final output model could include the interaction with online public administrations and the ACSI index. If it is necessary to increase the ACSI index, a more detailed examination could be carried out of those elements within the index that do not currently reach a satisfactory level. This would allow for a more focused approach to be taken with regard to quality, customer satisfaction and any areas requiring improvement. With these two indicators, we would determine exactly the levels of satisfaction and interaction. It would be appropriate to measure ACSI in all EU countries in order to compare these results among individual EU member countries. We consider this to be the other limit of this study, but the scope of the research did not allow us to process these data in the study. Future research should therefore focus on the final output model, including the interaction with online public administration and the ACSI index. If it were to turn out that it is necessary to increase the ACSI index, it would be possible to take a closer look at which elements from that index do not reach a high level and, thus, to focus on the areas of quality, satisfaction or complaints. With these two indicators, we would determine exactly the levels of satisfaction and interaction. It would be appropriate to measure ACSI values in all EU countries so that we could compare these results among individual EU member countries.