The Efficiency of National Innovation Systems in Post-Soviet Countries: DEA-Based Approach

Abstract

The efficiency of the national innovation system (NIS) is widely considered to be the most important factor of innovation-based economic growth. Using the wide spectrum of different metrics for measuring the efficiency of NIS, modern studies focus mainly on high-income or upper-middle-income countries, while the effectiveness of the NIS in post-Soviet countries has not been studied enough. The post-socialist transformation has led to different models of economic development in these countries, which can be divided into three groups: a group with developed European institutions, a group with a focus on the European path of development, and, finally, a group of countries with an economic model of “state capitalism”. These models formed the trajectory of innovative development. The main purpose of this study is to compare the performance of NIS in post-Soviet countries and to find out whether differences between development institutions can help explain differences in the performance of NIS. The study applies the DEA methodology and considers NISs as homogeneous economic agents, which transform the same types of inputs (knowledge gained using human and financial resources) into the same types of positive outcomes (innovative products and services). The results of a study conducted on data for the period 2011–2018 show that there is no evidence to support the hypothesis that EU institutions or the type of economic model of the country directly relate to the effectiveness of the NIS. The example of Kazakhstan shows that NIS can be effective, even with strong state intervention in the economy. Taken together, the results of the paper suggest that the structure of R&D expenditures by sources of funding and types of research plays an important role in the formation of effective NIS.

1. Introduction

In the modern world, innovation creates the basis for the competitiveness of the economy on a global scale [1]. Therefore, most of the developed countries of the world build and develop national innovation systems in order to turn the development and launch of innovative products and services on the market from random luck into a streamlined and efficient business process [2]. The efficiency of the national innovation system (NIS) is the most important factor in economic growth and sustainable development [3,4,5]; therefore, a comparative analysis of NIS’ productivity in various countries is an interesting subject to study and can open up new directions for improving and correcting the functional and institutional structure of each of them.

Despite the large differences in the metrics that are used to describe the NIS in different countries [6,7,8,9,10,11,12], in general, NISs are homogeneous economic agents in the sense that they transform the same types of inputs (knowledge gained through the use of human and financial resources) into the same types of positive outcomes (innovative products and services). Therefore, the data envelopment analysis (DEA) methodology can be successfully applied to assess the comparative efficiency of various NISs. The DEA methods, algorithms, and their implementation has been developed for today so well that they allow solving the complex problems of assessing the comparative efficiency of homogeneous economic agents, taking into account many practical nuances, including those that arise in the field of organizing the innovation process [13,14,15,16].

Since the last decades, an increasing number of empirical studies have measured the NISs of different groups of countries/regions and across a set of different indicators of inputs and outputs [2,13,14,15,16,17], including the studies taking into account the multi-stage and dynamic nature of the innovation process. However, most studies focus on high-income or upper-middle-income countries [2], and the effectiveness of the NIS in post-Soviet countries has not been studied enough. The study of the development trajectories of the post-Soviet NIS is of interest from the point of view that in the past, these countries were part of a single state and had the same institutional structure and development model, but in subsequent years, they chose significantly different directions for development. Thus, Latvia, Lithuania and Estonia are currently full members of the European Union (EU) with market economies, Ukraine and Georgia adhere to market models of development, and for countries such as Russia, Kazakhstan, the model of state capitalism is more typical. Such significant differences in the institutional structure of the economy can influence the choice of the NIS model and its productivity.

The main purpose of this study is to compare the efficiency of NISs in post-Soviet countries and examine if the EU institutions contribute to a higher productivity of the NIS.

The main research questions that were posed in this paper can be formulated as follows:

• Do the EU institutions contribute to the higher productivity of the NIS?
• Does the economic model of state capitalism lead to inefficiency of the NIS?

The study applies the DEA methodology and considers NISs as homogeneous economic agents, which transform the same types of inputs (knowledge gained using human and financial resources) into the same types of positive outcomes (innovative products and services). Modeling the efficiency of NIS, we are following the “narrow” definition of NIS offered by Nelson [3], which focuses on R&D institutions and their interaction with firms. The inputs of the model are the number of researchers (including technical staff, per million people), R&D expenditure (% of GDP), and payments for intellectual property (% of GDP). The outputs are high-technology exports (% of manufactured exports and % of GDP) and receipts for the use of intellectual property (% of GDP).

After calculating the efficiency of the NIS in the sense in which it is given by a set of input and output parameters, we check the factors that can potentially influence the efficiency of the NIS. It is mainly the factors that characterize the quality of the institutional environment in each country.

The rest of the paper is organized as follows. Section 2 reviews existing literature on NIS evaluation using the DEA approach. In Section 3, we describe the theoretical model of NIS, the concept of NISs efficiency, which we use in our study, DEA models, and non-parametric correction techniques applied for results verification. Since DEA is a data-driven methodology, Section 4 thoroughly analyzes the data and variables used in this study. Section 5 reports the results of the empirical analysis and their discussion. Section 6 concludes.

2. Literature Review

The notion of national innovation system was introduced in [18,19]. In subsequent studies, this concept was developed to cover all participants of the innovative process at the sectorial, regional and national levels [3,20].

Although there exists a large amount of research papers devoted to the theoretical investigation of processes behind NISs, there has been a limited number of empirical studies of this matter. The cluster analysis is used in order to investigate empirically the structural similarities of NISs. Balzat and Pyka [8] investigated the structure and functioning of NISs in the OECD area. The main result of their study is a categorization of national systems of innovation into different clusters. Godinho et al. [7] worked with a three-level structure of clusters based on 29 indicators that are aggregated along eight major dimensions. Based on such a mapping, they proposed a taxonomy of NISs and performed their ranking. Castellacci and Archibugi [9] explored the cross-country distribution of knowledge in a large sample of developed and developing economies. The results showed the existence of three technology clubs characterized by fairly different levels of innovative capability.

The functioning of a NIS is usually described by the relatively large number of indicators. In order to combine them into a smaller number of dimensions, composite indicators are widely used. The most commonly employed approach to constructing such an indicator is to aggregate original factors using predefined weights, which are independent of the dataset under analysis. Another approach to construct a lower number of dimensions is factor analysis, which implicitly uses the structure of the dataset and the correlations between the indicators. Using such an analysis, Fagerberg and Srholec [10] selected four principal factors and explained their role in developing and sustaining the strong innovation capabilities of NIS. Castellacci and Natera [21] applied panel cointegration analysis that extended cointegration tests to panel data to uncover the dynamics of national innovation systems. Further work [11], by means of the cointegration analysis, explored the global evolution of NISs describing the cross-country distribution of many technological, economic and social factors that broadly contribute to defining NIS.

Maloney [22] questioned the validity of some popular measures of the performance of the NIS and proposed a simple neoclassical model with four factors of production, which include labor, physical capital, human capital, and knowledge. The model implies constant returns to scale.

The DEA approach, which is also grounded in the neoclassical theory, however, is the more frequently used in the literature than other methods for empirical investigation of NIS. Thus, the following discussion focuses on the contributions related to DEA for the NIS evaluation.

Johnes and Johnes [23] proposed to use DEA to measure the research contribution of scientific organizations. The pioneering works of Rousseau and Rousseau [24,25] clearly showed the potential of DEA in the measurement of a country’s research impact and in policy and management contexts. The following studies show the scientific validity of the DEA approach in NIS assessment.

The EU integrates a fairly homogeneous set of countries in terms of innovative opportunities; therefore, many studies have been conducted on the analysis of European innovation systems [26,27,28,29,30]. The fact that there is an established data collection (European Innovation Scoreboard) used by the European Commission also plays a significant role here. Yesilay and Halac [31] performed an assessment of innovation efficiency in emerging Europe and Central Asia (EECA) countries. There is also a sufficient number of international NISs comparisons [32,33,34,35]. Chung [36] proposed to analyze a NIS through regional innovation systems. There is quite a large number of studies that are devoted to this approach [37,38,39,40,41,42].

An interesting approach using DEA-based bilateral comparison was applied in [43] for performance measurement of the NISs in Asia and Europe. In this approach, each country in Asia is evaluated with respect to its counterpart in Europe and vice versa. Such an intercontinental comparison leads to a better discrimination between the two NISs.

The following papers show a variety of models and methods that are used in conjunction with the DEA for the evaluation of innovative systems. Filippetti and Peyrache [44] combined composite indicators and DEA. Cai [45] made a comparative study based on DEA and panel data analysis. Kou et al. [15] applied a multi-period and multi-division approach together with DEA for measuring the efficiencies of NISs in OECD countries. Zabala-Iturriagagoitia et al. [12] used a global Malmquist productivity index to investigate the performance of innovation systems in EU and shifts of the technological frontier. Yang et al. [46] used multi-level frontiers in DEA to rank countries. Multiobjective methods were also used with the DEA to perform innovation efficiency assessment [16,47,48]. In [49,50,51], bias-corrected efficiency scores of NISs were computed using the bootstrap technique. The Tobit model was used in order to explain inefficiency of NISs [34,52] or to measure the influence of environmental factors [53].

Many studies examined the innovation process in two simplified stages: knowledge production, and knowledge commercialization. Thus, multi-stage DEA models, where the outputs of the first stage (intermediate outputs) become inputs for the second one, are widely used in the evaluation of NISs [14,35,47,54,55,56]. Chen and Hung [57] employed the actor–network theory to construct a three-stage R&D model of an innovation system.

Finally, we should mention the review paper of Kotsemir [58], where a comprehensive survey of DEA literature on measuring national innovation systems efficiency was conducted. It is also important to note two valuable papers [42,51], where a broad literature review was focused on the input and output variables used in DEA models for the NISs evaluation.

Using the proposed approach, this paper attempts to provide evidence for the following research questions (RQ).

RQ1: Do the EU institutions contribute to a higher productivity of the NIS?

Since 2000, when the Lisbon strategy was adopted by the European Union, significant resources have been directed to stimulate innovation in the EU. Stimulation of innovative development is carried out both in the form of grants and subsidies (for example, within the Horizon programs), in the form of encouraging the exchange of scientific knowledge, designing of common development strategies, etc. However, the path to an innovative economy is not straight and smooth [59].

Though the idea of the high efficiency of the European NIS in comparison with post-Soviet countries prevails in most literature, some studies show that the Baltic countries, for more than 10 years of their presence as EU members and having access to its resources and innovative infrastructure, have not demonstrated significant results in the development of NIS. Some of these comparative studies are discussed below.

Makkonen [60] examined the performance of NIS in the countries of East Central Europe, the Baltic countries, and Russia after the disintegration of the Soviet Bloc. Based on the principal component analysis (PCA), he came to the conclusion that members of the EU seem to be in a better position compared to non-EU member countries. Alnafrah and Mouselli [61] explored the influence of NIS on the creation of new enterprises in the Baltic state countries. They used ordinary least squares (OLS) regressions for a set of variables extracted from PCA based on indicators collected from the Eurostat database. The results indicate that Baltic states are well on the way to developing their NISs in most aspects of innovation activities. However, some recommendations are proposed to each country for improving its innovative capacities and infrastructure.

Wirkierman et al. [62] compared European NIS based on cluster analysis using data collected from community innovation surveys across several EU countries. The results show that Baltic countries rank well below average with respect to innovation inputs and outputs; for these countries, innovation policies do not seem to lead to relevant firm innovation efforts nor outcomes.

Another study [63] focused on the assessment of 10 countries of the Baltic region, including the North-Western Federal District of the Russian Federation, which is considered part of the Baltic region. As a result, the composite index of the NIS is constructed using 15 indicators structured in five dimensions: human resources, infrastructure, research, innovative milieu, and framework conditions. While Lithuania, Latvia, and Russia have significant structural differences in their NIS, they are characterized by the lowest level of accumulated potential in the Baltic region. Estonia displayed an average level of the composite index of NIS capacity across the Baltic region.

RQ2: Does the economic model of state capitalism lead to inefficiency of the NIS?

Following the definition of state capitalism proposed in [64], we consider it as an economic model where the state owns or controls a large share of the means of production and a system where the state intervenes in the economy to protect and advance the interests of large-scale businesses. As will be shown in more detail below, among the post-Soviet countries, Russia, Kazakhstan and Uzbekistan can be described as countries with a model of state capitalism. The model of state capitalism largely copies the methods and approaches of the socialist “planned” economy, and in this sense, the countries using this model have undergone much less post-socialist transformation.

An increasing number of studies are devoted to the problems of the post-Soviet transformation of the innovation system. Thus, Radosevic [65] investigated post-socialist transformation in Eastern Europe and the former Soviet Union from a neo-Schumpeterian perspective. He argued that the post-socialist transformation of NIS is an open-ended process and that their technological development involves the misalignment of different levels and parts of innovation systems.

Mussagulova [66] explored the impact of the Soviet past on innovation in post-Soviet states. This study examines aspects of innovation policy related to R&D and explores the dependence on the Soviet innovation system, which has shaped the modern R&D landscape in post-Soviet countries. Three post-Soviet countries (Estonia, Ukraine, and Kazakhstan) belonging to different income groups were selected for detailed analysis. The results show that Estonia, to a lesser degree than Ukraine and Kazakhstan, bears the consequences of the Soviet R&D system, which displayed weak innovation linkages between the science and education as well as the private sector.

Alnafrah [51] studied NISs in BRICS economies. The results show that Russia is the most inefficient among BRICS countries in producing scientific and technical knowledge (KPP), knowledge commercializing process (KCP) and total innovation process. Assessing the efficiency of NISs in developing countries was performed by Choi and Zo [35]. According to this study, Belarus is highly efficient both in KPP and KCP, Armenia has the highest efficiency score in knowledge production and a low KCP efficiency score, Georgia is efficient in KCP while it has medium efficiency score of KPP, and Kazakhstan, Ukraine, and Russia show medium KPP efficiency and above average KCP efficiency.

Sharma and Thomas [33] measured the R&D efficiency among 22 countries exhibiting R&D expenditures of more than 0.75% of their GDP. Russia was only one of the post-Soviet countries included in this data set. Each country was evaluated according to four different models. Russia has a high efficiency score in the first two models, and it is efficient in the rest of the two models.

The above-mentioned studies do not give a categorical answer to these RQs. Thus, this study attempts to answer the research questions raised and extends the existing gap in the literature by investigating the NISs of post-Soviet countries.

3. Methodology

3.1. The Model of National Innovation System

The term “national innovation system” was first proposed in the study of Freeman in 1987 [67] for the description of “the network of institutions in public and private sectors whose activities and interactions initiate, import, modify and diffuse new technologies”. Later, this concept was developed by Nelson [3], Metcalfe [5], Dosi et al. [68], Lundvall et al. [69] and many others. While all these scholars emphasize the role of institutions, they differ on what organizations and institutions to include as a component in NIS. So, some focus only on R&D organizations (“narrow” approach) and others consider a much broader socio-economic context, including cultural attitudes, learning processes, and tacit knowledge, generated in the process of user–producer interaction.

Modeling NIS’s efficiency, we are following the “narrow” definition of NIS offered by Nelson [3], which focuses on R&D institutions and their interaction with firms. We suppose that state science and technology policies, the financial system, the education and training system, labor market institutions, and other components of NIS from the “broad” definitions [70] are also included in the model indirectly as intermediate products (Figure 1). This definition allows implementing a traditional black-box measurement of the innovation system’s efficiency in the frame of the DEA model.

Despite the criticism on the possibilities of biased efficiency estimation while neglecting information embedded in intermediate products [15], this model has two clear advantages: (a) focusing on the main result of the innovation process, namely, obtaining economic benefits from the production of competitive innovative products; and (b) having no need to collect large volumes of semi-structured and unstructured data to measure the effectiveness of intermediate processes, such as cultural context, tacit knowledge generation, and state policies. The second advantage became critical when modeling the efficiency of NISs in post-Soviet countries because of the lack of open and reliable statistical data.

Since DEA is a data-driven methodology, it is very important to explain the choice of indicators and their meaning. This influences a lot the entire concept of efficiency. That is why we briefly discuss the indicators in the section, devoted to methodology. It is important to note that the choice of specific numerical indicators for use as the input and output of the DEA model is, unfortunately, significantly limited by the availability of publicly available statistical data sets that are comparable for different countries. Thus, the former countries of the Soviet Union, which are currently members of the EU, have extensive statistics on innovation activity, which are published annually in the European Innovation Scoreboard collections and reflect all areas of the innovation process. Russia, Kazakhstan, Belarus, and Armenia are attempting to unify statistical systems in some areas within the framework of the joint activities of the Customs Union, while, for example, Moldova, Ukraine and Georgia are gravitating toward the global rules of statistical accounting. Therefore, the only reliable source of comparable statistical data is the World Bank database https://data.worldbank.org/indicator (accessed on 10 June 2022) with a quite limited set of statistical indicators that can be selected as inputs and outputs for assessing the effectiveness of the NIS.

Moreover, not all post-Soviet countries have even the minimum necessary set of statistical indicators for analysis. For example, there are no data on the number of researchers in Armenia, Azerbaijan, Belarus, Kyrgyzstan, Turkmenistan and Tajikistan, no data on payments for intellectual property in Armenia and Tajikistan, and no data on high-tech exports and R&D expenditures in Turkmenistan. In addition, the maximum possible observation period according to the available data is only 8 years from 2011 to 2018, while earlier periods are also of significant interest for understanding the development dynamics and differences in the NIS of the post-Soviet countries. Thus, considering the above limitations according to the data in this study, the development of the NIS of Latvia, Lithuania, Estonia, Moldova, Ukraine, Russia, Kazakhstan, and Uzbekistan in the period from 2011 to 2018 was studied on a set of indicators representing the inputs and outputs of the NIS, as indicated in

Table 1. Inputs and outputs of the DEA model for NIS in post-Soviet countries (mean for 2011–2018).

Country	Inputs				Outputs
	Research and Development Expenditure (% of GDP) 1	Researchers in R&D (Per Million People) 2	Technicians in R&D (Per Million People) 3	Charges for the Use of Intellectual Property, Payments (% of GDP) 4	Charges for the Use of Intellectual Property, Receipts (% of GDP) 4	High-Technology Exports (% of Manufactured Exports) 5	High-Technology Exports (% of GDP) 5
Estonia	1.614	3407.747	698.483	0.234	0.062	20.978	9.050
Lithuania	0.938	2906.469	462.476	0.113	0.047	11.738	4.347
Latvia	0.610	1797.627	391.190	0.156	0.029	16.057	4.003
Moldova	0.298	741.751	68.316	0.241	0.060	4.583	0.241
Ukraine	0.615	1087.540	197.825	0.428	0.077	6.291	1.264
Russia	1.055	2997.127	472.413	0.369	0.040	11.931	0.565
Kazakhstan	0.151	667.237	126.473	0.071	0.001	32.212	1.378
Uzbekistan	0.161	508.923	56.341	0.047	0.006	6.291	0.037

Notes: ¹ include both capital and current expenditures in the four main sectors: business enterprise, government, higher education and private non-profit. R&D covers basic research, applied research, and experimental development. ² professionals who conduct research and improve or develop concepts, theories, models techniques instrumentation, software of operational methods. R&D covers basic research, applied research, and experimental development. ³ people who perform scientific and technical tasks involving the application of concepts and operational methods, normally under the supervision of researchers. R&D covers basic research, applied research, and experimental development. ⁴ payments and receipts between residents and nonresidents for the authorized use of proprietary rights (patents, trademarks, etc.) and for the use, through licensing agreements, of produced originals or prototypes and related rights. ⁵ exports of the products with high R&D intensity, such as in aerospace, computers, pharmaceuticals, scientific instruments, and electrical machinery.

.

We see that even the limited number of the countries included in the sample can still be divided into three groups: the first group of countries with developed European institutions (Lativia, Lithuania, and Estonia), then a group with a focus on the European path of development (Ukraine and Moldova), and, finally, a group of countries with economic model of “state capitalism” (Russia, Kazakhstan, and Uzbekistan). Thus, the given sample of countries can be considered representative.

With such a choice of sets of input and output indicators the NIS with a minimum number of researchers (including technical staff), financing R&D and payments for intellectual property, that provides the greatest contribution to the export of manufactured products and GDP, as well as maximum receipts for intellectual property will be considered as efficient. In other words, with a minimum of resources consumed for the production of knowledge and their acquisition in the form of ready-to-use intellectual property objects, the system produces a maximum of a high-tech commercial product that is competitive in the world markets and intellectual property objects.

3.2. Data Envelopment Analysis

The paper uses the non-parametric DEA method to measure countries’ technical efficiency of innovations. Based on Farrell’s [71,72] ideas involving a two-input/one-output model under the constant returns to scale (CRS) assumption, Charnes et al. [73] introduced the first DEA model known as CCR model. Later, Banker et al. [74] extended the CCR model to variable returns to scale (VRS). DEA is used to measure the relative efficiency of units (known as DMUs, short for decision making units) that operate under similar conditions and transform resources (inputs) into outputs. The main advantage of this approach is that the modeling of the relationship between inputs and outputs does not require the assignment of weights. Charnes et al. [73] determined the measure of efficiency in a form of ratio $u^{\mathsf{T}}{Y}_0/v^{\mathsf{T}}{X}_0$, where $v={\left(v_1,\dots ,v_m\right)}^{\mathsf{T}}$ and $u={\left(u_1,\dots ,u_s\right)}^{\mathsf{T}}$ are unknown input and output weights, $X_0$ and $Y_0$ are input and output vectors of DMU${}_0$. By restricting $u^{\mathsf{T}}{Y}_j/v^{\mathsf{T}}{X}_j\le 1$ for each DMU${}_j$, $j=1,2,\dots ,n$ and transforming the fractional model to a linear one, they obtained the following multiplier form of the CCR model:

$$\begin{array}{cc}\hfill E_0=& \sum _{k=1}^r{u}_k{y}_{k0}\hfill \\ \hfill & \sum _{k=1}^r{u}_k{y}_{kj}-\sum _{i=1}^m{v}_i{x}_{ij}\le 0,j=1,2,\dots ,n\hfill \\ \hfill & \sum _{i=1}^m{v}_n{x}_{n0}=1,\hfill \\ \hfill & u_k,v_i\ge \epsilon >0,k=1,2,\dots ,r,i=1,2,\dots ,m\hfill \end{array}$$

(1)

where 0 states for the unit (NIS) under assessment, $X_j={(x_{1j},\dots ,x_{mj})}^{\mathsf{T}}$ and $Y_j={(y_{1j},\dots ,y_{rj})}^{\mathsf{T}}$ are the input and output vectors of DMU${}_j$, the $\epsilon$ is a non-Archimedean infinitesimal, that is used to constraint input and output weights from being zero, $E_0$ is the efficiency score of DMU${}_0$ obtained by solving problem (1). The optimal solution will be such weights for inputs and outputs, in which the ratio of the weighted sum of outputs to the weighted sum of inputs will be maximum for all units. The measure of efficiency (efficiency score) always lies in the range from 0 to 1.

Due to computational reasons, the following dual problem is solved to evaluate the efficiency scores:

$$\begin{array}{cc}\hfill E_0=& min\theta -\epsilon \left(\sum _{i=1}^m{s}_i^{-}+\sum _{k=1}^r{s}_k^{+}\right)\hfill \\ \hfill & \sum _{j=1}^n{X}_j{\lambda }_j+S^{-}=\theta X_0\hfill \\ \hfill & \sum _{j=1}^n{Y}_j{\lambda }_j-S^{+}=Y_0\hfill \\ \hfill & {\lambda }_j\ge 0,j=1,\dots ,n\hfill \\ \hfill & S^{-}={\left(s_1^{-},\dots ,s_m^{-}\right)}^{\mathsf{T}}\ge 0\hfill \\ \hfill & S^{+}={\left(s_1^{+},\dots ,s_r^{+}\right)}^{\mathsf{T}}\ge 0\hfill \end{array}$$

(2)

where $S^{-}$ and $S^{+}$ are the input and output slack vectors, respectively. In this input-oriented model, the possibility of proportional contraction of inputs keeping outputs constant is sought. Solving model (2) may lead to computational inaccuracies that result in misleading solutions due to the choice of $\epsilon$ [75,76]. Hence, model (2) is solved in two stages. At the first stage, model (2) is solved by omitting the slacks by simply putting $\epsilon =0$. After that, in the second stage $\theta$ is replaced by ${\theta }^{*}$, and the sum of the slacks is maximized [77].

In the output-oriented model, the level of output is maximized keeping levels of the inputs constant:

$$\begin{array}{cc}\hfill & max\eta \hfill \\ \hfill & \sum _{j=1}^n{X}_j{\lambda }_j+S^{-}=X_0\hfill \\ \hfill & \sum _{j=1}^n{Y}_j{\lambda }_j-S^{+}=\eta Y_0\hfill \\ \hfill & {\lambda }_j\ge 0,j=1,\dots ,n\hfill \\ \hfill & S^{-}={\left(s_1^{-},\dots ,s_m^{-}\right)}^{\mathsf{T}}\ge 0\hfill \\ \hfill & S^{+}={\left(s_1^{+},\dots ,s_r^{+}\right)}^{\mathsf{T}}\ge 0\hfill \end{array}$$

(3)

In model (3), we do not use infinitesimal $\epsilon$, implying that the model is solved in two stages in the same way as model (2). The efficiency score in model (3) is estimated as $1/{\eta }^{*}$.

This study used an output-oriented approach, as it was assumed, according to the proposed variables, that countries have more capacity to control outputs than inputs. However, from the computational point of view, there is no difference which of the models to solve, since it is established that the efficiency measures for the input and output CCR models coincide [77].

The production possibility set T of CCR model is defined as follows:

$$T=\left\{(X,Y)|\sum _{j=1}^n{X}_j{\lambda }_j\le X,\sum _{j=1}^n{Y}_j{\lambda }_j\ge Y,{\lambda }_j\ge 0,j=1,2,\dots ,n\right\}$$

(4)

Banker, Charnes, and Cooper [74] developed the model of variable returns to scale (VRS) by restricting the sum of $\lambda$-variables equal to one in the set of constraints that determine the production possibility set of the CCR model. However, we use the CCR model in our study because our literature review showed that the output-oriented CCR model is most frequently used for NISs evaluation. The review of Kotsemir [58] also confirms this statement.

Efficient units, whose efficiency score is equal to 1 and all optimal slacks are zero, form the efficient frontier in the input-output space $(X,Y)$. An efficient frontier is part of the boundary of a production possibility set. The units with an efficiency score of less than one can approach the efficiency frontier by the proportional increasing of outputs. The projection ${(X_0-S^{-*},\eta }^{*}{Y}_0+S^{+*})$ of an inefficient unit (NIS) onto the efficient frontier found by solving model (3) represents a target for this unit, i.e., such input and output values at which the unit becomes efficient. The target values of inefficient unit enable one to assess the current performance gap and define the best practice. In order to benchmark such inefficiencies, a reference set is also identified. Based on the solution of model (3), the reference set for the inefficient unit is defined by

$$R_0=\left\{j|{\lambda }_j>0,j=1,2,\dots ,n\right\}$$

(5)

The reference set is useful to see what are the efficient units that force the DMU${}_0$ to be inefficient.

In order to investigate the question of how the efficiency of the NIS of the post-Soviet countries has changed over time, we will use the so-called window method [78,79,80] where each period of time is considered an independent unit. It is compared not only with the rest of the NISs at a given time period, but also with itself at different periods of time. This approach allows avoiding the shift of the efficiency frontier and the bias of estimates, and increases the discriminating power of the method.

3.3. Non-Parametric Correlation

After calculating the efficiency of the NIS in the sense in which it is given by a set of input and output parameters, we check the factors that can potentially influence the efficiency of the NIS. First, these factors characterize the quality of the institutional environment in each country. Since both the NIS efficiency ratio and various institutional environment quality indices most often do not have a normal distribution and/or are measured on ordinal scales, it is convenient to use nonparametric correlation to assess the influence of a particular factor. In this paper, in order to study the dependencies between rank variables, Spearman and Kendall rank correlations were used. The calculations were carried out in the software package STATISTICA 12.1.

4. Data

Since it is important to observe the dynamics of input and output indicators in order to understand the results of the DEA models, this section is devoted to the analysis of changes in all indicators in the countries under study in the period 2011–2018. As it was mentioned earlier, the only reliable source of comparable statistical data for post-Soviet countries is the World Bank database with a quite limited set of statistical indicators. As inputs we consider the research and development expenditure (% of GDP), researchers in R&D (per million people), technicians in R&D (per million people), and charges for the use of intellectual property, payments (% of GDP). The outputs of the model are high-technology exports (% of manufactured exports), high-technology exports (% of GDP), and charges for the use of intellectual property, receipts (% of GDP).

Analyzing the dynamics of the input parameters (Figure 2, Figure 3, Figure 4 and Figure 5), the share of R&D expenditure in the Estonian GDP during the study period decreased quite strongly from 2.28% in 2011 to 1.25% in 2016, followed by a slight increase to 1.4% compared to 2018 (Figure 2). In Latvia, Lithuania and Moldova, this indicator fluctuates slightly, while in Ukraine, it increases at the beginning of the period (up to 0.76%), and after 2013, it constantly decreases noticeably (down to 0.47% in 2018). In Russia, the share of R&D expenditure is constantly growing, except for the last year 2018, while in Kazakhstan and Uzbekistan, it remains almost at the same level.

As for the number of researchers, in Estonia and Lithuania, it slightly increases by the end of the study period, while in Latvia, on the contrary, it slightly decreases (Figure 3). In Ukraine, Moldova, Russia and Uzbekistan, the number of researchers is gradually decreasing. In Kazakhstan, one can note a sharp increase in the number of researchers between 2011 and 2012 (almost 1.6 times). Further, the annual growth in the number of researchers continues until 2015, after which the decline begins.

As for the change in the number of technical staff involved in R&D, it should be noted that the data for Lithuania and Latvia contained many gaps, which were extrapolated using the average method. In most countries, there are not too noticeable fluctuations in the number of technicians. The exceptions are Ukraine, where there is a trend toward a reduction in technical staff, and Kazakhstan, where the number of technicians increases significantly in 2012 and 2013 (annual growth of more than 2.5 times), and then decreases steadily (Figure 4).

In terms of the volume of payments for use and intellectual property, Ukraine is leading among the post-Soviet countries for almost the entire period under study (Figure 5). Only in 2015 and 2016, Russia was slightly ahead of it. The values of this indicator in Moldova grew in two waves: from 2011 to 2014 and then a slight decline in 2015 until the end of the observed period. In Estonia, there is a gradual decrease in the share of payments for intellectual property, and in Lithuania, a gradual increase. In Latvia, this indicator decreases slightly at the beginning of the period, and then, on the contrary, begins to grow almost to the previous values. Kazakhstan has also seen a slight undulating increase in payments for intellectual property. However, in Uzbekistan, the changes in the values of this indicator are most noticeable: at the beginning of the period, the share of payments for intellectual property increases strongly, during the 2014–2016 period it drops to almost zero, and then again restores the values of the peak year of 2012 by the end of the observed period. Such surges in intellectual property payments may be evidence of a very limited number of companies operating under licensing agreements.

Analyzing the dynamics of the indicator of receipts for the use of intellectual property, we note that in almost all countries, it fluctuated quite a lot over the period 2011–2018, and no clear trend can be identified in any country (Figure 6). The only exception is Kazakhstan, where income from the use of intellectual property is practically zero throughout the entire observation period. For Uzbekistan in the 2011–2013 period, there is a slight increase in this indicator, and later it is also almost equal to zero.

In terms of the share of high-tech products in the total volume of manufactured exports, Kazakhstan is the undisputed leader among the post-Soviet countries throughout the entire study period (Figure 7). Its share first grows and reaches a peak in 2015 (43.4%), and then gradually decreases to 32.1% in 2018. In Estonia, following Kazakhstan, the share of high-tech exports in manufactured exports ranges from 17.9% to 23.5% with similar behavior. In Lithuania, Ukraine and Uzbekistan, the indicator values fluctuate slightly, while in Latvia, there is a trend of a steady increase in the share of high-tech products in manufactured exports from 9.5% in 2011 to 20.5% in 2018. In Russia, the share of high-tech exports grows steadily from 2011 to 2015, after which it begins to decline. It is most likely due to the imposition of sanctions against Russia in 2014–2015 [81]. However, the most noticeable decrease in the indicator occurs in Moldova: from 7.4% in 2011 to 2.5% in 2018.

If we consider the share of high-tech exports in the country’s total GDP, then Estonia is the leader in this indicator throughout the entire period, although the indicator tends to decrease (Figure 8). In Latvia and Lithuania, the share of high-tech exports in GDP, on the contrary, grows steadily from 2011 to 2018 and is practically comparable to Estonia’s figures in 2018. In Ukraine, Kazakhstan, and Moldova, there is also a decrease in the share of high-tech exports, but the trend is undulating. In Russia, there is an increase in the indicator in 2011–2016 and a decrease in 2017–2018.

5. Results and Discussion

Our calculations show that of all the studied countries, the highest efficiency of NIS during 2011–2018 was observed in Lithuania, Estonia, and Kazakhstan (Figure 9 and Figure 10).

The efficiency of the NIS of Latvia, Moldova, and Ukraine fluctuated during the study period in the range from 0.5 to 1. The efficiency score of the NIS of Latvia grows from 0.59 in 2011 and reaches its maximum values in 2016 and 2018. The efficiency of the NIS of Moldova is maximal in 2011, 2014 and 2016, decreasing in 2017 and 2018. The efficiency score of Ukrainian NIS grows from 0.57 in 2011 to 1 in 2015, but deteriorates after.

The efficiency of the NIS in Uzbekistan reaches its maximum levels in 2013 and 2014, but by 2018, it decreases to 0.23. The NIS of Russia does not reach maximum efficiency at all during the study period, but slowly grows from 0.25 in 2011 to 0.39 in 2018. Such a low efficiency of the NIS of Russia is most likely due to the large proportion of researchers traditionally engaged in fundamental research and the insufficient development of applied research.

The leaders of our rating are Lithuania, Estonia, and Kazakhstan. Regarding the NIS efficiency of Lithuania and Estonia, this results in good agreement with Makkonen [60] and Alnafrah and Mouselli [61]. However, contrary to expectations, the NIS of Kazakhstan has high efficiency as well. In terms of size, the NISs in Lithuania and Estonia are relatively larger and more developed than in Kazakhstan. Although Lithuania and Estonia are not among global leaders in terms of R&D intensity, they are at the top among the examined countries (Table 1). In Kazakhstan, the share of expenditure on R&D in GDP is on average only 0.15%, which keeps Kazakhstan in the lower part of the UNESCO ranking of countries by R&D spending for the Data for the Sustainable Development Goals https://uis.unesco.org/ (accessed on 10 June 2022).

There are several possible explanations for this outcome. First, let us look at R&D financing by source of funds and types of research. It is well known in the literature that strong spending by the business sector is regarded as an underlying factor for success of national innovation systems [82]. Studies show that government support for R&D is essential, but direct government funding is less effective than indirect support through tax incentives and other R&D policy instruments in many countries [83,84,85].

In Lithuania and Estonia, the business funded R&D share is about 40% on average in 2011–2018; the direct government funding in Estonia is larger than in Lithuania. However, what is notable is that R&D funded from abroad accounts for a rather large share in these countries, particularly in Lithuania (Figure 11). Such a high demand for R&D from foreign companies and funds can ensure the demand for high-tech products from these countries in foreign markets, especially in the EU. At the same time, Latvia is an example of relatively low efficiency of NIS, despite the high financial support from the EU. However, businesses did not show interest in financing R&D in this country. Thus, a balanced structure of R&D financing with a sufficiently high level of financing by the business sector is one of the prerequisites for the effectiveness of the NIS.

In Kazakhstan, the largest share of R&D funding comes from the republican budget (over 50%), which links this country more with Russia than with the other leaders in our ranking. Kazakhstan is among regional leaders according to the Central and South Asia regional ranking of the 2018 Global Innovation Index and the following years. The most striking indicator of the success of innovative development in Kazakhstan is a high share of high-tech exports as a percentage of exports of manufactured goods. On the one hand, manufactures exports accounted for about 13% of merchandise exports in the examined period, as Kazakhstan mainly exports raw materials. Therefore, the share of high-tech export in GDP is rather low (Figure 8), especially compared to Estonia. On the other hand, the country has significantly improved scientific performance and commercial relevance of research since 2008, when the technology commercialization project was launched by the World Bank and implemented together with the Kazakh government. The project lasted until the end of 2015 and cost USD 20.53 million https://projects.worldbank.org/en/projects-operations/project-detail/P090695 (accessed on 10 June 2022). However, researchers emphasize that the innovation system in Kazakhstan has an imitation nature, focusing on borrowing technologies instead of creating new ones [86]. This fact, coupled with the inefficiency of state financing of innovations, raises concerns about the effectiveness of NIS in Kazakhstan in the future, unless certain reforms are carried out.

Russia is the anti-leader of our ranking. It should be noted that the percentage of GERD financed by the government in Russia was between 65% and 69% in the period under review. A considerable part of the government’s budget allocations for R&D in Russia was directed to the research for defense and national security. About 30% of government expenditure on civilian R&D went for basic research. However, in 2011–2018, basic R&D expenditures accounted for only 0.15% of the national GDP. The share of basic research in financing civilian research by the government decreased in Russia from 15% in 2011 to 11.5% in 2018. Estimations based on data from the Russian Treasury https://roskazna.gov.ru/ispolnenie-byudzhetov/federalnyj-byudzhet/ (accessed on 10 June 2022). Many experts believe that fundamental research in Russia is underfunded, and this is one of the main reasons for the inefficiency of Russian NIS https://www.rbc.ru/economics/21/09/2018/5ba3bc4f9a7947172541a5ff (accessed on 10 June 2022).

It should be noted that in Estonia, the average share of expenditures for basic research was 25.4% in 2011–2018, calculated based on the Statistics Estonia Database https://stat.ee/ (accessed on 10 June 2022).

As can be seen from Figure 12, in Estonia, a largest share of R&D spending went to experimental development. According to Eurostat glossary, “Experimental development is systematic work, drawing on existing knowledge gained from research and practical experience that is directed to producing new materials, products and devices; to installing new processes, systems and services; or to improving substantially those already produced or installed”. That is, considerable funds were allocated for supporting knowledge exchange and commercialization in Estonia as well as in Lithuania. At the same time, in Russia and in Ukraine, this share is even greater than in the leading countries. Accordingly, the share of basic research and applied research, especially in Russia is very low. Thus, we again conclude that a balanced structure of R&D spending is preferable for NISs to be effective.

Another possible explanation of obtained results can be given from institutional perspective. It is not only funding that matters for a diverse but effective research ecosystem. Institutions play an important role in promoting innovation and knowledge transfer [88]. In order to examine the role of institutions, we considered several indicators of business environment from the World Bank project Doing Business to find the linkage between the effectiveness of NIS and the quality of institutions which are essential for innovation activity business. In September 2021, World Bank Group management decided to discontinue the Doing Business report. However, historical data are available and are the valuable source of regulations that enhance business activity.

The overall years ranking on the Ease of Doing Business reflects the overall quality of business environment. We calculated the nonparametric correlation coefficients between the mean efficiency score of NISs and the median of the Ease of Doing Business (EDB) ranking as well as the medians of its components, the most related to the NIS efficiency: starting business, protecting minority investors, getting credit, trading across borders, enforcing contracts (

Table 2. Nonparametric correlation between NIS efficiency score and indicators of business environment and legislation.

Indicator	Spearman Correlation	Kendall Correlation
Ease of doing business	−0.81 **	−0.64 **
Protecting minority investors	−0.77 **	−0.55 *
Enforcing contracts	−0.24	−0.29
Starting business	−0.90 ***	−0.79 ***
Getting credit	−0.12	−0.04
Trading across borders	−0.36	−0.36

Note: ***—significance at 1% level, **—at 5% level, *—at 10% level.

).

The rank of EDB strongly negatively correlated with the NIS efficiency score, supporting the hypothesis that the higher the quality of institutions associated with business environment, the higher the NIS performance on average. Among components of EDB score and ranking, starting business and protecting minority investors are the most correlated with the NIS efficiency score. At the same time, regulations of getting credit, enforcing contracts and trading across borders, which we consider a priory associated with the effectiveness of NIS, do not show significant correlation.

The bubble chart (Figure 13) shows the relationship between EDB rank (the median for 2011–2018) and innovation performance (the average NIS efficiency score). The trend line gives an indication of the expected innovation performance according to the quality of institutions. Economies appearing above the trend line are performing better than expected, and those below are performing below expectations. The size of the bubble reflects the level of corruption. We used the Corruption Perceptions Index from Transparency International https://www.transparency.org/ (accessed on 10 June 2022). For clarity, we used the inverse of the index value multiplied by 100, so the larger the bubble, the higher the level of corruption in the state.

From Figure 12, we see that the EU states have the lowest level of corruption; these countries have the most business-friendly regulations. However, Kazakhstan, with its high level of corruption, which is one of the biggest problems in doing business in this country, overperformed Latvia in terms of NIS efficiency. Latvia had one of the lowest innovation indexes in the EU during the examined period and the lowest index in private investments in research and development, despite the considerable support from the EU Regional Policy [89]. Thus, being in the EU and having financial and regulatory support does not guarantee the effectiveness of the NIS.

Looking at the relationship between the inefficiency of the NIS and the economic model of state capitalism, we used the results of Wright [64], namely, the proposed key dimensions of this economic model: state ownership, statism, and threatening vs. non-threatening government.

Figure 14 illustrates that there is no direct linkage between NIS efficiency (the size of the bubble on the scatterplot), the type of state capitalism, and the level of government intervention in the economy. The three leaders in terms of NIS efficiency belong to different categories of state capitalism. Estonia is classified as a market-oriented state with low government threat to business and low state ownership. Lithuania is typified as an interventionist state with a high government threat to business and low state ownership. Kazakhstan is typified as an interventionist entrepreneurial state with a high government threat to business and high state ownership.

Thus, the results of the correlation of NIS efficiency scores and indices characterizing the quality of institutions and the economic model indicate that there is no evidence to support the hypothesis that EU institutions or the type of economic model of the country directly relate to the effectiveness of the NIS. However, the quality of the institutions in the business environment is an important factor in the effectiveness of the NIS. Ease of starting business and the level of protection for minority investors are the most influential factors of NIS efficiency.

6. Conclusions

The main aim of this research is to compare the efficiency of national innovation systems in post-Soviets countries and examine if the EU institutions contribute to a higher productivity of the NIS. This study particularly stands out for applying “narrow” definition of NIS offered by Nelson [3], which focuses on R&D institutions and their interaction with firms and considering other components of NIS from the “broad” definitions [70] as intermediate products. This definition allows to implement traditional a black-box measurement of innovation system’s efficiency in the frame of DEA model.

The evidence from this study highlights that post-Soviet countries demonstrate the various levels of efficiency of national innovation systems. The highest efficiency of NIS during 2011–2018 observed in Lithuania, Estonia, and Kazakhstan. The efficiency of the NIS of Latvia, Moldova, and Ukraine fluctuates during the study period in the up-middle range. The efficiency of the NIS in Uzbekistan reaches its maximum levels in 2013 and 2014, but by 2018, it decreases to 0.23. The NIS of Russia is the lowest, does not reach maximum efficiency at all during the study period but is slowly growing. Therefore, there is no evidence to support the hypothesis that EU institutions (RQ1) or the type of economic model of the country (RQ2) directly relate to the effectiveness of the NIS. The example of Kazakhstan shows that even with strong state intervention in the economy, the NIS can be effective, at least within the framework of the concept of efficiency that is used in our study. These findings add to a growing body of literature on the efficiency of NISs in developing countries and have important managerial implications.

Our analysis provides further evidence that the structure of R&D expenditures by sources of funding and types of research plays an important role in the formation of effective NIS. Countries with more efficient NISs have a more balanced structure of R&D expenditures, both in terms of funding sources and types of research. These findings are important for policy makers, as they clearly show that if a state substitutes business in R&D financing, support shifts toward applied research. Underfunding fundamental research leads in the long term to a decrease in the quality and efficiency of NIS.

The results of non-parametric correlation suggest that Ease of Doing Business contributes to the commercialization of research and increases the return on investment in R&D and therefore influence the efficiency of NIS. The ease of starting a business and the protection of minority investors are most related to the effectiveness of the NIS. This is consistent with many studies on the important role of SMEs in shaping innovation ecosystems. Our results give additional evidence in support of this hypothesis.

Considerable progress has been made in developing DEA-applications for measuring efficiency of complex systems. The proposed DEA-model despite its simplicity has two clear advantages: (a) focusing on the main result of the innovation process, namely, obtaining economic benefits from the production of competitive innovative products; and (b) having no need to collect large volumes of semi-structured and unstructured data to direct measurement of effectiveness of state policies. Since the correlation analysis shows the relationship between EDB rank and DEA efficiency scores, we may conclude that the DEA model, with constant returns to scale assumption, is suitable for NIS analysis of post-Soviet countries.

Our work clearly has some limitations. The most important limitation lies in restricted data availability. For this reason, countries such as Armenia, Azerbaijan, Belarus, Georgia, and a few others were excluded from the study. As is known, the results of DEA analysis are significantly sensitive to the sample size and therefore may change when the above countries are included in the sample. Despite this, we believe our work could be a starting point for more thorough investigation of the changes in the effectiveness of NIS in post-Soviet countries. Future work will concentrate on calculating comparable indicators in the area of science and technology in the rest of post-Soviet countries. Furthermore, this analysis could be extended to compare results obtained with other empirical methods, including other DEA models.