R&D in Europe: Sector Decomposition of Sources of (in)Efficiency

Abstract

Measuring the efficiency of research and development (R&D) expenditure and innovation policy has gained attention in recent years. This research examines the efficiency of 29 selected European countries for the period ranging from 2007 to 2017 in achieving and obtaining R&D goals. The methodology applied is the data envelopment analysis approach with the inclusion of the missing data approach. The contributions of this research include the following: dynamic analysis is conducted to track changes of (in)efficiencies over time; the decomposition of the efficiency is done by separating the main variables of interest into the private, higher education, and government sectors; and the robustness of the results is evaluated, which is often ignored in the literature. The results of the analysis are discussed with possible directions for inefficient countries. The rankings provided in the empirical part of the study confirm previous findings on disparities between the European countries with respect to innovation and the R&D sector.

1. Introduction

Sustainable economic growth is heavily dependent on innovation and research and development (R&D) as primary driving forces [1,2,3,4]. Knowledge-based economies are those that are most successful in achieving sustainable growth and improving economic welfare [5]. The last couple of years have seen an increasing interest in evaluating the performance of industries, regions, countries, and other relevant economic levels. The topic of innovation and its effects on increasing total factor productivity have been researched for several decades now [6,7,8]. Many different world organizations have introduced indices that measure the achievements at all stages of R&D, recognizing the importance of innovation and R&D in achieving sustainable economic growth (according to the European Innovation Scoreboard by the European Commission, Global Innovation Index by Cornell University, INSEAD (INStitut Européen d’ADministration des affaires (European Institute of Business Administration)), and the World Intellectual Property Organization).

The European Commission recognized these topics by launching the Green Paper on Innovation in 1995, Lisbon Strategy in 2000, the Europe 2020 strategy in 2010, as well as Regional Policy Contributing to Smart Growth in Europe, along with other relevant documents and strategies (for details please see [9]). Despite this, the desired level of progress has not been achieved by the European Union members, as the Europe 2020 target of R&D representing 3% of GDP has not been met by many countries [10]. The most recent plan and document is the Multiannual Financial Framework for 2021–2027 [11], where a strong focus is made on innovation. This means that an adequate and objective assessment of causes of this problem is needed. The interest in researching this topic has risen in the last couple of years, which can be seen in the growth of the number of research papers within this area of research. This is especially true for measurement of achievements of R&D, innovation, spillovers of knowledge, their effects on the economic growth, sustainability of the economic system, and other topics. There are some criticisms of the traditional indicators of such issues, as [12] state that such indicators offer partial views of the innovation process in a country. Since sustainable development requires (technological) innovation in order to succeed [13], it is important to obtain objective results in measuring the achievements in the area of innovation and R&D on the country, regional, or other levels important for the decision-makers, policy-makers, and other relevant economic subjects. The common approach of measuring the relative efficiency of innovation and R&D is the data envelopment Analysis (DEA). This approach has advantages in analysis of issues that fall in the domain of public sector activities [14], as well as advantages over other methodological approaches, such as the stochastic frontier analysis (SFA) and panel regression [15,16]. DEA is a non-parametric approach comparing the relative efficiency of the so-called decision-making units (alternatives that need to be compared and ranked one to another). Although the literature utilizing the DEA (and related approaches) within this area of research is constantly growing (please see the second section for a review), some gaps still exist. Some of the research shows rather peculiar results in terms of finding that those countries that have the most problems within the R&D sector are more efficient than those that are better-ranked via some official ranking systems. This means that a step within the modelling process is faulty. Furthermore, static analysis is often conducted. This means that a research either observes only one year in the empirical part or calculates the averages of several years. Such an approach could lead to misleading results. It is important to observe if changes are happening over time, and if so then in which direction. Additionally, the research utilizes variables that are aggregated at a country level. However, it is important to distinguish which sector contributes to the problems or improvements in the area of innovation and R&D. By using country-level data alone, a detailed analysis cannot be made. As [17] state that one of the important factors that can stimulate innovation is public financing, there exists a need to compare the effectiveness of public financing of such activities. This is why decomposition of total investment and research resources within the area of R&D needs to happen.

From an economic standpoint, science and technology driven by innovation should reduce the sense of social and economic stagnation in some countries [18,19,20] that have the ability to acquire and use new knowledge as a primary element of their national wealth [21]. These importance points are recognized in the literature. Thus, obtaining fast and reliable information on where a nation stands in respect to new knowledge, innovation, R&D, and related issues is of great interest to all those included in achieving a better future. Furthermore, the decomposition of R&D expenditure and employment should be done in a formal analysis. In this regard, [22,23,24] discuss in detail the issues of using total R&D expenditure in research due to business R&D (BERD) being heavily affected by the industrial structure of each country. Furthermore, obtaining information from sector disaggregation is important from a private research (private business sector R&D) perspective. This is due to preserving the industrial base in Europe and its competitiveness with private R&D, and the spillovers of private R&Ds to higher education and greater adoption of new technologies [25]. Other studies [26,27] emphasize three reasons why measuring the progress of innovation should be important. First, theoretical analysis and development of innovation theories require innovation assessment. Furthermore, it is important for the development and implementation of public policies. Finally, results obtained from innovation assessment are used in the development of firms’ regions’, or countries’ strategies. Thus, this research, although of a more technical nature, is helpful from an economic point of view, as some previous gaps within this area of research are filled and this analysis can provide additional knowledge and insights for the empirical assessment of an economy’s R&D efficiency. Furthermore, increasing knowledge development and technical change are found to be major sources of productivity and new job creation [28,29,30], which will contribute to the notion of sustainable economic growth. In order to facilitate this, the empirical assessment part should be correct and insightful. This is why the goal of this research is to objectively evaluate the (in)efficiency of selected European countries for the period from 2007 to 2017 based on relevant variables. Efficiency in terms of the definition of the DEA model is the main idea. From an economic point of view, as well as within the DEA methodology, the idea of efficiency in economics is to obtain as many outputs as possible based on as few inputs as possible. In ([31], p. 676) the economic efficiency is defined as “The term efficiency as commonly used generally refers to the ratio between the inputs employed and the outputs realized. More precisely, it refers to the maximization of output produced by a unit of input. If more output can be produced per unit of input, the efficiency increases.” And in ([32], p. 9): “To economists, efficiency is a relationship between ends and means. When we call a situation inefficient, we are claiming that we could achieve the desired ends with less means, or that the means employed could produce more of the ends desired”.

The novelty includes observing not only a dynamic approach so that changes can be observed over time, but also a decomposition of R&D expenditures and research staff variables will be made so that three sectors can be simultaneously observed: the higher education, government, and private sectors. Furthermore, the robustness of the results will be examined so that the findings will be reliable for use in future research as well. Finally, previous research often analyzes more developed countries (such as Western Europe, alongside OECD economies), while Eastern and Southern European countries are not often present in existing studies. This research includes those countries in the analysis as well, since those economies have more problems with innovation and R&D policies compared to the more developed European countries. In this way, detailed results can be obtained so that the policymakers for those countries can benefit from such an analysis.

The rest of the paper is structured as follows. The second section gives an overview of related research within this area. A methodology description is given in the third section, with empirical results analyzed in the fourth section. The fifth and final section concludes the research with recommendations for policy-makers and future work.

2. Related Literature Overview

Although related research has been growing in the last couple of years, there are several approaches to analyzing innovation and R&D. The approach that is mostly related to this research will be analyzed in more detail in the next paragraphs. Some other approaches not strictly related to this paper are briefly mentioned as follows. Regional analysis has become popular in recent years, since innovation clusters exist within some regions. Such papers include [5], where 271 NUTS-2 (Nomenclature of Units for Territorial Statistics) regions in the EU-27 (European Union) were compared based on average values of included variables for the period 2005–2010. The DEA modelling approach was utilized in this study. Another study [33] included 192 European Union regions in their analysis. DEA and structural modelling were combined, where the DEA efficiency scores were used in the second step of the structural modelling (which is sometimes criticized in the literature; please refer to [34]). In another study, 178 NUTS-2 regions (for Western Europe) were compared using a mixed spatial autoregressive model and difference in differences (DID) approach (years included were 2007, 2009, and 2011) [35]. The results indicated that an innovation intensity divergence exists between the examined regions. Other methodological approaches were utilized as well. Panel regression techniques are also popular within this area of research. One study [36] examined 11 European Union countries over the period 1991–2005 to estimate the patent intensity based on characteristics such as public expenditures on education per capita, institutional milieu promoting the development and application of new technology, the Ginarte–Park index of intellectual property protection, R&D expenditure per 1000 employees, value-added in industry as % of GDP, sum of exports and imports as % of GDP, and stock market value as % of GDP. Results were not surprising—Sweden, Finland, the Netherlands, and Germany were found to have the highest R&D efficiency. Other regression applications within these topics can be found in [37].

DEA applications can be found in the following papers. One study [38] observed 28 European Union member states in the year 2018. The authors used the following variables in the study: new doctorate graduates per 1000 people, lifelong learning (population aged 25–64 involved in education), degree to which individuals pursue entrepreneurial activities, public sector R&D expenditures, venture capital investments and private sector R&D expenditure per GDP, non R&D expenditure, employment in knowledge-intensive activities, and sales of new-to-market and new-to-firm innovations. The results indicated that half of the countries were DEA efficient, with strange outlier results, such as Romania being efficient but the UK being inefficient, despite the characteristics of input and output data for both countries. Such results could be due to using many variables in the analysis and the number of decision-making units (DMUs) barely exceeding one condition of the number of inputs and outputs in the analysis and not exceeding two other conditions. Another study [39] combined the K-means clustering with a DEA model for 30 provinces in China over the period 2016–2018. The number of researchers, R&D technical service institutions, total R&D funds, market share of leading products, and other factors were used as the main variables in the study. The authors first used the DEA approach to obtain efficiency scores and the second part of the analysis included the clustering method to obtain information on the (dis)similarities between provinces based on their purely technical and scale efficiencies. Another study [40] compared 44 countries via stochastic frontier analysis (SFA) and panel regression techniques for the period 1966–2003 to assess the effects of international R&D spillover on innovation efficiency. Scientific papers, patent cooperation treaty (PCT) patents, and other variables that consider international spillover ideas were included in the study. The authors found positive international R&D spillover in the production of scientific papers. One author [41] observed 28 European Union countries in three years (2011–2013) via categorical data combined with DEA. Many inputs and outputs were utilized, such as the categorical data from countries that were classed as frontier, secondary emerging, advanced emerging, or developed countries. Additionally, the total number of publications, the number of patents granted by the EPO (European Patent Office) and by the U.S. Patent and Trademark Office, the total number of full-time equivalent R&D personnel, the number of postgraduates employed in science and technology (measured in thousands), and the amount of R&D expenditure (in Euro per habitant) by the government, business, education, and employment sectors in high and medium-high technology manufacturing were explored. Thus, somewhat peculiar results were obtained, showing that some countries (such as Bulgaria, Croatia, and Romania) were found to be just as efficient as the best-performing countries according to some of the international rankings. The reasoning could lie in the fact that the number of countries did not satisfy the aforementioned conditions regarding the number of inputs and outputs in the analysis.

One study [42] included the more developed European countries alongside Japan, the United States, Australia, and Canada for the year 1993. The study was rather simple, with several input and output variables included (GDP, active population, R&D expenditure, publications and patents per country). Germany, the Netherlands, and Austria were the top performers within Europe. One landmark study [43] is the only paper (to the knowledge of the author) that focuses on Central and Eastern European (CEE) countries concerning innovation efficiency. The observed period was 2008–2015, with the variables including annual public and private spending on innovation; scientific and technical journal articles; human resources in science and technology; and numbers of graduates in tertiary education, science, math, computing, engineering, manufacturing, and construction. Based on the chosen set of variables, Romania and Slovakia were found to be the closest to the DEA efficient frontier classification.

By observing previous literature mentioned in this section, it can be seen that research mostly focuses on static analysis, with no robustness checks for results (either via different methodologies or by comparing the results to the international ranking systems). Furthermore, the expenditures on R&D and employment in the R&D sector are used as aggregated variables. However, data on these variables is available on a per-sector basis for higher education, government, and private business sectors. By performing the analysis with disaggregated data, the results can indicate which sector is more or less successful compared to others in achieving efficiency in the area of R&D. This will be done in the empirical section of the paper, with the methodology described in the next section.

3. Methodology Description

3.1. Data Envelopment Analysis

In order to compare the relative R&D efficiencies of European countries in the empirical part of the research, a data envelopment analysis (DEA) will be employed. The DEA represents a set of models and methods of mathematical modelling, with the main aim of comparing the relative efficiencies of the so-called decision-making units (DMUs). This method sits in the field of operations research, as it is a discipline that uses mathematical modelling to aid the decision-making process. The ranking system for the DMUs is constructed based on the idea that all DMUs use inputs in order to produce outputs, with the most efficient DMU being the one which can produce maximal values of outputs with minimal needed inputs. Details on the terminology and basic ideas, assumptions, and introduction can be found in [44,45,46,47,48,49,50].

The notation within this methodology is as follows. Each DMU j uses m inputs (x_1j, x_2j, ..., x_mj) for production of s outputs (y_1j, y_2j, ..., y_sj), j ∈ {1, 2, ..., n}. The matrix notation of all of the collected data is as follows: X and Y consist of all inputs and outputs, respectively, where X ∈ M_mn and Y ∈ M_sn. Each row in both matrices represents the DMU j, with ${\mathit{x}}_{\mathrm{o}}=\left(x_{1\mathrm{o}},x_{2\mathrm{o}},...,x_{m\mathrm{o}}\right)’$ and ${\mathit{y}}_{\mathrm{o}}=\left(y_{1\mathrm{o}},y_{2\mathrm{o}},...,y_{s\mathrm{o}}\right)’$, and x_o ≥ 0, x_o ≠ 0 and y_o ≥ 0, y_o ≠ 0. The slacks-based measure (SBM) model was developed in [34,35] and is the approach used in this study. The reason for this is the fact that it has advantages when compared to the most used models, namely the Charnes–Cooper–Rhodes (CCR) and Banker–Charnes–Cooper (BCC) models; the SBM is an additive model with unit invariance properties and the possibility of being non-oriented and non-radial. In other words, the optimization process does not depend on the equally increasing or decreasing output and input values of the DMU under consideration when it is being evaluated; the results are not sensitive to data translation (it is a dimensionally free model) [51]. The main SBM model is as follows:

$$\begin{array}{cc}\underset{\mathit{\lambda },{\mathit{s}}^{-},{\mathit{s}}^{+}}{\min }\hfill & \rho =\frac{1-\frac{1}{m}{\displaystyle \sum _{i=1}^m\frac{s_i^{-}}{x_{io}}}}{1+\frac{1}{s}{\displaystyle \sum _{i=1}^s\frac{s_r^{+}}{y_{ro}}}}\hfill \\ \mathrm{s.t.}\hfill & {\mathit{x}}_o=\mathit{X}\mathit{\lambda }+{\mathit{s}}^{-}\hfill \\ & {\mathit{y}}_o=\mathit{Y}\mathit{\lambda }-{\mathit{s}}^{+}\hfill \\ & \mathit{\lambda }\ge \mathbf{0},{\mathit{s}}^{-}\ge \mathbf{0},{\mathit{s}}^{+}\ge \mathbf{0}\hfill \end{array}$$

(1)

where s⁻ and s⁺ denote vectors of input and output slacks, respectively. The DMU under consideration is SBM-efficient iff $\rho$^* = 1 (i.e., if s⁻ = 0 and s⁺ = 0, no input excess and no output shortfalls exist). Inefficient DMUs can be projected to the efficient frontier in order to calculate the rates of input reduction or output increase needed for it to become efficient:

$$\begin{array}{l}{\widehat{\mathit{x}}}_o\Leftarrow {\mathit{x}}_o-{\mathit{s}}^{*-}\\ {\widehat{\mathit{y}}}_o\Leftarrow {\mathit{y}}_o+{\mathit{s}}^{*+}\end{array}$$

(2)

Relations in Equation (2) explain how much a DMU needs to reduce its inputs and increase its outputs in order to become efficient in terms of the definition of SBM efficiency.

Finally, often the DEA approach to evaluating efficiency has problems with missing data. One of the approaches includes deleting the DMUs from samples that have missing data. This could result in an insufficient number of DMUs in the analysis. Furthermore, if a specific set of countries needs to be evaluated and all of them are included in the sample, this approach is not the best one. Another approach is given in [52], in which it is advised to include penalties for missing data. For those missing values that refer to the input values, the researcher should put in values greater than the maximum available data value. The opposite is true for missing output values. As [52] has shown, doing this in the analysis will yield efficient values that are just as correct as those acquired from the approach involving deleting DMUs with missing data. However, the researcher will be able to compare those DMUs which would not be included in the analysis if the first approach for dealing with missing data was used. This research follows the approach in [52].

3.2. Multiple Criteria Decision Making (MCDM)

The robustness of the results will be evaluated via another branch of operations research, employing a MCDM model. The set of models within the MCDM methodology consists of many approaches that have the goal of obtaining a ranking system based on different criteria, on which the alternatives are compared one to another. When making decisions in real life (for economics, finance, etc.), the decision maker often has to compare alternatives based on different, often conflicting criteria. MCDM models have been developed to aid in making such decisions. Details on this methodology can be found in [53], which is followed for the basic terminology and ideas in this research. When the problem and objectives of a study are outlined, the alternatives have to be identified. Based on these alternatives, a mathematical model is formulated and the problem is solved. The results are presented to the decision maker, who either makes a final decision based on the results or some of the steps in the process are redefined. As a basic MCDM model, this study uses the multi objective optimization by ratio analysis (MOORA) model, as it is robust in respect to the specific criteria observed in [54]. Alternatives that need to be ranked are observed and their criteria are measured in order to be used in the ranking process. The data is collected on m alternatives with n criteria, x_ij, i ∈ {1, 2, ..., n}, j ∈ {1, 2, ..., m}, and each value is normalized as follows:

$$x_{ij}^{*}=\frac{x_{ij}}{\sqrt{{\displaystyle \sum _{j=1}^m{x}_{ij}^2}}}$$

(3)

which means that every value $x_{ij}^{*}$ will be in the range [0, 1]. If the value of the criterion should be the greatest possible, values in (3) are summarized, whereas the criterion which should be the smallest possible is detracted from the total sum:

$$y_j^{*}={\displaystyle \sum _{i=1}^g{x}_{ij}^{*}}-{\displaystyle \sum _{i=g+1}^n{x}_{ij}^{*}}$$

(4)

Now, $y_j^{*}$ represents the values of every alternative based on the normalized values of all criteria, and $y_j^{*}$ is used for the ranking process. The greater the value of $y_j^{*}$ is, the alternative will obtain a better rank. It is assumed in (4) that all of the criteria have the same weights. However, if the decision-maker has arguments why some criteria should have a greater weight compared to others, (4) can be modified as well.

4. Empirical Results

4.1. Main Results

For the empirical comparison of the efficiencies of selected European countries, the following data was collected from the European Commission’s Eurostat [55] and OECD [56] (The Organisation for Economic Co-operation and Development) databases. Yearly data for the period 2007–2017 was available for the following countries: Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, and the United Kingdom. The following variables were chosen based on a review of previous literature, as well as data availability for: intramural R&D expenditure in Euro per capita in the higher education sector, business enterprise sector, and government sector; total R&D personnel and researchers as a percentage of total employment in full-time equivalent work in the higher education sector, business enterprise sector, and government sector; number of scientific journal ranking (SJR) publications per million habitants; patent applications to the European patent office (EPO) per million habitants; percentage of high-tech exports out of the total exports in the higher education sector, business enterprise sector, and government sector. All of the variables were observed as outputs, with the exception of R&D expenditures, for which reciprocal values were observed so that input variables could be used in the analysis as well (the DEA approach needs both inputs and outputs for the optimization process). Previously mentioned research obtained results showing that some countries that are among the worst performers according to international institution rankings, such as the European Innovation Scoreboard classification, were ranked as the best performers in the analysis. The problem with those studies is that authors some of the variables are inputs. However, those variables should be observed as outputs. Please see [57], where Romania was shown to have more efficient R&D output than the UK. Since the DEA methodology asks that variables are identified as inputs or outputs, the following approach was taken. All of the variables have to be defined as inputs or outputs, where the idea is that those variables that have the smallest possible values are referred to as inputs, whereas outputs are defined as those that the researcher wants to have the greatest possible values. Furthermore, it is not possible to have only inputs in the analysis, nor only outputs. Finally, the missing data (which represented only 1.12% of the whole sample size) was penalized, as recommended in [52].

The SBM model was optimized for every year, with the efficiency scores for every country given in

Table 1. Efficiency scores in the slacks-based measure (SBM) model for the years 2007–2017.

Country	2007	2008	2009	2010	2011	2012	2013	2014	2015	2016	2017
Austria	0.255	0.330	0.323	0.334	0.385	0.411	0.433	1	0.606	1	1
Belgium	0.198	0.267	0.283	0.283	0.325	0.385	0.393	0.465	0.557	0.625	0.704
Bulgaria	0.002	0.002	0.003	0.002	0.008	0.003	0.004	0.006	0.006	0.006	0.006
Croatia	0.015	0.017	0.013	0.013	0.041	0.009	0.009	0.007	0.011	0.016	0.015
Cyprus	0.014	0.018	0.018	0.014	0.010	0.006	0.012	0.012	0.014	0.017	0.018
Czech Republic	0.066	0.075	0.062	0.063	0.168	0.097	0.107	0.117	0.154	0.132	0.165
Denmark	1	1	1	1	1	1	1	1	1	1	1
Estonia	0.037	0.053	0.053	0.057	0.134	0.068	0.070	0.060	0.089	0.081	0.095
Finland	1	1	1	1	1	1	1	1	1	1	1
Germany	1	1	1	1	0.563	1	1	1	1	1	1
Greece	0.030	0.040	0.030	0.019	0.103	0.026	0.031	0.036	0.045	0.042	0.047
Hungary	0.030	0.034	0.030	0.030	0.056	0.032	0.036	0.036	0.043	0.038	0.048
Iceland	1	1	1	1	1	1	1	1	1	1	1
Ireland	0.121	0.146	0.171	0.154	0.201	0.165	0.150	0.179	0.203	1	1
Italy	0.099	0.117	0.117	0.115	0.154	0.134	0.144	0.149	0.154	0.164	0.180
Latvia	0.008	0.014	0.006	0.008	0.012	0.016	0.019	0.026	0.021	0.013	0.017
Lithuania	0.009	0.015	0.007	0.010	0.119	0.021	0.030	0.038	0.033	0.026	0.034
Luxembourg	1	1	1	1	1	1	1	1	1	1	1
Malta	0.007	0.010	0.011	0.009	0.017	0.018	0.016	0.018	0.022	0.014	0.012
Netherlands	0.368	1	1	1	1	0.586	0.563	1	1	1	1
Norway	0.479	1	1	1	1	1	1	1	1	1	1
Poland	0.008	0.010	0.010	0.015	0.017	0.021	0.023	0.027	0.033	0.014	0.019
Portugal	0.036	0.047	0.040	0.037	0.110	0.031	0.035	0.037	0.046	0.046	0.050
Romania	0.002	0.003	0.002	0.002	0.003	0.003	0.004	0.004	0.005	0.005	0.006
Slovakia	0.010	0.011	0.009	0.015	0.063	0.020	0.020	0.028	0.045	0.028	0.033
Slovenia	0.105	0.152	0.151	0.150	0.201	0.201	0.203	0.193	0.179	0.174	0.187
Spain	0.101	0.126	0.119	0.128	0.209	0.116	0.115	0.116	0.121	0.125	0.143
Sweden	1	1	1	1	1	1	1	1	1	1	1
United Kingdom	0.159	0.184	0.162	0.159	0.163	0.157	0.145	0.168	0.185	0.160	0.169

. The best performing countries were: Denmark, Finland, Luxembourg, Sweden, Germany, Iceland, and Norway; whereas the worst performers were Bulgaria, Malta, Croatia, Latvia, Romania, and Slovakia. As defined in [44], the observed DMU is fully efficient (efficiency score equal to 1) only if it is not possible to improve any input or output without worsening some of the other inputs or outputs. This is called the Pareto–Koopmans efficiency, based on the work by economists Vilfredo Pareto and Tjalling Koopmans. Thus, the efficiency scores can be interpreted economically, since they are based on the definitions of efficiency from an economic analysis. However, the interpretations may depend on the model used in the study, as there are many different models within the DEA methodology. The methodological part of the paper states the model’s efficiency if the efficiency score is equal to 1 and the slacks regarding inputs and outputs are equal to zero. In the DEA approach, efficiency means achieving as many outputs as possible while using as few inputs as possible. The problem is choosing inputs and outputs. This research has chosen the inputs and outputs that were used in the previous literature, as well as those variables used by international organizations for calculation of their indices. In the SBM model used in this study, the efficiency score can be interpreted as the ratio of the mean input and output inefficiencies ([44], p. 101).

Firstly, the rankings in Table 1 are similar to the previous findings in [58], where Denmark, Sweden, Finland, and Germany were ranked as the best in technology transfers. In [33], the results show a distinct difference between Eastern and Southern European countries on one side and Western and Northern European countries on the other, based on the analysis of patent applications. In [59], alternative approaches of rankings were used (multi-attribute decision-making problems, MDMP) for a set of 27 European countries and Norway in 2012, where the best performers were found to be Sweden, Finland, Germany, and Denmark, while Romania, Bulgaria, Croatia, and Slovakia were found to be the worst performers. Thus, the best performing countries are those which have a better infrastructure for innovation and R&D, better education levels with greater PISA (Programme for International Student Assessment) results in schooling, and lower levels of corruption, whereas the opposite is true for the worst performers. Furthermore, some specific advantages of the best performers include having the best performance in human resources (Sweden, Finland, Ireland), excellent and effective research systems (Denmark, Netherlands, Sweden, the UK), finance and support (Finland, Sweden, Denmark), and firm investments (Sweden, Germany, Finland) [58]. Inefficient countries found here are in line with results in [43], where the below average CEE performers were Croatia, Bulgaria, and Lithuania.

Some of the unusual results in individual years include the following. Austria experienced a decline in the efficiency index value in 2015, which could have been due to the decrease of funding for research and experimental development of 2 percentage points in that year [60]. The increase of the efficiency scores in 2016 and 2017 that Ireland experienced compared to previous years could be due to a great amount of funding the country received in those years (e.g., €424 million in funding from Horizon 2020) [61]. Additionally, there was increased expenditure for innovation and R&D by government departments, showing an increase of over €40 million in 2017 compared to 2016, reaching the highest levels of expenditures since 2012 [62]. Finally, the Netherlands experienced a decline in efficiency scores in 2012 and 2013. The main causes could be the decrease of the percentage of the population holding doctorate-level education in the previous two years (2011 and 2012) [63], and that the country experienced a double blow in 2012 regarding the economic downturn in 2009, which made the total recovery longer [64]. The lower efficiency score in 2011 for Germany could be due to the decline of business R&D expenditure by foreign firms [65]. The results for 2007 for the Netherlands could be due to the worsening budgetary conditions in that year for the mentioned country [66], whereas Norway was one of the first economies to experience the consequences of the financial crisis of 2007–2008, with the economy weakening as early as 2007 [67].

Some of the main characteristics of the best performers in Table 1 are as follows. The authors of [68] focus on Denmark, where it was found that production in that country is heavily knowledge-based, with an open economy and good access to finance. The authors of [69] found that research priorities in Iceland include public–private collaborative publications, with [70] stating that health technologies, eco-technologies, industry, logistics, space, and ICT (Information and Communications Technologies) sectors represent priority sectors for public financing. Regarding Luxembourg, the government spending on R&D increased by more than 11 times in a 15-year period (2000–2015), which indicates the priorities of its government [71]. This country also has great science, technology, engineering, and math (STEM) competencies [72]. Regarding Finland, [73] states that this country is capable of pulling resources for the innovation and research sectors from the funds available within the EU. The European Structural and Investment Funds represent an important source of R&D and innovation activities in Finland, with almost 1.3 billion Euros received in the period 2014–2020. The Finnish government also channelled more public funding into innovation activities. This indicates that R&D is an important sector within that country. The reason why Sweden is among the most efficient countries could be because Swedish researchers are very open to cooperation with foreign researchers [74].

Furthermore, the dynamics of the worst performing countries are depicted in Figure 1, showing changes over time. This set of countries has faced problems in the last two decades, which have contributed to the current state of affairs. First, the majority of the worst performers include countries that have transitioned from a planned economy to a market economy. In those years of transition, these countries had other focuses outside of innovation and R&D policies. The main issues concern the business practices and legislation that need to be harmonized within European Union countries, as the majority of these countries are newer European Union member states. Some of these countries have a weak linkage between the business sector and the academic sector [75]. Malta faces problems with micro-enterprises (dominating over 95% of total business) within the private sector, which have limited resources regarding innovation and R&D, limited financing, and are vulnerable due to facing riskier R&D activities [76]. Other problems include high-skill sector labor shortages in Bulgaria and Romania, alongside highly-educated population emigration [77], a problem that is present in Croatia as well, where over the 2000–2010 period the total number of full-time equivalent (FTE) researchers decreased by almost a quarter [78]. Furthermore, Croatia’s other problems include a low level of public funding for research and a very fragmented research and innovation (R&I) system [79]. Finally, Latvia faces problems of low productivity and low capacity for absorption of new technologies [80], alongside low rates of risk-taking and entrepreneurship [81].

Table 2. Percentage increases of outputs and reductions of inputs for the year 2007.

Country	Exp	No. Papers	Pat	Emp Pvt	Empl Gov	Emp High	Res Pvt	Res Gov	Res High	R&D Pvt	R&D Gov	R&D High
Austria	99.58	77.18	0	45.59	634.59	109.15	92.22	740.52	115.31	0	58.97	1.9
Belgium	319.11	111.22	0	86.12	691.39	76.22	162.97	687.11	78.01	0	62.55	32.52
Bulgaria	4534.26	5116.31	139,209.86	15,845.53	775.15	4576.77	20,081.02	777.74	3978.81	93.82	0	92.44
Croatia	1208.25	1125.45	16,141.48	4389.08	633.31	1160.06	8262.54	705.31	1101.92	81.37	0	40.11
Cyprus	760.14	1746.32	4513.04	7996.26	3535.59	2461.33	9783.67	7292.21	2154.16	76.63	24.75	0
Czech Republic	546.67	1052.79	2373.07	831.79	864.56	888	1358.2	938.48	1040.81	19.31	0	43.47
Denmark	0	0	0	0	0	0	0	0	0	0	0	0
Estonia	1152.22	1281.03	2227.62	1792.43	1923.04	624.75	2406.84	1800.94	571.62	47.95	70.74	0
Finland	0	0	0	0	0	0	0	0	0	0	0	0
Germany	0	0	0	0	0	0	0	0	0	0	0	0
Greece	2000	919.6	4597.9	1667.76	2106.82	481.72	2352.34	2907.48	622.91	73.19	40.07	0
Hungary	646.86	2942.53	4094.29	2983.37	1914.76	2136.84	3342.41	2159.79	2294.76	32.04	0	31.53
Iceland	0	0	0	0	0	0	0	0	0	0	0	0
Ireland	20.58	142.29	104.45	218.75	1338.33	217.62	262.62	2066.14	211.54	0	67.76	10.64
Italy	831.21	609.69	230.55	594.65	808.9	400.1	1393.91	1120.49	633.88	20.73	32.15	0
Latvia	4560.98	10,913.7	15,040.89	11,860.97	3972.12	1486.33	21,306.09	4813.85	1537.16	65.35	20.85	0
Lithuania	1361.53	2340.73	48,752.21	5113.29	769.41	691.54	5843.27	856.48	677.74	85.46	9.25	0
Luxembourg	0	0	0	0	0	0	0	0	0	0	0	0
Malta	394.91	7303.53	6197.84	3178.34	25,142.36	2891.82	5217.65	36,535.1	3048.96	4.7	92.07	0
Netherlands	7.34	23.19	0	110.67	165.2	68.24	183.36	192.03	127.77	50.68	31.38	0
Norway	500.73	0	0	25.34	64.52	1.48	29.52	57.34	5.41	28.32	18.85	0
Poland	7498.41	5114.09	21,475.04	11,677.59	4913.22	2167.36	13,444.35	4378.01	1910.06	72.12	0	32.15
Portugal	919.71	932.74	8053.37	1814.67	1141.19	801.69	1889.82	987.84	610.25	55.53	30.73	0
Romania	4666.67	8255.95	149,703.33	8255.5	2757.12	8060.48	9627.91	2541.23	8065.29	85.62	0	68.04
Slovakia	4404.12	4647.56	15,474.15	10,055.89	3118.09	1660.3	12,828.57	2971.66	1427.71	63.41	0	50.04
Slovenia	1234.48	405.12	414.39	473.95	386.18	750.87	791.83	393.22	703.97	20.1	0	54.96
Spain	1497.71	742.34	983.8	710.92	826.83	415.52	1171.63	974.2	428.6	2.45	5.94	0
Sweden	0	0	0	0	0	0	0	0	0	0	0	0
United Kingdom	82.19	99.08	66.21	188.26	1121.48	53.24	281.73	1628.89	31.84	0	55.13	9.47

Note: R&D = research and development; Exp = exports variable; No. papers = the number of scientific journal ranking (SJR) papers; Pat = patents; Emp = employment; Res = researchers; pvt, gov, and high = private business, government, and higher education sectors, respectively (please see first paragraph in Section 4.1).

shows detailed results on how much each country needed to reduce inputs or increase outputs (in %) in the year 2007. The efficient countries from Table 1 have zero values for output increases and input reductions. Other countries that were found to be inefficient have positive values for reduction or expansion rates. A detailed table for the year 2017 is given in the Appendix in

Table A1. Percentage increases of outputs and decreases of inputs for the year 2017.

Country	Exp	No Papers	Pat	Emp Pvt	Empl Gov	Emp High	Res Pvt	Res Gov	Res High	R&D Pvt	R&D Gov	R&D High
Austria	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Belgium	0.00	37.26	22.82	23.54	104.46	0.00	14.07	77.83	5.02	0.00	0.00	21.92
Bulgaria	3497.75	4103.71	71,180.92	3872.19	1196.02	3081.59	3671.65	1055.82	2762.59	40.58	0.00	80.95
Croatia	1049.46	779.98	33,284.07	3498.10	861.31	644.58	4100.66	743.90	615.05	58.58	2.28	0.00
Cyprus	429.07	570.87	12,004.13	5214.08	590.12	941.03	6553.44	1419.38	910.23	81.14	0.00	40.20
Czech Republic	42.98	390.64	827.80	412.22	419.00	363.66	297.29	387.54	506.47	0.00	35.80	14.48
Denmark	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Estonia	113.28	309.45	2108.87	750.93	60.87	86.58	811.00	92.27	98.77	79.61	0.00	51.02
Finland	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Germany	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Greece	1269.45	375.22	10,548.02	1091.02	211.58	131.96	828.33	170.24	122.87	56.82	0.00	0.18
Hungary	530.38	1151.84	7511.22	1174.22	951.93	1128.96	940.59	711.38	1241.71	0.00	12.52	27.59
Iceland	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Ireland	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Italy	0.00	521.74	51.11	254.42	448.71	346.70	507.97	499.72	479.91	0.00	35.65	23.34
Latvia	0.00	345.74	1164.80	1568.66	468.27	65.57	866.33	426.31	127.04	95.67	90.23	79.60
Lithuania	721.05	609.21	13,212.43	1539.49	525.88	279.14	1305.96	382.16	248.27	74.11	0.00	1.36
Luxembourg	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Malta	0.00	2004.69	4117.12	1785.98	13,240.29	2112.11	2438.64	20,317.47	2283.57	0.00	90.83	7.80
Netherlands	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Norway	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
Poland	1264.81	3576.24	15,004.23	2698.69	3308.94	890.67	2979.71	3479.64	894.44	31.34	52.24	0.00
Portugal	898.05	637.42	7652.29	919.98	679.47	136.69	1086.33	804.87	123.79	58.59	11.61	0.00
Romania	1902.06	2650.70	47,170.97	8097.06	1734.94	3597.26	12,147.55	1715.56	3682.79	65.70	0.00	74.49
Slovakia	718.76	662.62	12,747.16	2616.70	785.45	458.64	2568.88	525.81	351.23	49.05	0.00	7.56
Slovenia	188.77	164.08	298.13	167.45	297.75	297.78	81.35	233.67	324.64	0.00	56.83	58.97
Spain	267.30	429.24	701.55	612.47	487.17	207.70	439.26	515.52	251.09	25.95	44.11	0.00
Sweden	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
United Kingdom	0.00	142.23	296.27	250.39	1236.75	37.49	277.41	1456.88	35.26	0.00	38.54	0.00

. In this way, policy makers can observe dynamic changes over these 11 years, see what has been done, and decide what requires more work. The greatest increase of outputs should be for the variable patent applications to the European Patent Office (EPO) per million inhabitants. This finding is in line with [82], which showed that European patent applications are constantly lagging behind the US and Japan. This same problem is prominent in 2017 (Table A1). Furthermore, by observing the average percentages in the last three columns (for R&D expenditures in all three sectors), the greatest decrease in their reciprocal values is needed in the private business sector. This result is in line with [83]. The needed changes for the worst performing countries in Table 1 can be seen in Table 2 and Table A1, which have to be done to achieve better results in the future. By comparing the needed changes in Table 2 and Table A1, it can be seen that input reductions and output increases have improved in 2017 compared to 2007, meaning the needed input reductions for the majority of countries (as well as output increases) are smaller. This means that some progress has been made over the years. The number of patents is still the greatest problem in 2017, especially for Croatia, Cyprus, Greece, and Hungary.

4.2. Checking for Robustness

To test the robustness of the results, a three-fold approach was taken. Firstly, the MCDM methodology was utilized to rank the countries based on the same variables. Within the MCDM modelling, all of the variables can be observed as outputs (i.e., the criteria by which the countries are compared to one another should be the greatest criteria possible). To obtain objective results, all of the criteria were given equal weights. Based on the optimal values of the functions that were optimized, the ranking system was obtained (detailed rankings are shown in the Appendix in

Table A2. Multi objective optimization by ratio analysis (MOORA) rankings used to check robustness.

Country	2007	2008	2009	2010	2011	2012	2013	2014	2015	2016	2017
Austria	10	9	10	9	12	10	9	9	8	7	7
Belgium	11	11	11	10	11	8	8	8	7	8	8
Bulgaria	29	29	28	28	29	29	28	28	28	28	28
Croatia	22	22	21	23	23	25	25	26	25	21	21
Cyprus	25	24	24	25	26	27	27	27	27	23	23
Czech Republic	14	14	15	16	13	14	14	13	12	12	12
Denmark	5	4	6	5	7	4	5	4	4	4	1
Estonia	18	18	17	17	16	17	17	16	15	15	15
Finland	1	1	1	1	2	1	2	2	1	2	3
Germany	6	6	8	4	5	3	4	5	5	5	6
Greece	21	20	20	20	17	19	19	17	16	16	16
Hungary	17	19	18	18	19	18	18	18	17	18	18
Iceland	2	3	3	3	3	9	10	24	22	24	25
Ireland	12	13	13	13	15	12	12	10	9	9	9
Italy	16	16	16	15	21	16	15	14	13	13	13
Latvia	24	25	27	26	24	26	26	23	26	26	26
Lithuania	20	21	22	22	18	23	21	19	21	20	19
Luxembourg	3	2	2	2	1	2	1	1	2	1	2
Malta	27	27	25	27	27	22	24	25	20	27	27
Netherlands	8	8	9	8	6	5	6	6	6	6	5
Norway	7	7	5	7	4	6	7	7	24	22	24
Poland	26	26	26	24	25	24	23	22	23	25	22
Portugal	19	17	19	19	20	20	20	20	19	17	17
Romania	28	28	29	29	28	28	29	29	29	29	29
Slovakia	23	23	23	21	22	21	22	21	18	19	20
Slovenia	13	12	4	11	9	11	11	11	11	11	10
Spain	15	15	14	14	14	15	16	15	14	14	14
Sweden	4	5	7	6	8	7	3	3	3	3	4
United Kingdom	9	10	12	12	10	13	13	12	10	10	11

). First, the values for every variable were normalized via Equation (3), and values in Equation (4) were calculated based on values from Equation (3). The second part of the MOORA includes the reference point theory and the min–max metric is used, where the distance of every alternative characteristic is minimized with respect to the number of alternatives [54]. The rankings based on DEA results (previous subsection) and the MCDM results (Table A2) were contrasted with one another and the correlations were calculated for every year. The correlation coefficients are shown in

Table 3. Correlations between rankings for data envelopment analysis (DEA) and multiple criteria decision making (MCDM) approaches.

Correlation	2007	2008	2009	2010	2011	2012	2013	2014	2015	2016	2017
Value	0.9821	0.9775	0.9699	0.9784	0.6342	0.9785	0.9848	0.9698	0.8578	0.8834	0.7538

, where it can be seen that the values are fairly high, even being as high as 98% in some cases (2007 and 2013). This confirms that the previously obtained results are meaningful and can be used in future research and within the decision-making process.

Next, the newest rankings from the European Innovation Scoreboard (EIS) [84] and Global Innovation Index (GII) [85] were obtained and compared to the rankings of the DEA model for the year 2017. The results are shown in Figure 2, where grey dots represent the scatter plot of DEA scores and the EIS values, whereas black dots depict the scatter plot of DEA scores and GII values. The correlations between the rankings are again very high, with over 78% correlation for the EIS ranking, which is based on 27 different indicators, while the GII is based on more than 80 indicators. Since the approach in this paper obtained similar rankings based on 12 variables, it can be said that (i) the rankings obtained in this research are reliable, as they are similar to those of international institutions; (ii) that such an approach can be used in the future to obtain meaningful results with less data, time, and resources; and (iii) that Figure 2 again confirms the validity of the previously obtained results.

The results obtained in this study should be discussed from the perspective of the role of structural and intrinsic differences in R&D decomposition between the countries. This has been recognized in the literature for a long time now [86,87,88]. Thus, the differences should be taken into account, as literature shows that significant differences exist in the rate of productivity returns from R&D investment [89,90,91]. The structural differences occur due to the differences in a country’s specific sector composition, whereas the intrinsic differences are due to countries under-investing in R&D [92]. The intrinsic differences were recognized as being dominant in the European Union in [93], which asks for policies to increase R&D spending, improve intellectual Property rights, and favoring foreign direct investment (FDI). The findings of this study are in line with findings in [94], where the structural effects have the greatest influence on R&D spending in the top performing countries in this analysis (and the opposite is true for the worst performers). They are also in line with [95], where it was found that Germany is experiencing a structural change towards technology-intense industries, whereas Denmark, Austria, and Sweden have experienced changes in the intrinsic effects. Another study [96] found that industrial specialization was the main driver of R&D intensity across countries (sample included 18 countries and 21 sectors). Furthermore, [97,98] found that the sectoral composition effect is greater in European countries, as the information and communications technology (ICT) sector is smaller in the European Union compared to the USA, with [99] finding that 85% of the gap between the European Union and USA in funding R&D is due to structural effects. Others claim that institutional differences (lower levels of government support for research in the EU) are the main reason for the R&D gap [100,101]. One study [102] found that European firms have both structural and intrinsic problems compared to US firms. Structurally, European firms are less able to transform R&D expenditures into productivity gains and the intrinsic problems occur due to a lower level of human capital. Thus, it is evident that future work needs to tackle these issues in more detail so that specific policy recommendations can be made for every country based on the structural and intrinsic differences among them.

5. Conclusions

Although the European Union has set ambitious goals regarding innovation policies and R&D, there are still problems in achieving the set goal of R&D representing 3% of GDP. Regional disparities have increased over time as well. An objective analysis of achievements and shortfalls is needed so that the required policy changes within a country can be made on time and to the best quality. The existing research has some shortfalls, including static analysis being used in the majority of cases, peculiar results being found in some of the research (namely, findings in which some of the worst-ranked countries in many international rankings have been found to be among the best-performers), and usage of aggregated R&D data without the separation of the sources of (in)efficiencies for individual countries. This is why the authors attempted to fill some of the gaps in the literature in this research. The policy recommendations for inefficient countries are as follows. Firstly, setting a common goal and targets to achieve it should not be a priority, as European countries have great differences among them in terms of economic, social, demographic, and other relevant characteristics. However, some guidance could be applied in inefficient countries. This would refer to the better overall education of the workforce, and especially those workers and researchers in the innovation and R&D sectors [103]. The literature here agrees that increasing investments in the education sector, especially higher education, alongside the STEM competencies, leads to better innovation results [104,105]. As a consequence, innovation represents one of the most important factors for sustainability [106]. The literature recognizes that innovation is a driver of sustainable policy-oriented thinking [107]. The openness of R&D researchers to collaboration at national and international levels is also important, due to the synergy it provides (see [108] for a list of important literature on this topic). Cooperation between the public sector and the private business sector is also advised, as the public sector affects the private one by offsetting any negative effects from public sector R&D on the labor costs in the private sector [109,110,111]. Since the majority of the private business sector in European countries is classified as small and medium enterprises, there is a problem in finding resources in terms of money, people, knowledge, and other relevant factors needed for innovation and R&D. Thus, external financing is an important factor that affects the whole process [112]. A variety of different measures have to be designed and access to finance is necessary; these factors depends on a firm’s characteristics [113].

It is important to mention other relevant roles played by sectoral distribution, firm size distribution, and path-dependence regarding the issues observed in this study. Thus, the recommendations for policymakers and future research are not unique and they depend on many factors that change over time. One landmark study [114] was the first major contribution to introduce localized technological changes, where it is stated that technological change is localized by learning characteristics. Another study [115] emphasizes the importance of learning in introducing new technologies (i.e., the learning process will generate necessary knowledge to introduce innovations in a greater manner than R&D activities). In this way, firms that learn how to generate new knowledge can obtain increasing returns due to greater learning abilities. The sizes of the firms that acquire new knowledge, the learning process, and the innovation process are also important. This was realized very early [116], where the link between a firm’s size and productivity was observed, showing that large firms were found to be more efficient than smaller ones. This was recently confirmed in [117], where in a sample of 117,000 firms in Europe the authors found that larger companies were more productive in the period 2002–2010. Similar conclusions were obtained in [118,119], where European and US firms were compared. The firm size of the firms that were R&D-intensive was found to be a significant factor in the total R&D intensity gap between the European and US firms. The combination of sectoral distribution and size was examined in [120], where analysis at the country level for the selected European Union countries showed that large countries had greater R&D intensity, with this intensity being affected by the industrial structure. Earlier studies were examined in [121], where the author concludes that the sectoral distribution of firm characteristics (e.g., a firm’s size) are affected by the technological paradigms of the individual sector. Furthermore, [98] found that the R&D intensity gap between 18 analyzed countries was due to across-sector variation (industrial specialization). Firm size was investigated in [121] for Italian firms, where it was found that policymakers need to provide more help for smaller companies to achieve better networking among them, which could compensate for firm size. Some of the shortfalls of this study include the following. Only available data were included in the study. This study uses a time series approach, for which the last available data upon writing this research was from the year 2017. This means that the included variables are not easy to measure and publicize on a more frequent basis. Furthermore, not all European countries had available data. Thus, the cross-section series approach suffered as well. Future work should extend the findings obtained here so that further dynamics can be observed. Additionally, a country-level approach was taken, which cannot indicate the differences between specific regions within a country. Although a decrease of R&D expenditure and researchers working in this sector was observed in this study, such variables are not measured at a sub-regional level (i.e., the data is not available).

Future research could utilize other models within the DEA approach, which could possibly provide other specific questions that could arise from the policy makers. Furthermore, if the data becomes available for other periods or countries (regions, sub-regions, etc.), such analysis should be extended to obtain full insights into the (in)efficiencies of those countries or regions that were not observed in this study. However, as previous literature did not focus on sources of (in)efficiencies in the same way that it was done here, this research provides contributions to the literature.