Efficiency Analysis of the Royalty System from the Perspective of Open Innovation

Abstract

The purpose of this study is to suggest efficiency improvement using the analysis of the efficiency of the royalty system for government-funded research institutes (GRIs) belonging to the National Research Council of Science & Technology (NST). Twenty three GRIs’ royalty incomes and expenses (2013–2015) were analyzed using the data envelopment analysis (DEA) model. First, Research Model 1 was used to find out if the obligated expense category’s distribution ratio were efficient. Five Charnes, Cooper and Rhodes (CCR) model organizations and 14 Banker, Charnes and Cooper (BCC) model organizations demonstrated 100% efficiency. With the exception of the obligated expense category, Research Model 2 was used. Seventeen CCR model organizations and 18 BCC model organizations demonstrated 100% efficiency. GRIs were divided into efficient and inefficient organizations using each model, and potential improvements and benchmarking decision-making units (DMUs) were found for inefficient organizations. Second, multiple regression analysis in Research Model 2 was used to analyze the cause of the efficiency to find factors that influenced the transfer of technology and license improvement. Third, there were efficiency differences among research organizations as a result of the efficiency analysis considering the research organization type with respect to the constant returns to scale (CRS) efficiency of Research Models 1 and 2. Thus, different policies should be applied to improve the efficiency. Finally, the possible improvements, future directions and limits of this study are discussed.

1. Introduction

Despite the increase of government spending on R&D in South Korea, there have been limits to enhancing the impact and technological commercialization of research outcomes. A new approach to the current mode of R&D is considered necessary to tackle this problem [1,2].

This study used data envelopment analysis (DEA) to analyze the efficiency of the royalty system. In general, higher royalty income means a more efficient operation of the system. To properly analyze this part, a key task is to determine accurately if an individual organization operates the system efficiently from the perspective of input and output. The management of the royalty system conducted by government-funded research institutes (GRIs) lacks efficiency from a comprehensive perspective since GRIs handle aims and results separately. Thus, finding out how the system can be improved is difficult.

This study originally began with awareness of the problem related to the existing system and is focused on the following three research questions. First, is GRIs’ royalty system efficient? Second, what are the variables that affect royalty income? Third, are there efficiency differences among research organizations?

2. Research Background

2.1. GRIs’ Royalty System

The previous domestic and international studies related to the royalty system are given below.

First, most research in Korea considers the government reimbursement method in the royalty system as a major obstacle to promoting technology transfers through analysis of legal validity and effectiveness.

More specifically, studies have been done on the legal contradiction in the government royalty system [3,4], the methods of collecting such royalties [4,5,6] and its management and utilization [3,5].

For the royalty system in non-profit organizations including GRIs, research on the system’s improvement is the main issue [7,8]. Recently, the issue of taxation of the monetary prizes for employee inventions has been reviewed [9,10].

Little research on the transfer or implementation of public technology has been conducted abroad, and most studies on the matter have been related to the private royalty system and private technology transfers and transactions. Research trends since 2000 have mainly focused on the candidate selection process and exploring the factors influencing technical royalties [11,12]. From 2009, studies began to appear on the process of determining the fee for applying the principal-agent models of financial theory and the adverse selection of information [13].

A royalty is defined as the amount of money paid by a licensee to a nation, R&D management agencies or organization that owns intellectual property rights (IPR) from R&D in return for acquiring the right to use such IPR, according to the Regulations on Management of National Research and Development Programs Article 2, No. 8 [14]. It was introduced based on the Regulations on the Treatment of Specified Research and Development Projects (2 June 1982, Ministry of Science and Technology Decree No. 187) [15]. The purpose of the royalty system was to encourage and expand the commercial use of the results of R&D projects, promote technology transfers generated through R&D projects conducted by GRIs to the private sector and reinvesting royalties in other R&D projects. This, it was thought, would form a virtuous cycle of R&D [16].

Royalties can be classified into those paid to the government and those paid to the proprietary agency. The former is the fee paid by for-profit organizations willing to own and utilize the IPR of a technology. The latter is the fee paid by the licensee to an organization that owns a specific IPR in return for acquiring the right to use that IPR. Since this study is on the royalty system of non-profit organizations including GRIs, the term royalty here will refer to the fee paid to the proprietary agency.

Details on the income and expenditure of royalties are as follows. Per Article 22, Paragraph 1, of the ‘Regulations on the Management of National R&D Projects, etc.’, the royalty shall be determined in agreement with the person who intends to conduct research and the head of the organization that owns the R&D IPR.

The average income from GRI royalties in the last two years was 90,079 million KRW, as

Table 1. GRI royalty income from national R&D (2015–2016) (unit: million KRW, %).

Institutes	2015	2016	Sum	Average
sum of 24 GRIs	83,526 (46.4%)	96,631 (53.6%)	180,181 (100.0%)	90,079

Source: royalty income from GRIs’ national R&D in 2017, Korea Institute of S&T Evaluation and Planning (KISTEP), 2017.

shows.

By type of organization, small and medium enterprises (SMEs) paid 65,781 million KRW in royalties (78.8 percent), corporations 14,284 million KRW (17.1 percent) and others 3461 million KRW (4.1 percent). In 2016, SMEs paid 64,701 million KRW (66.3 percent) in royalties, corporations 20,773 million KRW (21.5 percent) and others 11,787 million KRW (12.2 percent).

By payment method, fixed royalties accounted for 38,910 million KRW (46.6 percent) and running royalties 44,616 million KRW (53.4 percent) in 2015; the figures for 2016 were 45,500 million KRW in fixed royalties (45 percent) and 43,513 million KRW in running royalties (54.9 percent).

On the basis of Article 22 and Article 23 of the ‘Regulations on the Management of National R&D Projects, etc.’, at least 65% of royalties should be used for the expenses of the management of intellectual property rights (5%), compensation for participating researchers (50%) and technology transfer and commercialization expenses (10%). The remaining amount (less than 35%) could be used in R&D reinvestment, agency operation expenses, IPR application/registration, compensation for technology extension or as a reserve fund for royalties.

2.2. Concept of DEA

DEA analysis was developed in the late 1970s by Charnes, Cooper and Rhodes (1978) [17] for evaluating efficiency. This method derives the relative efficiency value of decision-making units (DMUs) in similar environments by simultaneously considering multiple input and output variables [18,19]. The efficiency value varies from 0–1, and a number closer to one denotes higher comparative efficiency. Recently, DEA analysis has been recognized as the proper method for measuring the efficiency of the public and non-profit sectors and is utilized at public organizations, hospitals, banks, schools and other facilities.

DEA models vary according to the goals and characteristics of data. Among these models, the most widely used is the CCR model developed by Charnes, Cooper and Rhodes (1978) [17], assuming constant returns to scale (CRS). This model starts with the hypothesis that the relationship between input and output is the same at a constant rate regardless of scale. Thus, the technical efficiency (TE) value obtained from the model has the disadvantage that it cannot be distinguished from the values of pure technical efficiency (PTE) and scale efficiency (SE).

Another model is the BCC model, which assumes variable returns to scale (VRS) of different scales, developed by Banker, Charnes and Cooper (1984) [20]. This model eases the hypothesis that the relationship between input and output is the same at a constant rate regardless of size. In particular, variably-sized revenues cover all three revenue states: increasing returns to scale (IRS), constant returns to scale (CRS) and decreasing returns to scale (DRS) [19].

In addition, the DEA model can also be classified as input and output based. If the aim is to minimize input to achieve a fixed amount of output, which is measuring input-based efficiency, this model shows how much input is needed for similar performances. Output-based efficiency, which concentrates on maximizing output with fixed inputs, is a useful model for assessing the extent of various inputs for similar output performances.

Recent efficiency studies on DEA analysis have focused on measuring the efficiency of the public sector or government-run enterprises. For research on the public sector, Mourao (2015) [21] analyzed efficiency by comparing the tariffs of the French canal to those of surrounding nations. Li et al. (2015) [22] used the bootstrap DEA model to compensate the deviation derived from the BCC model and analyzed technical efficiency at China’s public hospitals. They concluded that the bootstrap DEA model was highly utilized as a benchmarking technology in later studies. Xiang et al. (2017) [23] used the DEA models in a variety of studies, including research on nature-based alternatives for urban regeneration related to China’s vulnerability to natural disasters. In addition, Biondi et al. (2013) [24] used the DEA model to assess corporate efficiency by assessing the efficiency of car dealers in the automotive industry, and Ohe (2016) [25] used the window DEA model to analyze the efficiency of Japanese hotels by size and region. One finding has been that larger hotels show higher efficiency.

Few studies have analyzed the efficiency at the level of national R&D programs or projects that are the subject of this study. By using the DEA and Tobit models, Hsu and Hsue (2009) [26] found that small enterprises conducting government-funded R&D projects showed higher efficiency. Lee, Park and Choi (2009) [27] discovered that among six government R&D projects in Korea, DEA and nonparametric analysis showed extremely high efficiency in industrial-academic cooperation and international exchange support projects in engineering, while the efficiency of projects supporting basic science researchers was low. Hwang et al. (2009) [28] attempted to analyze efficiency by using the DEA and Tobit models. Kim et al. (2009) [29] measured the relative efficiency of the nuclear program through a comparison of each project. Park et al. (2011) [30] assessed the efficiency of R&D projects supported by the government in scientific and technological areas such as papers and patents by utilizing science and technology investment and performance data of the National Science and Technology Information Service (NTIS).

3. Research Methodology

3.1. DMU Decision

To measure efficiency using the DEA model, the decision-making unit (DMU) should be determined in advance. When determining the DMU, homogeneity among DMUs should be presupposed, and Dyson et al. (2001) [31] suggested that activities of the businesses should be similar for comparison such as having related outputs (products and services) with similar characteristics, resources of a similar range operating as inputs and the operation of similar business environments.

Considering the homogeneity of the analyzed subjects, all but one of the 24 GRIs under the National Research Council of Science and Technology (NST) were selected; one was excluded because of a missing value of the input variables for the Korea Astronomy and Space Science Institute (KASI).

In addition, the 23 GRIs can be classified according to mission type. According to the NST’s criteria (2014) [32], the GRIs were classified into three groups: public infrastructure, industrialization and basic and future leading research organization.

Specifically, while five of these organizations (including DMU 1, DMU 2, DMU 3, DMU 4 and DMU 5) belong to the public and infrastructure types, six organizations (DMU 6–DMU 11) correspond to the industrialization types, and the remainder (DMU 12–DMU 23) fall under the basic and future leading types.

3.2. Selection of Input/Output Variables

The types of expenditure and income were divided into two research models for efficiency analysis as shown in

Table 2. Input and output variables used.

Division		Variables
Input variables (royalty expense, unit: %)	Research Model 1	Management of intellectual property
		Expenditure for commercialization
		Compensation for researchers
		Remaining expenditure
	Research Model 2	R&D reinvestment
		Organization operating expense
		Compensation for intellectual property’s application and registration, technical diffusion
		Reserve fund for royalty expense
		Others
Output variables (royalty income, unit: million KRW)		Technology transfer
		License
		Technical advice
		Others

.

According to the ‘Regulations on the Management of National R&D Projects, etc.’ as mentioned above, the expenditure on IPR should be at least 5%, technology commercialization expenditure should be at least 10% and participating researcher compensation should be at least 50%, then the remaining expenditure is less than 35%. Therefore, Research Model 1 was established to see if the composition ratio of mandatory expenditure items was efficient.

Research Model 2 was established to analyze the efficiency of remaining expenditure, which has different values among organizations.

Because of the difficulty in finding the factors directly relevant to performance because of the time-lag effect occurring between input and output and given that all input related to output cannot be measured, accurate assessment of efficiency faces limits. The real period of the time-lag effect that can be reasonably accepted between the input and output periods in R&D activities does not exist [33]. This study analyzed the royalty system’s efficiency based on collected royalty data from 2013–2015, assuming that royalty expenditures in 2013 were related to the royalty income in 2015.

The data of input and output variables for efficiency analysis are shown in

Table 3. Input variables of efficiency analysis (unit: %). DMU, decision-making unit.

DMUs	Research Model 1				Research Model 2
	Management of Intellectual Property	Expenditure for Commercialization	Compensation for Researchers	Remaining Expenditure	R&D Reinvestment	Organization Operating Expense	Compensation for Intellectual Property’s Application and Registration, Technical Diffusion	Reserve Fund for Royalty Expense	Others
DMU 1	0.000	0.000	50.000	50.000	0.000	0.000	10.345	39.655	0.000
DMU 2	7.188	0.000	55.272	37.540	16.853	0.000	0.000	18.131	2.556
DMU 3	21.850	0.000	48.902	29.364	29.364	0.000	0.000	0.000	0.000
DMU 4	13.721	7.900	50.728	27.651	26.611	0.000	1.040	0.000	0.000
DMU 5	0.000	0.000	50.594	49.406	0.000	49.406	0.000	0.000	0.000
DMU 6	0.556	0.000	58.462	40.982	3.505	32.227	5.250	0.000	0.000
DMU 7	21.888	17.191	50.661	10.261	5.791	0.000	1.891	0.000	2.578
DMU 8	13.041	0.000	55.068	31.890	23.178	8.713	0.000	0.000	0.000
DMU 9	10.124	0.000	55.396	34.479	25.024	6.686	2.770	0.000	0.000
DMU 10	8.747	4.342	44.310	42.600	39.319	0.000	3.282	0.000	0.000
DMU 11	1.155	0.000	62.600	36.245	49.347	0.452	0.251	-13.805	0.000
DMU 12	0.205	17.137	50.487	32.170	4.669	25.936	1.565	0.000	0.000
DMU 13	5.179	9.960	50.199	35.060	1.992	0.000	0.000	33.068	0.000
DMU 14	27.269	17.319	50.749	4.663	3.122	0.000	0.000	0.833	0.708
DMU 15	12.261	34.483	49.808	3.448	0.000	0.000	3.448	0.000	0.000
DMU 16	38.017	0.000	53.719	8.264	0.000	0.000	8.264	0.000	0.000
DMU 17	0.000	0.000	50.000	50.000	44.988	0.000	5.012	0.000	0.000
DMU 18	5.053	0.000	49.734	45.213	9.309	0.000	0.798	0.000	35.106
DMU 19	17.456	0.000	36.686	45.858	25.740	20.118	0.000	0.000	0.000
DMU 20	5.000	10.000	50.000	35.000	0.000	0.000	10.000	25.000	0.000
DMU 21	10.073	4.964	55.766	29.197	21.314	4.964	2.920	0.000	0.000
DMU 22	13.274	0.000	52.786	33.940	23.383	8.653	1.904	0.000	0.000
DMU 23	0.000	0.000	59.091	36.364	0.000	0.000	0.000	36.364	0.000

Source: Data analyzed by the authors.

and

Table 4. Output variables of efficiency analysis (unit: million KRW).

DMUs	Technology Transfer	License	Technical Advice	Others
DMU 1	0	353	0	34
DMU 2	100	2781	0	0
DMU 3	0	2287	0	0
DMU 4	99	1541	0	8
DMU 5	112.7379	1380.593	0	37,664.598
DMU 6	1512	10,129	0	7
DMU 7	435	33,172	0	0
DMU 8	6	4114	11	580
DMU 9	22	1052	994	0
DMU 10	21	4654	238	0
DMU 11	669	2287	0	798
DMU 12	179	4482	0	452
DMU 13	0	173	0	20
DMU 14	0	973	0	0
DMU 15	0	581	0	1
DMU 16	0	294	0	0
DMU 17	26.4	509.4	0	0
DMU 18	18	1581	0	0
DMU 19	56	615	7	0
DMU 20	0	13.2	0	0
DMU 21	0	1452	0	0
DMU 22	9	2070	0	0
DMU 23	0	21	0	0

Source: Data analyzed by the authors.

below.

3.3. Decision of the DEA Model

As previously mentioned in this paper, the DEA model can be divided into the CCR and BCC models according to the assumption of the scale effect and be classified as an input- or output-based models according to the purpose of the efficiency measurement.

In this study, both the CCR and BCC models were applied and their results compared and analyzed. If the purpose of measuring R&D efficiency were to improve it rather than evaluate it, it would have been reasonable to discuss how to increase output while fixing, rather than reducing, input (Hwang 2009). The calculation standard model was adopted to raise royalty income (performance).

This study used the open source software R for analysis of the DEA model and the DEA functions created by Lee and Oh (2012).

4. Results of Analysis

In this study, the results of efficiency are presented through various models, factors influencing royalty income and the difference in efficiency considering the type of research organization.

4.1. Efficiency Results in the Research Model 1

Table 5. Efficiency results of output-oriented Research Model 1. PTE, pure technical efficiency; SE, scale efficiency; IRS, increasing returns to scale; CRS, constant returns to scale.

DMUs	TE	PTE	SE	Cause of Inefficiency	Sum of Weights ${\displaystyle {\displaystyle \sum }_{\mathbf{i}=0}^{\mathbf{n}}}\mathsf{\lambda }\mathbf{i}$	Return to Scale RTS
DMU 1	0.259	1.000	0.259	SE	0.988	IRS
DMU 2	0.300	0.357	0.839	PTE	0.916	IRS
DMU 3	0.315	0.820	0.384	SE	0.717	IRS
DMU 4	0.096	0.135	0.709	PTE	0.910	IRS
DMU 5	1.000	1.000	1.000	-	1.000	CRS
DMU 6	1.000	1.000	1.000	-	1.000	CRS
DMU 7	1.000	1.000	1.000	-	1.000	CRS
DMU 8	0.553	0.671	0.825	PTE	0.776	IRS
DMU 9	1.000	1.000	1.000	-	1.000	CRS
DMU 10	0.543	1.000	0.543	SE	0.807	IRS
DMU 11	0.527	1.000	0.527	SE	0.876	IRS
DMU 12	1.000	1.000	1.000	-	1.000	CRS
DMU 13	0.013	0.019	0.679	PTE	0.894	IRS
DMU 14	0.065	1.000	0.065	SE	0.454	IRS
DMU 15	0.052	1.000	0.052	SE	0.334	IRS
DMU 16	0.144	1.000	0.144	SE	0.202	IRS
DMU 17	0.373	1.000	0.373	SE	0.988	IRS
DMU 18	0.183	0.351	0.522	PTE	0.851	IRS
DMU 19	0.106	1.000	0.106	SE	0.631	IRS
DMU 20	0.001	0.001	0.639	PTE	0.883	IRS
DMU 21	0.090	0.091	0.990	PTE	0.929	IRS
DMU 22	0.247	0.361	0.684	PTE	0.828	IRS
DMU 23	0.021	1.000	0.021	SE	0.736	IRS
Average	0.386	0.731	0.581		0.814

Source: Data analyzed by the authors.

shows the results of the efficiency analysis of the input variables, which include expenses of IP management, technical commercialization, participant researcher compensation and remaining expenditure and output variables including technology transfer, licensing, technical advice, and so on.

From the efficiency analysis of the CCR model, five organizations such as DMU 5 showed high efficiency with a value of one, but the overall average efficiency was low at 0.386.

From the analysis of BCC model, 14 organizations (60 percent) showed high efficiency with a value of one, and the average efficiency was high at 0.731.

By comparatively analyzing PTE and SE, 10 and eight organizations had lower efficiency due to scale inefficiency and technical inefficiency, respectively.

The most inefficient organization in scale was DMU 23 (0.021), which is entirely due to inefficiency in scale. Therefore, its inefficiency could be removed from the model through the adjustment of input variables. For inefficiency caused by mere technical inefficiency, a variety of measures is needed, which include the improvement of the royalty expenditure system.

The Return to Scale (RTS) analysis of the scale found 18 organizations, except five in a scale-optimized state including DMU 5, showing a feasibility state.

Thus, organizations in a feasibility state with respect to scale should scale up because they can gain additional benefits by boosting their scale of input.

4.1.1. Analysis of Potential Improvement in Research Model 1

Table 6. Surplus input and output shortage in Research Model 1.

DMUs	Surplus Input				Output Shortage
	Management of Intellectual Property	Expenditure for Commercialization	Compensation for Researchers	Remaining Expenditure	Technology Transfer	License	Technical Advice	Others
DMU 1	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 2	0.000	0.000	0.000	0.000	871.897	0.000	0.838	5.301
DMU 3	0.116	0.000	0.000	0.000	386.089	0.000	2.452	1.697
DMU 4	0.000	0.000	0.000	0.000	0.000	7454.713	1.323	0.000
DMU 5	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 6	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 7	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 8	0.000	0.000	0.000	0.000	885.964	0.000	0.000	0.000
DMU 9	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 10	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 11	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 12	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 13	0.000	0.000	0.000	0.398	198.485	0.000	14.048	13,219.054
DMU 14	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 15	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 16	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 17	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 18	0.000	0.000	0.000	0.000	578.819	0.000	1.941	12,872.057
DMU 19	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 20	0.000	0.000	0.000	0.000	189.928	0.000	21.478	13,338.341
DMU 21	0.000	0.000	0.000	0.000	1064.790	0.000	0.006	4.348
DMU 22	0.000	0.000	0.000	0.000	813.505	0.000	1.449	3.828
DMU 23	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
Average	0.005	0.000	0.000	0.017	216.934	324.118	1.893	1714.984

Source: Data analyzed by the authors.

below shows the input to be reduced and the output that can be increased to make an inefficient organization efficient by analyzing the surplus input and output shortage.

The surplus input is the additional amount of input that can be reduced while maintaining the same amount of output, and the output shortage is the amount of difference between the targeted and existing amounts of output.

The analysis of surplus input found extra amounts of resources in the IPR management fees and remaining expenditure, so reductions of 0.005 and 0.017 for each of them are necessary.

Because of the output shortage, the others category had the lowest output, followed by licensing and technology transfer fees. This shows that many of the subject organizations see low results in the others category and must strive for improvement.

For DMU 3, the surplus input for the management fees in IPR was 0.116, and the output shortage included: for technology transfer, 386.089, technology advisory, 2.452, and others, 1.697. Thus, DMU 3 should reduce 0.116 percent of expenditures for the maintenance of IPR and increase 396.089 million KRW, 2.452 million KRW and 1.697 million KRW, respectively, in technology transfer, technology advice and others.

4.1.2. Benchmarking DMUs in Research Model 1

To raise efficiency in inefficient organizations, the organizations should follow reference DMUs and reduce their inputs or raise their outputs. The reference DMUs can provide information on what the inefficient organizations should change.

Table 7. Reference DMUs in Research Model 1.

DMUs	PTE	Reference DMUs (Weight Values)	Frequency
DMU 1	1.000		0
DMU 2	0.357	DMU 6(0.757), DMU 16(0.123), DMU 19(0.120)	0
DMU 3	0.820	DMU 6(0.242), DMU 16(0.407), DMU 19(0.350)	0
DMU 4	0.135	DMU 5(0.002), DMU 6(0.345), DMU 7(0.460), DMU 16(0.005), DMU 19(0.189)	0
DMU 5	1.000		5
DMU 6	1.000		7
DMU 7	1.000		4
DMU 8	0.671	DMU 5(0.023), DMU 6(0.587), DMU 9(0.016), DMU 16(0.293), DMU 19(0.082)	0
DMU 9	1.000		1
DMU 10	1.000		2
DMU 11	1.000		0
DMU 12	1.000		2
DMU 13	0.019	DMU 5(0.357), DMU 7(0.210), DMU 10(0.059), DMU 12(0.356)	0
DMU 14	1.000		0
DMU 15	1.000		0
DMU 16	1.000		6
DMU 17	1.000		0
DMU 18	0.351	DMU 5(0.342), DMU 6(0.381), DMU 19(0.277)	0
DMU 19	1.000		7
DMU 20	0.001	DMU 5(0.350), DMU 7(0.189), DMU 10(0.090), DMU 12(0.371)	0
DMU 21	0.091	DMU 6(0.621), DMU 7(0.289), DMU 16(0.089), DMU 19(0.001)	0
DMU 22	0.361	DMU 6(0.547), DMU 16(0.246), DMU 19(0.207)	0
DMU 23	1.000		0

Source: Data analyzed by the authors.

below shows the reference DMUs and their weight values under the variable model of returns to scale. The weight value means the degree of influence of the efficient organizations on the inefficient ones.

As shown in the table, DMU 19 was selected seven times as a benchmarking DMU. This shows that DMU 19 had a greater influence in determining the efficiency of other organizations.

In the case of DMU 2, the reference DMUs were DMUs 6, 16 and 19, and the comparative efficiency was only 35.7 percent. The weight values of the reference DMUs of DMU 2 were 0.757 for DMU 6, 0.123 for DMU 16 and 0.120 for DMU 19. For DMU 2 to become a more efficient organization, its target values should be the sum of the multiplied values of the input and output values and the weight values of the reference DMUs. The result is shown in

Table 8. Target value of the input and output of DMU 2.

Criteria	Input				Output				Weight
	Management of Intellectual Property	Expenditure for Commercialization	Compensation for Researchers	Remaining Expenditure	Technology Transfer	License	Technical Advice	Others
DMU 6	0.556	0.000	58.462	40.982	1512	10129	0.000	7.000	0.757
weight	0.421	0.000	44.256	31.023	1144.584	7667.653	0.000	5.299
DMU 16	38.017	0.000	53.719	8.264	0.000	294	0.000	0.000	0.123
weight	4.676	0.000	6.607	1.016	0.000	36.162	0.000	0.000
DMU 19	17.456	0.000	36.686	45.858	56	615	7	0.000	0.120
weight	2.095	0.000	4.402	5.503	6.720	73.800	0.840	0.000
DMU 2 target value	7.192	0.000	55.265	37.543	1151.304	7777.615	0.840	5.299

Source: Data analyzed by the authors.

below.

4.2. Efficiency Results in Research Model 2

Table 9. Efficiency results of output-oriented Research Model 2.

DMUs	TE	PTE	SE	Cause of Inefficiency	Sum of Weights ${\displaystyle {\displaystyle \sum }_{\mathbf{i}=0}^{\mathbf{n}}}\mathsf{\lambda }\mathbf{i}$	RTS
DMU 1	1.000	1.000	1.000	-	1.000	CRS
DMU 2	1.000	1.000	1.000	-	1.000	CRS
DMU 3	1.000	1.000	1.000	-	1.000	CRS
DMU 4	1.000	1.000	1.000	-	1.000	CRS
DMU 5	1.000	1.000	1.000	-	1.000	CRS
DMU 6	1.000	1.000	1.000	-	1.000	CRS
DMU 7	1.000	1.000	1.000	-	1.000	CRS
DMU 8	1.000	1.000	1.000	-	1.000	CRS
DMU 9	1.000	1.000	1.000	-	1.000	CRS
DMU 10	1.000	1.000	1.000	-	1.000	CRS
DMU 11	1.000	1.000	1.000	-	1.000	CRS
DMU 12	1.000	1.000	1.000	-	1.000	CRS
DMU 13	1.000	1.000	1.000	-	1.000	CRS
DMU 14	1.000	1.000	1.000	-	1.000	CRS
DMU 15	1.000	1.000	1.000	-	1.000	CRS
DMU 16	0.211	0.506	0.417	SE	2.397	DRS
DMU 17	0.160	0.285	0.561	PTE	2.618	DRS
DMU 18	0.109	1.000	0.109	SE	0.656	IRS
DMU 19	1.000	1.000	1.000	-	1.000	CRS
DMU 20	0.008	0.023	0.342	PTE	3.588	DRS
DMU 21	0.356	0.368	0.968	PTE	0.792	IRS
DMU 22	0.427	0.432	0.988	PTE	0.908	IRS
DMU 23	1.000	1.000	1.000	-	1.000	CRS
Average	0.794	0.853	0.886		1.216

shows the results of the efficiency analysis of the input variables, which include R&D reinvestment, agency operation expenses, IPR application registration and technology diffusion contributor compensation, technical fee provision and the others and output variables, which include technology transfer, licensing, technical advice, and so on.

While the CCR model showed high efficiency with a value of one among the 17 organizations and an overall average efficiency as high as 0.794, the BCC model demonstrated high efficiency with a value of one among the 18 organizations and an average efficiency as high as 0.853.

By comparatively analyzing PTE and SE, two and four organizations had lower efficiency due to scale inefficiency and technical inefficiency, respectively.

The most inefficient organization in scale was DMU 18(0.109). Its inefficiency could be removed from the model through adjustment of the input variables. For inefficiency caused by mere technical inefficiency, a variety of measures are needed, which include the improvement of the royalty expenditure system.

The RTS analysis of the scale found three organizations showing a feasibility state and three showing an infeasibility state (excluding the 17 in a scale-optimized state). Thus, organizations with a feasibility state scale (DMU 18, DMU 21, DMU 22) should scale up, while organizations in an infeasibility state scale (DMU 16, DMU 17, DMU 20) should scale down for the additional benefits.

4.2.1. Analysis of Potential Improvement in Research Model 2

Table 10. Surplus input and output shortage in Research Model 2.

DMUs	Surplus Input					Output Shortage
	R&D Reinvestment	Organization Operating Expense	Compensation for Intellectual Property’s Application and Registration, Technical Diffusion	Reserve Fund for Royalty Expense	Others	Technology Transfer	License	Technical Advice	Others
DMU 1	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 2	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 3	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 4	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 5	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 6	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 7	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 8	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 9	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 10	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 11	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 12	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 13	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 14	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 15	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 16	0.000	0.000	4.816	0.000	0.000	0.000	0.000	0.000	1.000
DMU 17	17.361	0.000	3.793	0.000	0.000	0.000	0.000	19.030	7.360
DMU 18	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 19	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
DMU 20	0.000	0.000	6.552	25	0.000	0.000	0.000	0.000	1.000
DMU 21	0.000	0.000	0.000	0.000	0.000	166.455	0.000	102.65	109.44
DMU 22	0.000	0.000	0.000	0.000	0.000	188.504	0.000	73.668	292.85
DMU 23	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
Average	0.755	0.000	0.659	1.087	0.000	15.433	0.000	8.493	17.898

Source: Data analyzed by the authors.

below shows the input to be reduced and the output that can be increased to make an inefficient organization efficient using analysis of the surplus input and output shortage.

The analysis of surplus input found that it would be necessary to reduce the amounts for technical fee provision, R&D reinvestment, IPR application/registration and technology diffusion contributor compensation in sequence.

As a result of the analysis of the output shortage, the others category had the lowest output, followed by technology transfer fees and technical advice fees, respectively.

For DMU 16, the surplus input for the IPR application/registration and technology diffusion contributor compensation was 4.816, and the output shortage in the others category was one. Thus, DMU 16 should reduce expenditures for IPR application/registration and technology diffusion contributor compensation by 4.816 percent and increase one million KRW in the others category to increase efficiency.

4.2.2. Benchmarking DMUs in Research Model 2

To raise efficiency in inefficient organizations, appropriate reference DMUs should be followed and should reduce their inputs or raise their outputs.

Table 11. Reference DMUs in Research Model 2.

DMUs	PTE	Reference DMUs (Weight Values)	Frequency
DMU 1	1.000		0
DMU 2	1.000		0
DMU 3	1.000		0
DMU 4	1.000		1
DMU 5	1.000		0
DMU 6	1.000		2
DMU 7	1.000		0
DMU 8	1.000		2
DMU 9	1.000		0
DMU 10	1.000		3
DMU 11	1.000		0
DMU 12	1.000		0
DMU 13	1.000		0
DMU 14	1.000		0
DMU 15	1.000		4
DMU 16	0.506	DMU 15(1.000)	0
DMU 17	0.285	DMU 4(0.920), DMU 10(0.080)	0
DMU 18	1.000		0
DMU 19	1.000		0
DMU 20	0.023	DMU 15(1.000)	0
DMU 21	0.368	DMU 6(0.103), DMU 8(0.187), DMU 10(0.423), DMU 15(0.287)	0
DMU 22	0.432	DMU 6(0.132), DMU 8(0.503), DMU 10(0.286), DMU 15(0.078)	0
DMU 23	1.000		0

Source: Data analyzed by the authors.

below shows the reference DMUs and their weight values under the variable model of returns to scale.

As shown in the table, DMU 15 was selected four times as a benchmarking DMU, which means DMU 15 had a greater influence in determining the efficiency of other organizations.

In the case of DMU 17, the comparative efficiency was only 28.5 percent with reference DMUs, DMUs 4 and 10. The weight values of the reference DMUs compared to DMU 17 were 0.920 for DMU 4 and 0.080 for DMU 10. For DMU 17 to become a more efficient organization, its target values could be calculated regarding the weight values of the reference DMUs, as shown in

Table 12. Target value of the input and output of DMU 2.

Criteria	Input					Output				Weight
	R&D Reinvestment	Organization Operating Expense	Compensation for Intellectual Property’s Application and Registration, Technical Diffusion	Reserve Fund for Royalty Expense	Others	Technology Transfer	License	Technical Advice	Others
DMU 4	26.611	0.000	1.040	0.000	0.000	99.000	1541.000	0.000	8.000	0.920
weight	24.482	0.000	0.957	0.000	0.000	91.080	1417.720	0.000	7.360
DMU 10	39.319	0.000	3.282	0.000	0.000	21	4654	238	0.000	0.080
weight	3.1455	0.000	0.262	0.000	0.000	1.68	372.32	19.04	0.000
DMU 17 target value	27.627	0	1.219	0	0	92.76	1790.04	19.04	7.36

Source: Data analyzed by the authors.

below.

4.3. Analysis of Factors Affecting Royalty Income (Performance)

The DEA analysis itself is meaningful, but holds even greater significance when conducted on the causes of efficiency. To analyze the impact of royalty expenditures on royalty income, a canonical correlation analysis was conducted. After the effects of mandatory expenditures and the remaining expenditures were assessed on royalty income, no correlation was found in Research Model 1 through the canonical correlation analysis. The results of this analysis in Research Model 2, which was conducted to study the impact of other subdivided remaining expenditures for royalty income, also showed no correlation.

In Research Model 2, however, multiple regression analysis was conducted for each type of royalty income (i.e., technology transfer, licensing or technology consulting, others) as dependent variables. The results showed that the effective variables exercising significant influence on technology transfer and licensing fees were derived. The organized results can be seen in

Table 13. Multiple regression analysis results.

Dependent Variable	Independent Variable	Estimates	t-Value	p-Value
Technology transfer	R&D reinvestment	−0.151	−1.949	0.068 *
	Organization operating expense	0.297	2.510	0.022 **
	Compensation for intellectual property’s application and registration, technical diffusion	1.962	2.728	0.014 **
	Reserve fund for royalty expense	−1.429	−2.942	0.009 ***
	Others	−0.368	−0.791	0.440
License	R&D reinvestment	−0.084	−0.080	0.927
	Organization operating expense	0.768	0.478	0.639
	Compensation for intellectual property’s application and registration, technical diffusion	28.297	2.903	0.001 ***
	Reserve fund for royalty expense	−1.592	−0.242	0.812
	Others	29.827	4.726	0.000 ***

* denotes sig (p-value) < 0.1, ** denotes sig (p-value) < 0.05, *** denotes sig (p-value) < 0.01.

below.

Through the results of the above analysis, R&D reinvestment, expenses for agency operations, IPR registration fees, compensation for contributors of technological distribution and the accumulation of royalties were found to have significant effects. Specifically, R&D reinvestment and the accumulation of royalties had negative (−) effects, and the agency’s operating expenses, IPR registration fees and compensation for contributors of technological distribution had positive (+) effects. This means that to increase technology transfer fees, reductions in R&D reinvestment cost and accumulation of royalties are needed, and more focus is required on organizational operating expenses, IPR registration fees and spending on compensation for technological contributors.

Furthermore, licensing royalties were found to have a significant impact on IPR registration fees, compensation for technology spreaders and others. IPR registration fees, compensation for technology contributors and others all showed positive effects and led to higher licensing royalties.

4.4. Results of Comparative Analysis of Differences in Efficiency Considering Research Organization Type

In this section, the differences in the efficiency by research organization type, such as public infrastructure, industrial and basic and future leading research will be covered. Since the efficiency values derived from DEA analysis did not assume parameters, this analysis sought to determine if a statistically-significant difference existed through the Kruskal–Wallis test, which is the non-parametric mean difference analysis. The results are shown in

Table 14. Efficiency difference by research organization type. VRS, variable returns to scale.

Model	Efficiency	Kruskal–Wallis t-Test
		χ2	Sig (p-Value)
Research Model 1	CRS efficiency	11.063	0.004 ***
	VRS efficiency	2.216	0.330
Research Model 2	CRS efficiency	6.911	0.032 **
	VRS efficiency	5.496	0.064 *

* denotes sig (p-value) < 0.1, ** denotes sig (p-value) < 0.05, *** denotes sig (p-value) < 0.01.

below.

The analyzed results show significant differences in the characteristics of each research organization in the CRS efficiency (CCR model) of Research Model 1 and the CRS efficiency and VRS efficiency (BCC model) of Research Model 2.

These differences result from the various factors in the results by research organization type. For the basic and future leading types, the target of research performance is SCI theses. However, for the public infrastructure types, the targets of research performance are economic effect such as cuts in the budget by value assessment and the number of on-site applications of developed technologies. It could be the same for the industrial type such as the amount of royalty income and the number of patents and their registration. Therefore, efforts are needed to devise policy depending on performance to improve the efficiency of each research organization’s royalty system.

5. Conclusions and Future Research

To evaluate the comparative efficiency of the royalty systems of 23 GRIs, the DEA method, a common way of analyzing the efficiency of the public sector, was utilized. Considering that almost no DEA research exists to analyze the royalty systems of industrial-academic organizations, it is important to note that this study has broadened the DEA model’s applicability.

The results of the empirical analysis are as follows. First, to examine if the composition ratio of the mandatory expenditure item was efficient, the analysis result of Research Model 1 showed that five organizations in the CCR analysis model and 14 in the BCC analysis model had 100 percent efficiency. According to the results of Research Model 2, which is based on the expenditures of items other than mandatory ones, 17 organizations in the CCR analysis model and 18 in the BCC analysis model showed 100 percent efficiency. Additionally, it was possible to distinguish between effective and ineffective organizations according to the research model and to identify potential improvements and benchmarking DMUs for ineffective organizations.

Second, as a causal analysis of efficiency, multiple regression analysis in Research Model 2 showed the factors influencing improvements in technology transfers and royalty. To boost technology transfer fees, R&D reinvestment cost and the accumulation of contributors’ fees must be reduced, and the focus needs to be shifted toward operating expenses, IPR registration fees and compensation for technological contributors. In addition, for improvement of licensing royalties, more expenditures for IPR registration, compensation for those helping distribute the technologies and the others category are needed.

Third, the analysis of differences in efficiency by research organization type found a difference in efficiency according to organization type in the CRS efficiency of Research Models 1 and 2. To improve the efficiency of the royalty system by research organization type, a variety of policies and efforts should be made to meet specific targets.

Furthermore, the results of this study can serve as important reference material in helping to overcome the limitations of technology transfer and commercialization.

We are aware of the limitations and shortcomings of this research. First, since the data used for the study were limited to royalty data from 2013–2015, generalization errors in the results could occur. Analysis of the ensuing change in efficiency can be another research subject. Second, since interpretation can vary due to the selection of input and output variables, which is a quality of the DEA model, varying efficiency values could be obtained due to changes in variables. In addition, this study focused on analyzing the relations between the input and output factors of GRIs only. To find the process of the realization of GRIs’ technologies through the private sector, the study should be more dynamic with more dimensions such as relationships among the government, GRIs, academia and industry.

Lastly, this study was limited to an evaluation of the comparative efficiency of the GRIs’ royalty system, but further studies on improving the system and stimulating methods of R&D foundation at universities should be conducted with other meticulous analysis models and methods. In this way, the role of the GRIs will expand and grow stronger.