Best Practices in Knowledge Transfer: Insights from Top Universities

Abstract

The impact of knowledge transfer induced by universities on economy, society, and culture is widely acknowledged; nevertheless, this aspect is often neglected by university rankings. Here, we considered three of the most popular global university rankings and specific knowledge transfer indicators by U-multirank, a European ranking system launched by the European Commission, in order to answer to the following research question: how do the world top universities, evaluated according to global university rankings, perform from a knowledge transfer point of view? To this aim, the top universities have been compared with the others through the calculation of a Global Performance Indicator in Knowledge Transfer (GPI KT), a hierarchical clustering, and an outlier analysis. The results show that the universities best rated by global rankings do not always perform as well from knowledge transfer point of view. By combining the obtained results, it is possible to state that only 5 universities (Berkeley, Stanford, MIT, Harvard, CALTEC), among the top in the world, exhibit a high-level performance in knowledge transfer activities. For a better understanding of the success factors and best practices in knowledge transfer, a brief description of the 5 cited universities, in terms of organization of technology transfer service, relationship with business, entrepreneurship programs, and, more generally, third mission activities, is provided. A joint reading of the results suggests that the most popular global university rankings probably fail to effectively photograph third mission activities because they can manifest in a variety of forms, due to the intrinsic and intangible nature of third mission variables, which are difficult to quantify with simple and few indicators.

1. Introduction

It is widely recognized that universities contribute to the social, economic, and cultural development of the regions in which they operate [1,2,3,4,5,6,7]. They are devoted to teaching and research activities, but also to carry on the so called “third mission”, portrayed as a contribution to society [8,9,10,11,12].

Although the “third mission” considers a wide set of activities that can be classified into the social, enterprising, and innovative dimensions [13,14], the focus of this paper is devoted to the more traditional activities related to Knowledge Transfer (KT) [15,16,17,18,19]. KT aims to maximize the two–way flow of technology, Intellectual Property, and ideas. On one side, research organizations advance in research and teaching; on the other side, the business and public sector drive innovation, a process that leads to overall economic and social benefit [20,21]. It is an essential source of innovation and a mechanism for the dissemination of research results [22] that has changed significantly in the past year from the more traditional concept of commercialization and monetization towards a “more rounded approach which supports both co–creation and the dissemination of research results with, and to, non–academic third parties” [22].

Today KT is a straightforward activity that includes research collaboration with industry, patenting, consultancy activity, licensing of new technologies, startup and spinoff creation, entrepreneurship programs, incubation, and more [23]. For this aims European Commission has defined a number of policies, to promote KT through different channels, from Intellectual Property (IP) management and citizen engagement to academia-industry collaborations.

Furthermore, previous research has shown that the impact of research on the economy and the society can come from intentional and unintentional mechanisms [24]. The KT process, defined as the movement of know-how, technical knowledge, or technology from one organizational setting to another [25], represents the intentional way, which implies transaction costs to transfer technology from one organization to another. Unintentional mechanisms include positive externalities and other types of spillovers [26,27] without associated transaction costs.

In this complex scenario, while performance indicators relative to research (in terms of quality of publications and number of citations) and teaching (in terms of student-to-staff ratio, student evaluation, etc.) are widely known and used, less is known about how KT can be characterized and evaluated [23,28]. Furthermore, most of the best-known global university rankings completely lack instruments to evaluate KT activities [29,30]. Recently the European Commission [31] has invested a significant effort in producing the U-Multirank (UMR) ranking [32]. It is based on a different approach compared with the existing global university rankings and furthermore it includes a set of indicators focused on KT.

The research question addressed in this paper is: how do the world’s top universities, evaluated according to global university rankings, perform from a KT point of view? Accordingly, we:

-

Identify and analyze three of the best-known global university rankings in order to identify the world’s top universities and, at the same time, evaluate the coherence between rankings;

-

Search and select a set of specialized KT indicators for evaluating the world top universities from the KT point of view;

-

Verify if the world’s top universities, according to the global rankings, continue to best perform from KT point of view.

The paper is organized as follows. Section 1 contains the information about the global university rankings analyzed in the paper (Academic Ranking of World Universities (ARWU), QS World University Rankings^® (QSWUR) and Times Higher Education World University Rankings (THEWUR)) and the reference to U-Multirank with its specific knowledge transfer indicators. Section 2 illustrates the fourth-steps methodology used in terms of Global Performance Indicator in Knowledge Transfer (GPI KT), for the top 10 universities and the top 100 universities in the rankings, the hierarchical clustering and outlier analysis. Section 3 presents the methodology application and the results obtained, subdivided into different steps, which are the classification of the world top 10 Universities, their classification with respect to U-Multirank KT indicators, the performance of universities according to GPI KT, the hierarchical clustering results, and the outlier analysis results. Section 4 contains discussion of the results and a description of the “successful cases”, i.e., the organization of technology transfer service, relationship with business, entrepreneurship programs and, more generally, third mission activities adopted by the 5 best universities. Section 5 illustrates the conclusion of the work.

1.1. Global University Rankings

Nowadays, rankings permeate multiple sectors and address multiple dimensions of individual and organizational behavior [33]. There is a wide range of public measures such as different types of ratings, benchmarks, and rankings, especially for universities [34,35]. Broader access to higher education has resulted into an increasing demand for information on academic quality and has led to the development of university ranking systems or league tables in many countries around the world [36,37]. The increase in the relevance of rankings of academic institutions is a fairly recent phenomenon that has emerged since the late 1980s [38], primarily because prospective students wanted to be informed about academic quality [36]. However, rankings have been increasingly used to compare and quantify success far beyond the question of student choice, and now influence researchers, employers, public opinion, and, most importantly, academic evaluators, governments, and companies [39,40,41,42,43]. They have become a tool for promoting the growth of universities in the international context, increasing their competitiveness, enhancing the attractiveness of the educational and research system [44], namely a true marketing, benchmarking, and branding tool [29].

Although the scientific literature has already highlighted the presence of some critical aspects related to university rankings, such as the inhibition of regional contributions from universities [45], in recent years university rankings have been able to create impact [42,46] and influence higher education, policy, and public opinion [47,48].

Numerous global university rankings have been proposed so far, they are based on different parameters and indicators [41,49,50] and try to provide an all-round evaluation. Some of the most well-known by insiders and popular are the following:

• The Academic Ranking of World Universities (ARWU) was first published in June 2003 by the Center for World-Class Universities (CWCU), Graduate School of Education (formerly the Institute of Higher Education) of Shanghai Jiao Tong University, China, and updated on an annual basis. Since 2009 the Academic Ranking of World Universities (ARWU) has been published and copyrighted by Shanghai Ranking Consultancy, a fully independent organization on higher education intelligence that is not legally subordinated to any universities or government agencies. ARWU uses six objective indicators to rank world universities, including the number of alumni and staff winning Nobel Prizes and Fields Medals, the number of highly cited researchers selected by Clarivate Analytics, the number of articles published in journals such as Nature and Science, the number of articles indexed in Science Citation Index—Expanded and Social Sciences Citation Index, and the per capita performance of a university. More than 1800 universities are ranked by ARWU every year and the best 1000 are published [51].
• The QS World University Rankings^® (QSWUR) lists and ranks over 1000 universities from around the world, covering 80 different locations; it continues to rely on a remarkably consistent methodological framework, compiled using six simple metrics that effectively capture university performance. Universities are evaluated according to the following six metrics: Academic Reputation, Employer Reputation, Faculty/Student Ratio, Citations per faculty, International Faculty Ratio, International Student Ratio [52].
• The Times Higher Education World University Rankings (THEWUR) includes almost 1400 universities across 92 countries, standing as the largest and most diverse university rankings ever to date. It is based on 13 carefully balanced and comprehensive performance indicators and is trusted by students, academics, university leaders, industry, and governments. Its performance indicators are grouped into five main areas: teaching (the learning environment); research (volume, income, and reputation); citations (research influence); international outlook (staff, students, and research) and industry income (knowledge transfer) [53].

1.2. U-Multirank 2020 (UMR)

Very often the global university rankings have been criticized [29,37,40,50] because, for example, using a single set of indicators, they compare different types of institutions [42], evolving from a “semi-academic exercise” to an international business tool [41] and an important “instrument for the exercise of power” [48]. To overcome these limitations, in the last few years, other rankings have been proposed. The most important and popular is UMR, launched by the European Commission in collaboration with Bertelsmann Foundation and Banco Santander, based on the results of a feasibility study covering 150 universities which was carried out in 2010/11 [54].

UMR is based on a different approach compared with the existing global university rankings. It compares university performances considering different activities that they are engaged in, taking into account the diversity of the higher education sector and the complexity of evaluating educational performance [32,55,56]. UMR has been developed based on a number of design principles, user-driven, multidimensionality, comparability, multilevel nature of higher education, and methodological soundness [57]. It is considered a transparency tool for higher education stakeholders [58] and takes into account five aspects and dimensions of the universities’ performance: (1) teaching and learning, (2) research, (3) knowledge transfer, (4) international orientation and (5) regional engagement. The UMR web tool allows users to compare universities but also study programs. Based on empirical data, it compares institutions with similar profiles (‘like-with-like’) and allows users to develop their own personalized rankings by selecting indicators in terms of their own preferences. Each of the five aspects evaluated by UMR are ranked from A to E, with A and E indicating “very good” and “weak” performance, respectively.

In this paper, the choice to use of UMR was motivated by the fact that it uses a focused quality model for an in-depth evaluation of KT dimension based on the following indicators [59]:

• Co-publications with industrial partners: the percentage of a department’s research publications that list an author affiliated with an address that refers to a for-profit business enterprise or private sector R&D unit (excluding for-profit hospitals and education organizations).
• Income from private sources: the percentage of external research revenues (including not-for-profit organizations) coming from private sources, excluding tuition fees. Measured in €1.000s using Purchasing Power Parities and computed per FTE (full time equivalent) academic staff.
• Patents awarded (absolute numbers): the number of patents assigned to inventors working at the university in the respective reference period.
• Patents awarded (size-normalized): the number of patents assigned to inventors working at the university over the respective reference period, computed per 1.000 students to take into consideration the size of the institution.
• Industry co-patents: the percentage of the number of patents assigned to inventors working at the university during the respective reference period, which were obtained in cooperation with at least one applicant from the industry.
• Spinoffs: the number of spinoffs (i.e., firms established on the basis of a formal KT arrangement with the university) recently created by the university (computed per 1000 FTE academic staff).
• Publications cited in patents: the percentage of the university’s research publications that were cited in at least one international patent (as included in the PATSTAT database).
• Income from continuous professional development: the percentage of the university’s total revenues that is generated from activities delivering Continuous Professional Development courses and training.
• Graduate companies: the number of companies newly founded by graduates and computed per 1000 graduates.

2. Methodology

The research question (RQ) addressed in the paper is: how do the world top universities, evaluated according to global university rankings, perform from a knowledge transfer point of view?

And after having answered to the RQ, the final goal is to identify the best practices in knowledge transfer that can be adopted by universities that want to improve their performances.

In order to answer to RQ the methodology described in this section (Figure 1) was defined and followed.

The first step was to identify and analyze three of the best-known global university rankings in order to extract the world top universities and, at the same time, evaluate the coherence between rankings.

The rankings selected are the Academic Ranking of World Universities (ARWU), the QS World University Rankings^® (QSWUR) and the Times Higher Education World University Rankings (THEWUR), as described in the previous section.

Each ranking has its own specificities; thus, to define “top universities”, we considered the union T_k of all three rankings in 2020, so that:

$$T_k=T_k^{\mathit{ARWU}}\cup T_k^{\mathit{QSWUR}}\cup T_k^{\mathit{THEWUR}}$$

(1)

where T_k^ARWU, T_k^QSWUR and T_k^THEWUR are the top k universities in the rankings QRWU, QSWUR and THEWUR.

The sets T₁₀ and T₁₀₀ were then determined and furthermore the coherence between rankings was tested by using Spearman’s correlation among the top 100 positions in QRWU, QSWUR and THEWUR.

The second step was to search and select a set of specialized KT indicators for evaluating the world top universities from the KT point of view. For this aim, the UMR2020 previously described was used. Among the 9 UMR2020 indicators, the following 5 were selected for being used for the T₁₀ and T₁₀₀ elaboration; this is due to the fact that they had the minimum number (less than 8%) of null value:

(1)

Co-publications with industrial partners

(2)

Patents awarded (absolute numbers)

(3)

Patents awarded (size-normalized)

(4)

Industry co-patents

(5)

Publications cited in patents

The five indicators were quantified for all universities included in T₁₀ and T₁₀₀, and a composite indicator called Global Performance Indicator KT (GPI KT), obtained as the average of the 5 previous indicators, was also defined and used for determining the global performance in knowledge transfer for each university. Then, a comparison was made between the T₁₀ obtained from the global universities ranking and the top performer universities in KT included in T₁₀₀, also by using radar plots for graphically expressing the macroscopic differences.

The goal of this analysis is to verify if the world top universities, according to the global rankings, continue to best perform from KT point of view.

The third step was to investigate the universities included in T₁₀₀ in order to identify groups of similar universities, in terms of KT indicators, through a data-driven approach based on the hierarchical clustering. The goal is to understand from the natural aggregation in groups the presence of common characteristics capable of explaining the different levels of performance in knowledge transfer. Hierarchical clustering algorithms allow to group similar items in an unsupervised way [60]. Compared with optimization-based clustering methods, such as K-means [61], this particular class of algorithms follows an alternative approach that entails the advantages of being deterministic and not requiring to fix the number of clusters a priori. An agglomerative hierarchy linkage algorithm was used, which starts by considering each point, corresponding to a data vector, as a cluster, and proceeds by iteratively merging the closest pairs of clusters, until ending up with one cluster that includes all data points. Vicinity of two points $i$ and $j$ is quantified by their Euclidean distance $\left|\right|i-j|{|}_2$, while the distance between clusters $A$ and $B$ is evaluated as $d\left(A,B\right)=\underset{i\in A,j\in B}{\min }||i-j|{|}_2,$ namely the minimum distance between points in the two clusters. The algorithm used allows to obtain dendrograms, which can help with the interpretation of the results.

As a last step, we tried to understand how the universities included in T₁₀ perform if compared with the others in T₁₀₀ for each of the 5 UMR2020 indicators. This is in order to understand from the KT perspective what are the strengths of those universities that the global rankings identify as the best performers. For all the universities in T₁₀₀ and for each of the 5 UMR 2020 indicators, the median absolute deviations (MAD) criterion [62] was used in order to compare each university in T₁₀ with the distribution of the remaining universities in T₁₀₀. In details, we computed the scaled MAD factor for each KT indicator as:

$$\mathrm{MAD}= \mathrm{c}· \mathrm{median} \left(\left|\mathrm{T}100-\mathrm{median}\left(\mathrm{T}100\right)\right|\right),$$

(2)

where c = −1/($\surd 2$·erfcinv(3/2)); then, we determined whether each university in T₁₀ represented an outlier for the distribution of items in T₁₀₀, considering three scaled median absolute deviations (MAD) away from the median as the threshold for outlier detection [63,64].

3. Methodology Application and Results

In this section the step-by-step application of the methodology presented in Section 2 is described together with the obtained results.

3.1. First Step

The first step of the methodology is to identify the set T₁₀ and T₁₀₀. For k = 10 the set T₁₀ results to be the union of the top ten universities included in each of the selected rankings: QRWU, QSWUR and THEWUR (

Table 1. The University Ranking: World Top Ten Universities.

	ARWU		QSWUR		THEWUR
Rank	University	Country	University	Country	University	Country
1	Harvard University	US	Massachusetts Institute of Technology (MIT)	US	University of Oxford	UK
2	Stanford University	US	Stanford University	US	California Institute of Technology (CALTECH)	US
3	University of Cambridge	UK	Harvard University	US	University of Cambridge	UK
4	Massachusetts Institute of Technology (MIT)	US	University of Oxford	UK	Stanford University	US
5	University of California, Berkeley	US	California Institute of Technology (CALTECH)	US	Massachusetts Institute of Technology (MIT)	US
6	Princeton University	US	ETH Zurich—Swiss Federal Institute of Technology	CH	Princeton University	US
7	Columbia University	US	University of Cambridge	UK	Harvard University	US
8	California Institute of Technology (CALTECH)	US	University College London (UCL)	UK	Yale University	US
9	University of Oxford	UK	Imperial College London	UK	University of Chicago	US
10	University of Chicago	US	University of Chicago	US	Imperial College London	UK

). Similarly, the T₁₀₀ was determined, according to Equation (1).

Thus T₁₀ = {California Institute of Technology (CALTECH), Columbia University, ETH Zurich—Swiss Federal Institute of Technology, Harvard University, Imperial College London, Massachusetts Institute of Technology (MIT), Princeton University, Stanford University, University College London (UCL), University of California, Berkeley, University of Cambridge, University of Chicago, University of Oxford, Yale University} included 14 universities.

The coherence between rankings, or their level of agreement, was evaluated by using Spearman’s correlation among the top 100 positions in each ranking. Spearman’s rank correlation coefficient or Spearman’s ρ is a nonparametric measure of rank correlation [65]. It is used to assess how well the relationship between two variables can be described using a monotonic function. The obtained Spearman’s correlations are: 0.89 between ARWU and QSWUR; 0.91 between ARWU and THEWUR; 0.95 between QSWUR and THEWUR. Therefore, all rankings are strongly correlated and thus they exhibit an underlying coherence.

3.2. Second Step

In the second step a set of specialized KT indicators for evaluating the world top universities from the KT point of view were identified. Starting from UMR2020, 5 indicators were selected:

(1)

Co-publications with industrial partners

(2)

Patents awarded (absolute numbers)

(3)

Patents awarded (size-normalized)

(4)

Industry co-patents

(5)

Publications cited in patents.

The indicators are expressed on percentage measurement scale except for (2) Patents awarded (absolute numbers) and (3) Patents awarded (size-normalized). Thus, they have been reported on a percentage measurement scale in order to be comparable with the others and thus allows to compute mathematical operation. Furthermore, a composite index given by the arithmetic mean of the five indicators of the UMR2020 dataset was introduced. It can be considered a global performance index of the KT (KT GPI) actions of the universities.

Table 2. Summary of UMR2020 KT indicators for T₁₀ Universities.

	Co-Publications with Industrial Partners	Patents Awarded (Absolute Number)	Patents Awarded (Size-Normalized)	Industry Co-Patents	Publications Cited in Patents	GPI KT
University of California, Berkeley (UCB)	7.10%	98.94%	2.00%	10.02%	2.30%	24.07%
Harvard University	8.00%	66.12%	1.45%	8.26%	3.30%	17.43%
Massachusetts Institute of Technology (MIT)	10.50%	57.72%	4.04%	5.86%	4.90%	16.60%
Standford University	9.40%	40.63%	1.68%	11.13%	2.80%	13.13%
California Institute of Technology (CALTEC)	7.60%	29.42%	10.88%	7.33%	2.10%	11.47%
ETH Zurich—Swiss Federal Institute of Technology	8.70%	6.94%	0.29%	38.91%	2.00%	11.37%
University of Cambridge	8.20%	4.86%	0.22%	25.59%	1.70%	8.11%
University of Chicago	6.80%	4.89%	0.24%	20.76%	2.00%	6.94%
Columbia University	7.80%	19.97%	0.54%	4.23%	1.70%	6.85%
University College London (UCL)	7.40%	8.30%	0.18%	14.88%	1.50%	6.45%
Yale University	6.70%	8.20%	0.51%	11.97%	2.10%	5.90%
University of Oxford	7.10%	6.38%	0.22%	12.75%	1.80%	5.65%
Imperial College London	10.20%	4.17%	0.20%	10.95%	1.90%	5.48%
Princeton University	7.30%	5.88%	0.61%	11.11%	1.20%	5.22%

shows, in descending order, how the universities in T₁₀ perform. The best performer results to be the University of California, Berkeley (UCB). In general, for the first indicator, co-publication with industrial partners, the Massachusetts Institute of Technology (MIT) and the Imperial College London are the best performers. The indicator “Patents awarded as absolute numbers” reveals that University of California, Berkeley is the best performer followed by the Harvard University, while if the indicator “Patents awarded (size-normalized)” is considered, the top university is represented by California Institute of Technology (CALTECH) followed by MIT. For Industry co-patents, the ETH Zurich—Swiss Federal Institute of Technology is the best and then the University of Cambridge; the last indicator, “Publications cited in patents”, shows that MIT and Harvard University are the most important.

It is also interesting to investigate how the 14 universities in T₁₀ perform when compared with other universities. According to Equation (1), it results that card (T₁₀₀) = 151 and after having deleted all the entries that contain missing data or null values card (T₁₀₀) = 123. For each of the universities in T₁₀₀, the five UMR2020 indicators of Table 2 were calculated together with the composite indicator GPI KT. Finally, the list was sorted in descending order. It is interesting at this point to verify how the universities in T₁₀ are ranked compared with those in T₁₀₀ and if they are still among the best performing ones. The results obtained are surprising. Only 2 of the universities included in T₁₀, namely Berkeley and Harvard, are placed in the first 14 positions while the remaining ones appear to drop a lot in the ranking (

Table 3. Performance of universities in T₁₀ according to GPI KT.

	Position in T10	Position in T100 according to GPI KT
University of California, Berkeley (UCB)	1	6
Harvard University	2	14
Massachusetts Institute of Technology (MIT)	3	15
Stanford University	4	26
California Institute of Technology (CALTEC)	5	30
ETH Zurich—Swiss Federal Institute of Technology	6	31
University of Cambridge	7	45
University of Chicago	8	62
Columbia University	9	65
University College London (UCL)	10	71
Yale University	11	79
University of Oxford	12	84
Imperial College London	13	88
Princeton University	14	93

). Massachusetts Institute of Technology (MIT) appears in position 15.

Table 4. Best performer universities in T₁₀₀ according to GPI KT.

	Co-Publications with Industrial Partners	Patents Awarded (Absolute Number)	Patents Awarded (Size-Normalized)	Industry Co-Patents	Publications Cited in Patents	GPI KT
Tsinghua University	5.60%	42.18%	29.09%	82.28%	1.20%	32.07%
University of California, San Diego (UCSD)	10.60%	98.98%	2.34%	10.02%	2.70%	24.93%
University of California, Santa Barbara	8.20%	98.94%	3.30%	10.02%	2.30%	24.55%
University of California, Los Angeles (UCLA)	8.00%	100.00%	1.85%	10.15%	2.50%	24.50%
Boston University	7.90%	4.46%	100.00%	6.85%	1.80%	24.20%
University of California, Berkeley (UCB)	7.10%	98.94%	2.00%	10.02%	2.30%	24.07%
University of California, Davis	6.70%	98.94%	2.19%	10.02%	1.70%	23.91%
The University of Tokyo	8.30%	19.11%	19.36%	70.63%	1.90%	23.86%
Seoul National University	8.10%	28.13%	28.27%	46.52%	1.80%	22.56%
Weizmann Institute of Science	4.90%	6.74%	89.68%	5.12%	3.50%	21.99%
Kyoto University	8.80%	12.93%	17.01%	57.71%	1.80%	19.65%
Tokyo Institute of Technology (Tokyo Tech)	11.10%	7.24%	3.62%	72.17%	1.70%	19.17%
Tohoku University	10.20%	13.98%	0.69%	68.89%	1.60%	19.07%
Harvard University	8.00%	66.12%	1.45%	8.26%	3.30%	17.43%

instead shows the details of the five UMR2020 indicators and the GPI KT for the first 14 universities included in T₁₀₀.

It is interesting to note how the Californian universities and, to a lesser extent, the Japanese and some Asian ones stand out in the KT even though they are not among the best if we consider the traditional global university rankings.

The radar plot in Figure 2 shows how the KT performances of the universities in T₁₀ are significantly different from those of the best universities in T₁₀₀. The best performer universities in KT have a recognized technological vocation.

3.3. Third Step

In the third step, the universities included in T₁₀₀ are investigated in order to identify groups of similar universities, in terms of KT indicators, by using the hierarchical clustering. The application of the hierarchical clustering algorithm defined provide n.4 clusters, organized as described in

Table 5. The clusters obtained on T₁₀₀.

N. cluster	Universities Contained into the Cluster
1	University of California, Santa Barbara University of California, Los Angeles (UCLA) University of California, Berkeley (UCB) University of California, Davis University of California, San Diego (UCSD) Harvard University KAIST—Korea Advanced Institute of Science &Technology
2	KU Leuven University of Toronto Boston University Weizmann Institute of Science The University of Queensland Université Grenoble Alpes The University of New South Wales (UNSW Sydney) The University of Melbourne University of Copenhagen Aarhus University The Hong Kong University of Science and Technology Sungkyunkwan University (SKKU) The University of Tokyo Korea University Kyoto University Seoul National University Tsinghua University
3	Stanford University California Institute of Technology (CALTEC)
4	Universities in the rest of the world

.

In Figure 3 the dendrograms resulted by the analysis are reported.

Of the top 14 universities within the 3 identified clusters (not considering the fourth—the rest of the world) we find only 4/14 that is the 29%. Specifically: MIT is absent, such as ETH, Imperial college London, Cambridge University, Princeton University, University of Chicago, Columbia University, Yale University, University of Oxford, University College London. Very interesting the creation of a unique cluster formed by Stanford University and CALTEC (cluster n.3). The emergence of a cluster (n.1) consisting, to the extent of 71%, of California universities, together with Harvard University and the Korean KAIST, appears very characteristic and in part coherent with the result obtained in Section 3.2. The clustering results to be in line with the ranking made in Table 4, in agreement with the GPI KT. In fact, the top best performers are all clustered between cluster #1 and #2. The emergence of California universities in both analyses would lead to the assertion that the industrial, social, and economic environment itself can affect the third mission activities of these universities.

3.4. Fourth Step

By using the median absolute deviations (MAD) criterion defined in formula (2), the aim of the step four is to understand how the universities included in T₁₀ perform if compared with the others in T₁₀₀ for each of the 5 UMR2020 indicators.

Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8 help show the difference between the MAD value of each KT indicator for the 14 top universities in T₁₀ and the outlier threshold: a positive difference is indicated with a red bar and denotes that the corresponding university is classified as an outlier with respect to the distribution of the specific KT indicator of the T₁₀₀ universities.

In Figure 4, with respect to the “Co-publication with industrial partners” indicator, 7 (bar in red) over 14 universities in T10 perform much better than the remaining ones. Figure 5, which refers to the indicator named “Patents awarded (absolute numbers)”, points out that only 3 universities outperform with respect to the remaining ones. Finally, Figure 8 show the result for the “Publications cited in patents” indicator and points out that only 4 universities in T10 outperform. The Figure 6 and Figure 7 show no outperformers universities.

4. Discussion

The results obtained are non-trivial and, in some cases, surprising. The clustering analysis in Section 3.3 suggests that the top 14 universities in the world (according to global university rankings) are not an isolated group compared with the others in terms of knowledge transfer and, more in general, third mission activities. With exception of Stanford University and CALTEC, which represent a separate cluster, and Harvard and Stanford present in the California cluster, all the other best universities do not “stand out” from the others, considering the available KT indicators.

The MAD based analysis carried out in Section 3.4 reveals that three indicators appear to be more suitable to measure the performance of universities in terms of KT activities: “Co-publications with industrial partners”, “Patents awarded (absolute numbers)”, “Publications cited in patents”. This is a very useful result since these aspects have been relatively under-investigated so far [66]. The relevance of this result needs to be interpreted in the context of similar findings presented in the scientific literature from KT perspective.

With respect to “Co-publications with industrial partners”, it can be observed that, in recent years, due to collaborations between academia and industry, which represent a great support in the research, development and innovation in the framework of the third mission [67] has grown more and more [68], taking different forms and returning different outputs that are sometimes difficult to measure [69].

Several indicators can be used to measure the outputs of the university-industry collaboration, such as patents/intellectual property rights, publications, and learning metrics [70]. Among these, the analysis of the co-publications with industrial partners is used as an explicit proxy for evaluating university-industry collaboration [69,71,72,73,74] and for studying the university’s entrepreneurial orientation [75]. The university–industry co-publications, which are jointly authored by university researchers and staff employed by business enterprises, can also help to solve the issue related to the insufficient publicly available information (number of research contracts, number, and type of joint projects with industry, number of licenses) [76].

Some authors demonstrated also that the university–industry co-publications have a significant positive influence on universities’ technology commercialization outputs in terms of patenting, spin-off formation, and technology licensing [77].

In the past it has been often thought that a publication carried out with the company is of less value than a publication carried out with academic partners; however, using a field-weighted citation impact metric around the globe, it has been demonstrated that publications made with a company have a higher citation impact (“The urban myth of less respect for collaborating with industry is busted”) [78].

However, co-publications with industrial partners, as well as other quantitative measures, are still a long way from being regarded as perfect measures of university-firm collaboration [76]. Considering only the publication count is not a reliable tool to estimate the success of university-industry cooperation [79] and not all research collaborations lead to co-publication [80].

Finally, co-publications with industrial partners offer an interesting new source of data for the evaluation of the collaboration between industry and academia, but it should be used for “domestic and international comparisons of research universities, only within non-evaluative multi-dimensional benchmarking frameworks rather than for university league tables” [72].

For what concerns “Patent awarded”, it has always been used as an indicator of invention [81], a valuable estimator of technology development [82], an indicator of KT outputs, or a signal of capability of exploitation and commercialization of research results [83]. As Eurostat reports: “a count of patents is one measure of a country’s inventive activity and also shows its capacity to exploit knowledge and translate it into potential economic gains” [84]. Although patents have grown over time and are a tool to create economic profit, there is still a disproportionately small number of real cases of technology transfers [85]. The universities that stand out for this indicator are generally small and private, such as Harvard and MIT, or very specialized and home of numerous Nobel Prize winners as Berkeley.

Lastly, concerning the “Publications cited in patents” indicator, some studies stated that they play a crucial role in establishing a link between science and technology [86] or, in other words, a representation of the knowledge flows between coded scientific knowledge (scientific papers) and coded technological knowledge [87]. The indicator “Publications cited in patents”, therefore, becomes an increasingly used statistical parameter [88] that can be interpreted as a measure of KT efficiency, i.e., the pipeline from research to exploitation to market.

Excluding Patent awarded (size-normalized) that do not allow the distinction between the 14 top universities and the others in the rankings, the other indicator, Industry co-patents, probably suggests that patenting in partnership between universities and businesses is not the most useful and efficient form of collaboration. Industry co-patents is an indicator quite controversial. There are several reasons for which patenting is not considered an optimal KT indicator: the poor understanding of the needs of the market by academics, their need for publication (publish or perish), very cumbersome academic procedures for patenting, unrealistic royalties, the company’s necessity to take high risk and large investments to bring the technology to the market [89]. Li et al. stated that the co-ownership has a negative impact on patent commercialization: industry-academia patents are less probable to be commercialized [90]. Cerulli et al. focalized their attention on the impact of academic patents on firm’s performance, and they stated that there is a positive impact on market power but a lower profitability [91]. Certainly, academic scientific research is beneficial to industry because allows it to enlarge their capability to explore and develop new solutions and technological fields [92]; however, there is still dramatically limited empirical evidence on the impact of academic patents on business performance [91].

Attempting to answer the research question “how do the world’s top universities, evaluated according to global university rankings, perform from knowledge transfer point of view?”, by analyzing the five knowledge transfer indicators obtained from UMR2020 dataset, and combining the results obtained by GPI KT calculation, the clustering analysis, and the MAD based analysis, it is possible to claim that:

-

According to the results in Section 3.2, only 4 universities in T₁₀ are clearly present in the identified clusters: Stanford University, CALTEC, University of Berkeley (UCB) and Harvard University

-

As emerge from Section 3.4, only 2 university over 14 (MIT, Harvard) result to be stand out with respect to the remaining ones if all the three indicators, “co-publications with industrial partners”, “patent awarded (as absolute number)” and “publications cited in patents”, are jointly used;

-

By combining the two previous results, it is possible to state that only 5 over 14 universities in T₁₀ (Berkeley, Stanford, MIT, Harvard, CALTEC) exhibit a high-level performance (they are included in the first 30 position over the 123) in KT if compared with the remaining universities in T₁₀₀;

-

A joint reading of the results obtained in Section 3 does not return a coherent and clear interpretation. The third mission and the process of knowledge transfer that takes place toward the territory can manifest itself in a variety of forms that are difficult to quantify with simple indicators. The complexity and the many facets of KT are such that they cannot be reduced to a fewer significant elements and that probably the quantitative data collected need to be integrated with qualitative data and context data.

-

The most popular global university rankings probably fail to effectively photograph third mission activities. They certainly do the best in the case of the other university missions: teaching and research.

In the following, for better understanding the success factors and best practices for improving knowledge transfer performances, a brief description of the 5 cited universities, in terms of organization of technology transfer service, relationship with business, entrepreneurship programs and, more generally, third mission activities, is provided.

As a general consideration they seem to be able to connect research results and market, and to incentivize businesses and stakeholders to collaborate with them. In other words, as showed in Figure 9, they have development organization structures, methods, and approaches for survive in the so-called Valley of Death [93], which represents “the gap between where publicly available research funding stops and where private investment or commercial funding starts” [94]. They represent successful cases to be imitated, and an inspiring collection of best practices to be adopted.

Harvard University. At Harvard University, the Office of Technology Development (OTD) connects innovators with industry partners, giving support to researchers/innovators in the advancement of their research through corporate partnerships, collaborations, and accelerator programs; the Office helps with the protection of IP to create a clear path forward for commercial development, with business development strategies for licensing or for the creation of new companies. The Office also provides support for industry partners by offering a single point of entry for engaging with Harvard researchers, accelerators, technology licensing, and new ventures. In order to bridge development gaps, Harvard University offers Accelerator programs that combine funding strategies, technical support, and business expertise to help promising innovations make the leap from the lab to the commercial sphere.

Regarding entrepreneurship, OTD’s Entrepreneurs in Residence (EIRs) engage directly with Harvard research groups to help advance technologies toward the launch of a startup.

Finally, the section “Impact” reports the number of new startup companies with their impact on society in terms of education, health care, food and agriculture, energy, sustainability, high-tech goods, and much more.

Massachusetts Institute of Technology. The Massachusetts Institute of Technology (MIT), one of the most vibrant hubs of innovation and entrepreneurship on Earth, dedicates a large part of its web site to the topic “Innovation”. MIT’s TLO (Technology Licensing Office) is engaged in the cultivation of an inclusive environment of scientific and entrepreneurial excellence and bridges connections from MIT’s research community to industry and startups by strategically evaluating, protecting, and licensing technology.

A separate website is dedicated to MIT’s Industrial Liaison Program (ILP) that is “industry’s most comprehensive portal to MIT, enabling companies world-wide to harness MIT resources to address current challenges and to anticipate future needs”. Nowadays more than 800 of the world’s leading companies collaborate with MIT researchers and together bring knowledge to bear on the world’s great challenges.

MIT Corporate Relations, the organizational parent of the ILP at MIT, is dedicated to finding connections to MIT faculty, departments, labs, and centers.

Great emphasis is placed on entrepreneurship with Martin Trust Center for MIT Entrepreneurship that seeks to advance knowledge and educate students in innovation-driven entrepreneurship by providing proven frameworks, courses, programs, facilities, and mentorship, and with the Program “Entrepreneur in Residence (EIR)”, a centerpiece of the Trust Center, where accomplished business leaders advise students on the challenges and benefits of startup life. MIT Startup Exchange is a program of MIT Corporate Relations that actively promotes collaborations and partnerships between MIT-connected startups and industry, principally ILP members.

Stanford University. At Stanford University, the Office of Technology Licensing (OTL) receives invention disclosures from Stanford faculty, staff, and students and evaluates them for their commercial possibilities, and, when possible, they also license them to industry. The office supports researchers providing numerous guides, for instance the “Inventor’s Guide” or the “Researcher’s Guide to Working with Industry”.

Great relevance is given to the concept of IMPACT; in fact, every year, OTL drafts an annual report where the number of issued patents, executed technology licenses, formed startups, and the amount of license income generated are reported.

The office “University Corporate and Foundation Relations” is a central university office that helps to foster relationships between Stanford University, companies, and private professional foundations. For corporations, there are engagement opportunities to collaborate with Stanford University, to connect to and recruit students, and to get executive education.

In the framework of Professional Education, Stanford University has the “Innovation and Entrepreneurship (SI&E) Certificate Program” to learn innovation and entrepreneurship as practiced at Stanford and in the Silicon Valley, and the “Stanford Idea-to-Market (I2M) course” to learn tools, techniques, and real-world expertise to make a business idea a reality.

University of California, Berkeley. The Office of Intellectual Property and Industry Research Alliances (IPIRA) provides a “one-stop shop” for industry research partners to interact with the campus. IPIRA’s mission is to establish and maintain multifaceted relationships with private companies, and thereby enhance the research enterprise of the Berkeley campus. IPIRA has promulgated technology transfer that generated billions of dollars in revenue and has created IP policies to promote social impact. Noticeably, the “Socially Responsible Licensing Program” serves as the gold standard for universities in the public health space.

About entrepreneurship, UC Berkeley helps students, faculty, researchers, and other innovators access a deep, interconnected ecosystem of resources for educating entrepreneurs, commercializing research, and advancing startups. The office supports entrepreneurs looking for funding, legal services, start-up guidelines, connections with ventures, and so on. The section “Berkeley Startups” provides a partial list of companies born out of licensing UC Berkeley IP rights.

CALTECH

In the California Institute of Technology (CALTECH), the Office of Technology Transfer and Corporate Partnerships (OTTCP) has the mission to drive the transfer of scientific and engineering knowledge created by the researchers to “maximize societal impact by developing partnerships with industry through the creation of new ventures, collaborations with corporations, and transfer of IP while nurturing an entrepreneurial environment” (https://innovation.caltech.edu/ accessed on 20 September 2022). The Istitute’s homepage showcases the following sections: “Corporate Partnerships” for productive collaborations and long-term partnerships with industry partners, in order to accelerate progress towards shared goals; “New Venture Creation & Entrepreneurship” illustrating the support to the formation of startup companies based on Caltech and JPL technologies by maintaining close relationships in the entrepreneurial community; “Patents & Licensing” whose goal is to make the technology transfer process and working with industry as easy as possible.

In a section dedicated to start ups, the program ”Entrepreneurs in Residence” is illustrated, i.e., a path to Launching a Venture, the Caltech Funding Sources, the Entrepreneurship Resources and so on.

Section “Impact”, with pay-off “Pushing interdisciplinary boundaries in the service of discovery”, described the inventions made by Caltech researchers since its founding. In the same page, the “Impact Report” and the “CALTECH Impact” report the outsized impact on science, technology, and society; its numbers convey the extent of the impact generated by research innovations, commercialization activities, and overall output of invention disclosures, patents, licenses, and startup companies. OTTCP teams connect companies, industry leaders, and other financial partners to Caltech and to the Jet Propulsion Laboratory’s research communities and help identify the types of strategic partnerships and opportunities that can advance business and investment goals.

5. Conclusions

This paper proposes a four steps methodology for answering to the following research question: “how do the world top universities, evaluated according to global university rankings, perform from knowledge transfer point of view?” and, as direct effect, to point out the success factors and best practices that can be adopted for improving performances in knowledge transfer.

Starting from the top universities in the most important global universities ranking (QRWU, QSWUR and THEWUR) the paper analyzes how they perform from third mission, knowledge transfer, relationship with business and entrepreneurship perspective. In order to do this, a set of specific KT indicators, obtained from U-multirank 2020, were selected and used to evaluate the performances of the top universities. The comparison carried out shows that the global universities rankings are unable to properly evaluate the knowledge transfer and third mission activities. In other words, the top universities in global ranking are not always as performing from the point of view of the third mission. Only 3 of the top universities in the global rankings come out on top after being evaluated with the specific U-multirank 2020 knowledge transfer indicators. Among the 30 top universities in KT, only 5 of the top T₁₀ universities are present.

The paper, in attempt to identify possible factors that characterize the best universities from the perspective of the KT, investigates the sample through a hierarchical clustering algorithm. The results obtained do not show a particular relationship between the clusters obtained and the top universities of global university rankings. On the other hand, the composition of the clusters is interesting. In some cases, it appears to be based on basis of geographical position (such as the presence of numerous Californian universities in the same cluster), thus suggesting that there are contextual factors that the purely quantitative analyzes used by global university rankings fail to grasp or bring out. Finally, the analysis based on the MAD indicator helps to identify three indicators that best help assess the performance of universities in terms of KT activities: Co-publications with industrial partners, Patents awarded (absolute numbers), Publications cited in patents. The paper also tries to explain, through a targeted bibliographic analysis, why these indicators are interesting and useful for interpreting the performances in the KT.

The paper proposes a small new step forward in the scientific landscape of university assessment of technology transfer and third mission, an assessment that is not easy with the means and indicators that currently exist, as reported by other authors [29,66]. In fact, KT indicators are poorly considered in the most popular global university rankings (the QS World University Rankings, the Academic Ranking of World Universities, and The Times Higher Education) except for U-multirank [32].

Several attempts have been made to develop patterns of KT indicators at the European level [22,94]. Often the difficulty lies in being able to measure only what is measurable, i.e., many activities are not yet measurable and quantifiable such as the KT activities that occur with unintentional mechanisms [24]. For example, learning, contacts, friendship, networks, impact, reputation, and publicity are “non-monetary currencies” [94]. In this regard, there is also the scientific debate about the measurement of the impact and benefits of the KT activities on society (e.g., the UK REF Impact Case Studies).

On the other hand, the challenge of evaluating universities from the third mission point of view is becoming increasingly relevant, since the topic of the impact of the third mission activities and research results on society and the territory is becoming more and more central, even in international policies. In general, governments and politicians are interested in evaluating outcomes of public investment in research, university actors want to demonstrate their contribution to society and different types of stakeholders [22], and industry uses university evaluation in order to assess and determine partnerships.