Universities as an External Knowledge Source for Industry: Investigating the Antecedents’ Impact on the Importance Perception of Their Collaboration in Open Innovation Using an Ordinal Regression-Neural Network Approach

Abstract

Within the highly complex ecosystem of industry-university collaboration in open innovation, three specific antecedents typically characterize the patterns of their interaction, i.e., motivations, barriers, and channels of knowledge transfer. However, an investigation of the extent to which these antecedents of opening up innovation impact the perceived importance of universities as an external knowledge source to the industry is still missing in the literature. Based on a research framework developed from a review of the literature, a two-stage ordinal regression, and neural network approach was performed to investigate this impact. In the first stage, the hypotheses of the proposed research framework were tested based on an ordinal regression, and those antecedents that significantly impacted the importance perception were revealed. In the second stage, an artificial neural network analysis was carried out to capture the complex relationships among the significant antecedents and the important perception of universities as an external knowledge source to the industry. On the whole, the findings of our study expand the existing open innovation literature and contribute to a more articulate view of the collaboration between industry and university in this field by providing a first perspective on which of the three antecedents has a significant impact on this perception and how such an impact can be predicted.

1. Introduction

Nowadays, we operate in a dynamic and globalized economy where complexity increases, networks spread, and interdependencies expand [1]. In such an environment, generating innovation is seen as fundamental to sustaining the advantages that firms have to maintain to remain competitive [1]. At the same time, innovation has been considered knowledge-intensive [2], and relying entirely on internal knowledge may not be enough to succeed in the new landscape of complexity and connectivity. Therefore, firms have to employ external knowledge from diverse sources to support development and innovation [3,4,5,6]. Emphasizing the importance of interaction with other actors to innovate, the concept of open innovation that was introduced at the beginning of this century by Chesbrough [7] has attracted increasing attention in both academic research and industrial practice, reflecting the changes of recent years in terms of technological, organizational, and societal developments [8]. Following the open innovation paradigm [7], a firm’s ability to employ external sources of knowledge becomes of great importance for innovation generation in the context of the turbulent and rapidly changing environment of our times, which requires continuous adaptations and reconfiguration of competencies [9].

The industry receives ideas, knowledge, and technologies from diverse external sources, which have been differently classified in the literature [10,11,12]. Distinctions have often been drawn between industrial and knowledge sources: competitors, suppliers, and clients are included in the first category, while universities, public research centers, research laboratories, and consulting firms are considered in the second one [10,12]. Among these partners, universities have been recognized as a special external knowledge source [13], since they are major providers of new knowledge and technologies [14].

Like any other strategic initiative, opening up boundaries to innovation with universities is related to different antecedents that firms have to deal with [15]. According to the extant literature, e.g., [16], three specific dimensions are commonly employed to capture the patterns of interaction between industry and university collaboration. They are as follows: why to engage in collaboration, that is, the motivations; what inhibits collaboration, that is, the barriers; and how to collaborate, that is, the channels of knowledge transfer. However, limited research on such antecedents is available in open innovation, with only a few recent studies addressing these antecedents in an integrative manner rather than independently [17,18]. Despite these advances that have been recently achieved related to university-industry collaboration in open innovation, a comprehensive investigation of the antecedents’ association with the perceived importance of their collaboration in such a context is still missing. Within this framework, we aim to explore the degree to which these three major antecedents influence the importance perception of collaboration between the two actor organizations in open innovation. For this purpose, we define the following research questions to guide our study:

RQ1. Given motivations, barriers, and channels of collaboration as major antecedents, which of them significantly impact the perception of universities as an external knowledge source for open innovation activities?

RQ2. How can this impact be predicted?

To answer these questions, we developed a conceptual framework to explore this impact. An exploratory approach was employed using the proposed research framework to collect and then investigate the empirical data based on a two-stage ordinal regression and artificial neural network (ANN) analysis. While ordinal regression can be easily interpreted and used in hypothesis testing, it has less capability to capture the diverse and non-linear relationships among the variables of highly complex phenomena, such as open innovation. An ANN approach is recommended in the literature to overcome this challenge [19] since it does not require a priori specification of the input-output relationships. With no prior assumptions, ANN is also an appropriate choice for explorative studies that look for insights beyond what traditional methods (e.g., ordinal regression) can provide [20]. Although ANN has been used to handle different problems in many fields [19,20,21,22,23,24,25], its employment in industry-university collaboration is still scarce [26]. Nevertheless, the existing studies argue that an ANN approach may be appropriate to model the complexity of innovation as it was found to have good results in modeling its dynamic and nonlinear nature [27,28]. Moreover, recent studies [20,29] have emphasized the advantages of integrating statistical (i.e., ordinal regression) and ANN models when compared to a single-step traditional statistical or predictive ANN analysis. Accordingly, a two-stage ordinal regression and neural network approach may lead to better modeling and prediction performance. Therefore, incorporating the results of ordinal regression into the ANN approach for a jointly explanatory and predictive analysis can offer a perspective that enables a more in-depth understanding of the key antecedents and their impact on the perception of universities as an external knowledge source to the industry in open innovation. As a result, answers to the two research questions of this study are provided.

The remainder of our article is structured as follows: First, we provide a research background and conceptual framework section, which also illustrates the hypotheses of this study. Next, we describe the research approach that includes the setting of our study, the constructs of the developed framework, and their measurement. After that, we present the mathematical modeling of the two-stage ordinal regression and neural network approach. Then, we analyze the data and report the results of our study. Finally, we discuss the findings of our research, conclude the final remarks, present the limitations of the study, and make suggestions for future research.

2. Research Background and Conceptual Framework

The ability to capture and exploit external knowledge has been long seen as of great importance in the development of innovative capabilities [30]. A number of current models, such as the Triple Helix [31] and its extension to the Fourth Helix and beyond [32], have already stressed the effect of firms’ collaboration with heterogeneous and diverse actors on innovation [33]. According to these models, other organizations (i.e., public entities, industry associations, civil society, and the natural environment of society) are often important connectors between universities and industries. This may be beneficial to their collaboration since it can stimulate knowledge production, innovation generation, and synergy between the involved stakeholders [31]. In this way, firms are expected to benefit more from external collaboration in their innovation activities. Among such external knowledge sources, universities are particularly large repositories of generic and more important new scientific knowledge (i.e., ideas, knowledge, or technologies developed by universities [34]), which can be used by firms to develop novel products and services [35]. Considering the heterogeneity of this knowledge as well as that of the partners, knowledge and technology transfer between industry and universities is expected to accelerate innovation [36].

There have been numerous studies looking into how firms access the scientific knowledge of universities and collaborate with them in the innovation process [37,38,39]. These studies underline that firms are constantly looking to integrate such scientific knowledge provided by university actors into their innovation funnel and to develop new ways to innovate. Moreover, models such as the quadruple helix have emphasized the flow of knowledge into all spheres of society [40], pointing to more open models of innovation processes among the collaborating partners [41]. At the same time, science is becoming more and more open [42,43,44], which has led to a large amount of scientific knowledge in almost any scientific area [42]. As academics generate and disseminate both explicit and tacit knowledge through their activities, easy access to the abundant scientific knowledge of universities is relatively high [45]. However, the nature of their knowledge tends to be more fundamental than specifically oriented (often referred to as “basic knowledge” and “applied knowledge”, respectively), as academics are more likely to generalize their findings than to create more particular and specialized ones [45].

In this context, open innovation between industry and universities becomes especially important in accessing those knowledge resources that firms cannot produce, for different reasons, internally. In such an approach, firms are considered the central agents [46], which have to combine the inflows and outflows of knowledge using pecuniary and non-pecuniary mechanisms to deal with the greater complexity of innovation and improve their innovation efforts [47]. Although industry has employed external knowledge sources to improve its innovation potential for decades, open innovation has become so attractive because it gathers into a single concept a collection of already existing activities and offers many opportunities for extension and development [48], including in its collaboration with universities. As a result, firms are increasingly adopting open innovation with universities to enhance their innovation processes, which is expected to contribute to better performance and a competitive advantage [49,50,51]. Indeed, the knowledge inflow from collaboration with universities usually has a positive influence on the involved firms [52], which can be highlighted in terms of both pecuniary and non-pecuniary benefits [53]. However, opening up the innovation to universities gives rise to some challenges that may be addressed in the same logic of pecuniary versus non-pecuniary proposed by Dahlander and Gann [53].

Within the highly complex and sophisticated ecosystem of industry-university collaboration [54], a number of determinants that describe the process are presented and analyzed in the literature. Nevertheless, three specific dimensions typically characterize the patterns of their interaction. They are motivations, barriers, and channels of knowledge transfer [16], which can be seen as major antecedents of the collaboration between firms and universities in open innovation. Their impact in such a context is thus reviewed next.

Regarding the first antecedent, motivation is less explored in the extant literature, although firms may engage in open innovation with universities for many different reasons [55,56]. The results of a study conducted by Lam et al. [50] in the Hong Kong environmental industry showed the integration of complex technologies and the development of new innovative products for increasingly demanding customers as among the most important drivers of industry-university collaboration in open innovation. An analysis of several motivation factors between automotive companies within a consortium and two universities in open innovation carried out by Lopes et al. [57] listed the development and sharing of knowledge to reduce the time to innovation, as well as the developing and sharing of technology, at the top of the list of the motives’ importance. Based on an extensive review of the extant literature, Baban and Baban [58] synthesized the motives of open innovation between industry and university and distinguished ten main components. The existing studies reveal the positive influence of motives in the automotive industry to achieve better competitiveness and success through open innovation [50,55].

Concerning the second antecedent, different challenges that industry is facing in its collaboration with universities in open innovation have been scrutinized in the extant literature, and various barriers have been identified by these studies. Lam et al. [50] found the unavailability of external partners capable of providing the necessary scientific knowledge and the possibility of disclosure of their intellectual property to these partners as the main barriers to collaboration between the two actor organizations. Differences in the orientation of research between industry and university (short-term applied research vs. long-term fundamental research) and the lack of adequate internal resources (particularly for small and medium enterprises) were identified as among the main barriers by Saguy and Sirotinskaya [59]. In the view of Galati et al., finding the best scientific partner that can solve technical or technological problems was seen as the main challenge [60]. Using a Delphi approach, Quiñones et al. [61] extracted the relevant barriers to university technology transfer and, through a case study, pointed out conflicting objectives between research and commercialization of the transfer results as the barrier most influenced by the other barriers. At the same time, they found the high costs of managing joint research projects and administrative bureaucracy as the barriers that impacted the other ones the most. Baban and Baban [58] also identified such barriers considering the extant literature and differentiated them into a list of fourteen main elements. There is also a commonly accepted negative meaning of the barriers [62], as they impede the adoption of open innovation between industry and university.

With regard to the third antecedent, the existing studies suggested that during industry-university collaboration in open innovation, scientific knowledge flows through diverse channels that can be characterized in different ways. Considering their dominant mode of governance, Alexander et al. [41] characterized each channel in terms of relational (informal) or transactional (formal) engagement. They hypothesized that the first style of engagement is more achieved through channels that stimulate open innovation, while the last one is more related to the closed cycle of innovation. Their opinion is in agreement with the prior work of Villasalero [14], which described the scientific knowledge transfer channels based on a continuum between revealing and selling strategies. After reviewing the extant literature, Baban and Baban [58] synthesized the existing connection between industry and university into twelve channels of scientific knowledge transfer, considering both types of strategies. According to Costa et al. [63], promoting multiple channels of collaboration rather than a single one may be more effective. However, maintaining a large number of channels may also be detrimental since it requires adequate resources, the right timing, and special attention [64]. Even so, the connections with the university support firms to achieve and sustain innovation and play an important role in shaping their performance [63].

In light of these considerations, we may conclude that the antecedents of opening up innovation between firms and universities have been studied rather independently than all together. Moreover, an investigation of the extent to which these antecedents influence the perceived importance of universities as an external knowledge source for industry is still lacking. In order to address this research gap and considering the above-presented argumentation regarding the influence of the three antecedents, we propose the research framework illustrated in Figure 1 and formulate the following expectations about the first research question of our study:

H1. 

The motives of collaboration have a positive and significant impact on the perception of the importance of universities as an external knowledge source to industry in open innovation.

H2. 

The barriers to collaboration have a negative and significant impact on the perception of universities as an important external knowledge source for industry in open innovation.

H3. 

The knowledge transfer channels of collaboration are more likely to have a positive and significant impact on the perception of universities as an external knowledge source to industry in open innovation.

Figure 1. The research framework.

Figure 1. The research framework.

3. Study Setting and Data Collection

There has been very little theoretical or empirical evidence related to the framework presented in Figure 1. One notable exception is the work [17], in which an integrative approach was employed to explore the impact of the three analyzed antecedents on the outcome of collaboration, its findings revealing their influence on both the benefits and drawbacks of collaboration between firms and universities in open innovation. Therefore, we employed an exploratory approach to address the research hypotheses of this study. For this purpose, our data source was represented by companies from industrial areas since firms from such agglomerations are not only contributing to increased economic growth and regional development [65], but are increasingly becoming more open to global competition and have to face both its opportunities and threats [66]. Moreover, collaborating with qualified local actors, such as universities, has been found to be among the most important drivers of their innovation activities [66]. Taking into account the great extent of the Italian experience in the field of business agglomeration and the continuous efforts of Romania to develop its industrial sectors based on agglomeration concepts, two industrial Italian and Romanian areas were employed for data collection. The first one is the Valenza Industrial District, which is seen as one of the most important components of the so-called ‘Made in Italy’ sectors, while the other is represented by one of the most successful stories in increasing competitiveness through the implementation of industrial parks in the Romanian economy, i.e., the Oradea Industrial Parks [67]. Although most of these firms compete in a world-class manufacturing environment, those from the Valenza Industrial District and Oradea Industrial Parks belong to low-tech and medium-to-high-tech industries, respectively. Moreover, most of the firms from the Valenza Industrial District are small enterprises, while those from industrial parks are medium-sized and large enterprises.

We exploit data from a self-administered survey conducted among companies located within the two industrial areas. Among other dimensions, the survey addressed each of the four components of the research framework depicted in Figure 1, using a scale from 1 (‘not important’) to 5 (‘very important’) for all of their items [18].

One of the main concerns regarding open innovation is related to the paradox of openness [68], which manifests because the development of innovation is often based on openness, while capturing its returns may demand protection and security. Therefore, it can be difficult to find adequate respondents as firms tend to protect innovation against imitation. Moreover, information about such people is not publicly available, so we followed a purposive sampling method to select these respondents. Since our study was an explorative one, the recommended sample size should be 100 or larger [69]. The considered sampling criteria included: (1) participants who were in charge of the innovation activities/owners of those firms that cooperate with universities and who agreed to respond to the survey questions; (2) respondents should come from different industries and firm sizes to cover as much diversity as possible. Prospective candidates were identified through a prescreening process, and in the end, we received 100 questionnaires, of which two were removed based on questionable responses. Therefore, a total of 98 questionnaires remained for the analysis.

Table 1. Sample information.

Survey Design
Data collection method	Self-administered survey
Sampling design	Purposive sampling
Total responses/Accepted responses	100/98
Sample Attributes
(a) Industry type
Type of industry	Frequency (%)
1 = High-tech industry (electronics)	12.24
2 = Medium high-tech industry (automotive)	38.78
3 = Low-tech industry (jewelry)	48.98
(b) Firm size
Size class	Frequency (%)
1 = Small and medium-sized enterprises (10 to 249 employees)	59.18
2 = Large enterprises (250+ employees)	40.82

summarizes the sample details. For internal consistency reliability, we found that Cronbach’s alphas for the OiM.Motives and OiC.Channels were 0.913 and 0.931, respectively, which represent high internal consistency [69]. Meanwhile, for the OiB.Barriers, the Cronbach’s alpha was 0.805, which indicates a good level of the internal consistency reliability of this antecedent.

4. Research Approach

4.1. Research Framework Constructs and Their Measurement

In line with the above-mentioned considerations, Figure 2 details the constituents of the proposed research framework and the measures behind each construct. The dependent variable y is represented by the perceived importance of universities as an external knowledge source for industry in open innovation. The OiM, OiB, and OiC antecedents were used to define the explanatory variables of our study, with all their items adapted from the extensive survey of Baban and Baban [58]. For this purpose, we introduce three new variables, following the approach proposed in Laursen and Salter [64] and Leiponen and Helfat [70].

The first new explanatory variable is named x1 and is constructed for any response $\mathrm{i}\hspace{0.33em}(\mathrm{i}=\overline{1, 98})$ of the survey as a combination of the 10 items of the motives OiM_j ($\mathrm{j}=\overline{1, 10}$) presented in Figure 2. Each of these 10 items is coded as 0 if the motive is not important, of little importance, or somewhat important (i.e., 1/2/3), and 1 if it is important or very important (i.e., 4/5). Then, the 10 items are added up so that the expression of x1 for the ith response is given as:

$$\hspace{0.17em}x{1}_{\mathrm{i}}={\displaystyle \sum _{\mathrm{j}=1}^{10}\mathrm{Oi}{\mathrm{M}}_{\mathrm{ji}},\hspace{0.33em}\mathrm{i}=\overline{1, 98}}$$

(1)

$$\mathrm{with}\hspace{0.33em}\mathrm{Oi}{\mathrm{M}}_{\mathrm{ji}}=\left\{\begin{array}{c}1,\hspace{0.33em}\mathrm{if}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{jth}\hspace{0.33em}\mathrm{motive}\hspace{0.33em}\mathrm{of}\hspace{0.33em}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{ith}\hspace{0.33em}\mathrm{response}\hspace{0.33em}\mathrm{received}\hspace{0.33em}\mathrm{a}\hspace{0.33em}\mathrm{value}\hspace{0.33em}\mathrm{of}\hspace{0.33em}4\hspace{0.33em}\mathrm{or}\hspace{0.33em}5\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\\ 0,\hspace{0.33em}\mathrm{if}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{jth}\hspace{0.33em}\mathrm{motive}\hspace{0.33em}\mathrm{of}\hspace{0.33em}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{ith}\hspace{0.33em}\mathrm{response}\hspace{0.33em}\mathrm{received}\hspace{0.33em}\mathrm{a}\hspace{0.33em}\mathrm{value}\hspace{0.33em}\mathrm{of}\hspace{0.33em}1,\hspace{0.33em}2\hspace{0.33em}\mathrm{or}\hspace{0.33em}3\end{array}\right.$$

(2)

Therefore, each x1_i$(\mathrm{i}=\overline{1, 98}$) gets a minimum and maximum value of 0 and 10, respectively. In this way, we assumed that the more important the motives for collaboration are, the stronger the influence they will have on the perception of universities as an external knowledge source for open innovation. Although the x1 variable is a relatively simple construct, its Cronbach’s alpha coefficient is equal to 0.882, which represents a relatively high internal consistency [69].

The second new explanatory variable is termed x2 and is constructed in a similar way to the x1 variable, using the 14 items of the barriers OiB_j ($\hspace{0.17em}\mathrm{j}=\overline{1, 14}$) shown in Figure 2. Thus, the expression of x2 for the ith response is defined as:

$$\hspace{0.17em}x{2}_{\mathrm{i}}={\displaystyle \sum _{\mathrm{j}=1}^{14}\mathrm{O}\mathrm{i}{\mathrm{B}}_{\mathrm{j}\mathrm{i}},\hspace{0.33em}\mathrm{i}=\overline{1, 98}}$$

(3)

$$\hspace{0.17em}\mathrm{with}\hspace{0.33em}\mathrm{O}\mathrm{i}{\mathrm{B}}_{\mathrm{j}\mathrm{i}}=\left\{\begin{array}{c}1,\hspace{0.33em}\mathrm{if}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{jth}\hspace{0.33em}\mathrm{barrier}\hspace{0.33em}\mathrm{of}\hspace{0.33em}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{ith}\hspace{0.33em}\mathrm{response}\hspace{0.33em}\mathrm{received}\hspace{0.33em}\mathrm{a}\hspace{0.33em}\mathrm{value}\hspace{0.33em}\mathrm{of}\hspace{0.33em}4\hspace{0.33em}\mathrm{or}\hspace{0.33em}5\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\\ 0,\hspace{0.33em}\mathrm{if}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{jth}\hspace{0.33em}\mathrm{barrier}\hspace{0.33em}\mathrm{of}\hspace{0.33em}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{ith}\hspace{0.33em}\mathrm{response}\hspace{0.33em}\mathrm{received}\hspace{0.33em}\mathrm{a}\hspace{0.33em}\mathrm{value}\hspace{0.33em}\mathrm{of}\hspace{0.33em}1,\hspace{0.33em}2\hspace{0.33em}\mathrm{or}\hspace{0.33em}3\end{array}\right.$$

(4)

As a result, the minimum and maximum values of each x2_i$(\mathrm{i}=\overline{1, 98}$) will be 0 and 14, respectively. Thus, we assumed that the more important the barriers to collaboration are, the stronger their impact on the perception of universities as an external knowledge source for open innovation. Its internal consistency is also at a good level, as its Cronbach’s alpha coefficient is equal to 0.844 [69].

The third new explanatory variable is called x3 and is constructed similarly to the x1 and x2 variables, using the 12 items of the channels OiC_j ($\mathrm{j}=\overline{1, 12}$) illustrated in Figure 2. Subsequently, the expression of x3 for the ith response is given as:

$$x{3}_{\mathrm{i}}={\displaystyle \sum _{\mathrm{j}=1}^{12}\mathrm{O}\mathrm{i}{\mathrm{C}}_{\mathrm{j}\mathrm{i}},\hspace{0.33em}\mathrm{i}=\overline{1, 98}}$$

(5)

$$\mathrm{with}\hspace{0.33em}\mathrm{O}\mathrm{i}{\mathrm{C}}_{\mathrm{j}\mathrm{i}}=\left\{\begin{array}{c}1,\hspace{0.33em}\mathrm{if}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{jth}\hspace{0.33em}\mathrm{channel}\hspace{0.33em}\mathrm{of}\hspace{0.33em}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{ith}\hspace{0.33em}\mathrm{response}\hspace{0.33em}\mathrm{received}\hspace{0.33em}\mathrm{a}\hspace{0.33em}\mathrm{value}\hspace{0.33em}\mathrm{of}\hspace{0.33em}4\hspace{0.33em}\mathrm{or}\hspace{0.33em}5\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\\ 0,\hspace{0.33em}\mathrm{if}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{jth}\hspace{0.33em}\mathrm{channel}\hspace{0.33em}\mathrm{of}\hspace{0.33em}\hspace{0.33em}\mathrm{the}\hspace{0.33em}\mathrm{ith}\hspace{0.33em}\mathrm{response}\hspace{0.33em}\mathrm{received}\hspace{0.33em}\mathrm{a}\hspace{0.33em}\mathrm{value}\hspace{0.33em}\mathrm{of}\hspace{0.33em}1,\hspace{0.33em}2\hspace{0.33em}\mathrm{or}\hspace{0.33em}3\end{array}\right.$$

(6)

In this way, each x3_i $(\mathrm{i}=\overline{1, 98}$) will have a minimum value of 0 and a maximum of 12. Again, we assumed that the more important the channels for collaboration are, the stronger their influence on the perception of universities as an external knowledge source for open innovation. The internal consistency of the x3 variable is high since its Cronbach’s alpha coefficient is equal to 0.909 [69].

We also include two control variables commonly used in innovation studies, which are the size of the firm and industry type [71,72]. The first one is measured by the number of employees, while for the other one, we consider the Eurostat classification of different industries [73] (see the sample attributes in Table 1). We label the firm size and industry type as x4 and x5, respectively.

4.2. Setting up the Mathematical Modeling

Our dependent variable is a discrete and multinomial-choice response with a logical order. Therefore, an ordinal regression based on the cumulative link model [74] was proposed to infer the dependence of the response on the explanatory variables. While ordinal regression can be interpreted and statistically tested, it is not capable of performing well with any type of non-linear model and any data distribution [75]. At the same time, ANNs have a higher capability to learn any relationships between the dependent and explanatory variables through an iterative process based on the data pattern, and they are highly robust and tolerant against noise [76,77]. In this way, ANNs can use the information hidden in data, although they are not capable of extracting it [78]. However, ANNs are not suitable for explanation and hypothesis testing because of their “black box” algorithm, which makes it difficult to determine how the decision is taken by the ANNs [77].

In order to take advantage of the advantages provided by both ordinal regression and ANN and address their shortcomings, a two-stage approach was adopted in our research. In the first stage, we used ordinal regression to examine the proposed research model and hypotheses of the study. Next, the significant explanatory variables obtained from the hypotheses testing of the initial ordinal regression analysis were employed as the inputs for the second stage ANN analysis. In this way, the complex relationships among these explanatory and dependent variables can be further investigated. Figure 3 illustrates the connection between the two stages.

4.2.1. The Ordinal Regression Modeling

Suppose $\mathrm{y}\in \{{\mathrm{C}}_1,\hspace{0.17em}{\mathrm{C}}_2,\dots ,{\mathrm{C}}_{\mathrm{k}}\}$ is the ordinal dependent variable with ${\mathrm{c}}_1\hspace{0.17em}\prec {\mathrm{c}}_2\prec \dots \prec {\mathrm{c}}_{\mathrm{k}}$ (where $\hspace{0.17em}\prec$ is an order relation between categories) and x = (x₁, x₂, …, x_q) a set of q explanatory variables, $\mathrm{x}\in {\mathrm{R}}^{\mathrm{q}}$ ($\hspace{0.33em}\mathrm{k}=\overline{1, 5}$ and q = 5 in our case). According to this model, the cumulative probabilities $\mathrm{P}(\mathrm{y}\le {\mathrm{C}}_{\mathrm{j}}\left|\mathrm{x}\right.)$ can be estimated as follows [79]:

$$\mathrm{P}(\mathrm{y}\le {\mathrm{C}}_{\mathrm{j}}\left|\mathrm{x}\right.)=\mathrm{P}(\mathrm{y}={\mathrm{C}}_1\left|\mathrm{x}\right.)+\mathrm{P}(\mathrm{y}={\mathrm{C}}_2\left|\mathrm{x}\right.)\dots +\mathrm{P}(\mathrm{y}={\mathrm{C}}_{\mathrm{j}}\left|\mathrm{x}\right.)$$

(7)

$$\mathrm{P}(\mathrm{y}={\mathrm{C}}_{\mathrm{j}}\left|\mathrm{x}\right.)=\mathrm{P}(\mathrm{y}\le {\mathrm{C}}_{\mathrm{j}}\left|\mathrm{x}\right.)-\mathrm{P}(\mathrm{y}\le {\mathrm{C}}_{\mathrm{j}-1}\left|\mathrm{x}\right.)$$

(8)

for $\mathrm{j}=\overline{2,\mathrm{k}}$, and considering by definition that $\mathrm{P}(\mathrm{y}={\mathrm{C}}_1\left|\mathrm{x}\right.)=\mathrm{P}(\mathrm{y}\le {\mathrm{C}}_1\left|\mathrm{x}\right.)\hspace{0.33em}\mathrm{and}\hspace{0.33em}$ $\mathrm{P}(\mathrm{y}\le {\mathrm{C}}_{\mathrm{k}}\left|\mathrm{x}\right.)=1$.

Let ${\mathrm{P}}_{\mathrm{j}}=\mathrm{P}(\mathrm{y}\le {\mathrm{C}}_{\mathrm{j}}\left|\mathrm{x}\right.)$ be the cumulative probability of category j. The general model for ordinal regression can be written as follows ([74], p. 308):

$$\mathcal{L}({\mathrm{P}}_{\mathrm{j}})={\mathsf{\theta }}_{\mathrm{j}}-{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{q}}{\mathsf{\beta }}_{\mathrm{i}}\cdot {\mathrm{x}}_{\mathrm{i}}}, \mathrm{j}=\overline{1,\mathrm{k}-1}$$

(9)

where $\mathcal{L}$ (p_j) is a link function that links the dependent variable y to the explanatory variables x, θ_j is the intercept of the regression equation or threshold for each C_j with ${\mathsf{\theta }}_1\hspace{0.17em}\le {\mathsf{\theta }}_2\le \dots \le {\mathsf{\theta }}_{\mathrm{k}-1}$, and ${\mathsf{\beta }}_{\mathrm{i}},\hspace{0.17em}\mathrm{i}=\overline{1,\mathrm{q}}$ are the regression coefficients. To implement the regression model (9), the unknown parameters ${\mathsf{\theta }}_{\mathrm{j}},\mathrm{j}=\overline{1,\mathrm{k}-1}$ and ${\mathsf{\beta }}_{\mathrm{i}},\hspace{0.17em}\mathrm{i}=\overline{1,\mathrm{q}}$ must be estimated from the sample data. The maximum likelihood estimation approach is usually employed for this purpose [74].

Different link functions $\mathcal{L}$(p_j) can be used in an ordinal regression model, depending on the distribution of the response variable values [74]. When the cumulative probabilities of the less important categories are lower and then rapidly approach 1 for the more important ones, the complementary log-log link function is recommended in the ordinal regression analysis [80]. This is our case (see Figure 4), and our model can be expressed as follows ([74], p. 308):

$$\ln (-\ln (1-{\mathrm{p}}_{\mathrm{j}}))={\mathsf{\theta }}_{\mathrm{j}}-{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{q}}{\mathsf{\beta }}_{\mathrm{i}}\cdot {\mathrm{x}}_{\mathrm{i}}}$$

(10)

and

$${\mathrm{p}}_{\mathrm{j}}=\mathrm{P}(\mathrm{y}\le {\mathrm{C}}_{\mathrm{j}}\left|\mathrm{x}\right.)=1-{\mathrm{e}}^{-{\mathrm{e}}^{\left({\mathsf{\theta }}_{\mathrm{j}}-{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{q}}{\mathsf{\beta }}_{\mathrm{i}}\cdot {\mathrm{x}}_{\mathrm{i}}}\right)}}$$

(11)

The probability $\mathrm{P}(\mathrm{y}={\mathrm{C}}_{\mathrm{j}}\left|\mathrm{x}\right.)$ of a category j can then be obtained from relations (8) and (11).

Figure 4. The cumulative probabilities percentages of the ordinal dependent variable.

Figure 4. The cumulative probabilities percentages of the ordinal dependent variable.

The regression coefficients in the model (9) are generally more difficult to interpret, as an estimated coefficient β_i represents the change in the link function for each unit change in the corresponding explanatory variable, holding the other regressors constant [81]. More generally, a positive/negative regression coefficient indicates that as the value of the explanatory variable increases/decreases, the probability of a higher category of the dependent variable increases/decreases, while the other regressors in the model are held constant [82]. Thus, the analysis should be focused on the sign and statistical significance of the regression coefficients, since the signs give the direction of the effect of the explanatory variables on the dependent variable, and their statistical significance can be used to test the hypotheses of the study.

4.2.2. The ANN Modeling

The architecture of an ANN includes one input layer, one or more hidden layers, and one output layer. The ANNs with a single hidden layer are recognized as shallow networks, while those with two or more hidden layers are known as deep neural networks [83,84]. Each layer of an ANN consists of a number of processing units that are also called neurons or nodes. The input layer comprises the initial units that each handle one explanatory variable, while the output layer contains the units that each represent one category of the dependent variable. The units in the hidden layers are the main components in an ANN, which uses the ${\mathrm{x}}_{\mathrm{i}},\hspace{0.17em}\mathrm{i}=\overline{1,\mathrm{p}}$ features of the input layer/hidden units in lower layers and produces a numerical output to the hidden units in the higher layers/output layer, ${\mathrm{y}}_{\mathrm{j}}=\hspace{0.17em}\mathsf{\varphi }\left({\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{p}}{\mathrm{w}}_{\mathrm{j}\mathrm{i}}{\mathrm{x}}_{\mathrm{i}}+{\mathrm{b}}_{\mathrm{j}}}\right)$ [83]. Considering a hidden unit j, w_ji are the components of the weight matrix, b_j represents the bias term, and φ(^.) stands for the activation function. The extant literature [19,85] shows different types of ANNs that are closely related to the learning algorithm, the process through which the unknown information from the data is captured by the adjustment of the weight coefficients w_ji.

The capacity of an ANN to model complex and nonlinear relationships between the input and output units depends on the size and number of nodes in the hidden layer(s) [77,86]. As these hyperparameters of network structure are increased, the capacity of ANN to recognize such relationships and thus produce higher classification accuracy is enhanced. However, the probability of overtraining the model is also increased for networks that are too large [77,86]. Although different approaches are described in the literature [87,88,89], there may be no easy way to define the size and number of nodes in the hidden layer(s). Nevertheless, Huang and Babri [90] rigorously proved that N distinct samples can be precisely learned by a single-hidden-layer feedforward network (SLFN) with at most N hidden nodes. The sufficient condition for the activation function is that it be any bounded non-linear function that has a limit at infinity [90]. Thus, the upper bound of the number of hidden nodes for an SLFN is given by [90]:

N_SLFN ≤ N

(12)

Moreover, Huang [91] rigorously proved that the required hidden nodes for a two-hidden-layer feedforward network (TLFN) are as follows ([91], p. 275):

$${\mathrm{N}}_{\mathrm{TLFN}}=2\cdot \sqrt{({\mathrm{n}}_{\mathrm{o}}+2)\cdot \mathrm{N}}$$

(13)

where N represents the number of distinct samples, while n_o stands for the number of outputs. Huang [91] further proved that a TLFN with (N₁)_TLFN and (N₂)_TLFN hidden nodes in the first and second layers, which are given by ([91], p. 278):

$$\begin{array}{l}{({\mathrm{N}}_1)}_{\mathrm{TLFN}}=\sqrt{({\mathrm{n}}_{\mathrm{o}}+2)\cdot \mathrm{N}}+2\cdot \sqrt{\mathrm{N}/({\mathrm{n}}_{\mathrm{o}}+2)}\\ {({\mathrm{N}}_2)}_{\mathrm{TLFN}}={\mathrm{n}}_{\mathrm{o}}\cdot \sqrt{\mathrm{N}/({\mathrm{n}}_{\mathrm{o}}+2)}\end{array}$$

(14)

can learn the N distinct samples with negligibly small errors. It should be noted that (N₁)_TLFN and (N₂)_TLFN in expression (14) are also related to the upper bounds of the required hidden nodes for a TLFN.

Among the existing ANN architectures, the multilayer perceptron (MLP), a fully connected layer feedforward network that is considered a universal approximator for nonlinear functions [85], is one of the most used in classification and regression problems. Although a MLP with a single hidden layer (SMLP) is considered enough for most problems, a MLP with two hidden layers (TMLP) is recognized to solve some problems more efficiently [86], especially when large numbers of input samples are employed [91]. Usually, there is no need to employ a multi-layer feed-forward neural network with more than two hidden layers [78]. Therefore, the architectures of such MLP networks adapted to learn ordinal categories based on the approach of Cheng et al. [92] were used in our study. The activation function of the hidden nodes of this adapted MLP can be any of the commonly used activation functions. Nevertheless, different recommendations on how to choose them are available in the literature, including the goodness-of-fit tests presented in [93,94]. Since the complementary log-log link function in relation (10) is the inverse of the cumulative distribution function (cdf) of the extreme value (or log-Weibull) distribution that can be written as $\mathrm{F}({\mathrm{z}}_{\mathrm{i}})=1-{\mathrm{e}}^{-{\mathrm{e}}^{{\mathrm{z}}_{\mathrm{i}}}}$, this expression is proposed in our study as the activation function for each output node of the MLP network instead of the softmax that is generally employed for a standard classification of MLP.

As a result, the topology of our MLP network is as follows ([95], pp. 291–292):

Input layer: J₀ = r nodes, a_0:1, a_0:2,…, a_0:r with a_0:j = x_j ($\mathrm{j}=\overline{1,\mathrm{r}}$).

The ith hidden layer ($\mathrm{i}=\overline{1, 2}$): J_i nodes, a_i:1,…, a_i:Js,…, a_i:Ji, with a_i:s = φ_i(c_i:s) and ${\mathrm{c}}_{\mathrm{i}:\mathrm{s}}={\displaystyle \sum _{\mathrm{j}=0}^{{\mathrm{j}}_{\mathrm{i}-1}}{\mathrm{w}}_{\mathrm{i}:\mathrm{j},\mathrm{s}}\cdot {\mathrm{a}}_{\mathrm{i}-1:\mathrm{j}}}$ where: the upper bound J₁ for SMLP is according to relation (12); the J₁ and J₂ for the TMLP are given by relation (14); φ_i is the activation function for the layer i; w_i:j,s is the weight leading from node j of layer i − 1 to node s of layer i; and a_i-1:0 = 1. Different types of activation functions can be used for the hidden layers. One of the most commonly employed is the logistic activation function defined by ${\mathsf{\varphi }}_{\mathrm{i}}(\mathrm{z})=\frac{1}{1+{\mathrm{e}}^{-\mathrm{z}}}$ [85], which is also used in our study.

Output layer: J_I = k nodes, a_I:1,…, a_I:s,…, a_I:k with a_I:s = φ_I(c_I:s) and ${\mathrm{c}}_{\mathrm{I}:\mathrm{s}}={\displaystyle \sum _{\mathrm{j}=0}^{{\mathrm{j}}_1}{\mathrm{w}}_{\mathrm{i}:\mathrm{j},\mathrm{s}}\cdot {\mathrm{a}}_{\mathrm{i}-1:\mathrm{j}}}$ where a_i-1:0 = 1. The activation function for this layer is given by ${\mathsf{\varphi }}_{\mathrm{i}:\mathrm{s}}(\mathrm{z})=1-{\mathrm{e}}^{-{\mathrm{e}}^{{\mathrm{z}}_{\mathrm{s}}}}$.

4.2.3. Performance Measures

Several measures are available to assess the performance of multiple classification models. One of the most useful of these is accuracy, which returns the proportion of classes that were estimated correctly [96,97]. Mathematically, accuracy can be expressed as follows ([96], p. 54):

$$\mathrm{a}\mathrm{c}\mathrm{c}=\frac{{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}\mathrm{I}({({\widehat{\mathrm{C}}}_{\mathrm{j}})}_{{}_{\mathrm{i}}}={({\mathrm{C}}_{\mathrm{j}})}_{{}_{\mathrm{i}}})}}{\mathrm{N}},(\forall )\mathrm{j}\hspace{0.33em}=\overline{1,\mathrm{k}}$$

(15)

where $\mathrm{I}({({\widehat{\mathrm{C}}}_{\mathrm{j}})}_{{}_{\mathrm{i}}}={({\mathrm{C}}_{\mathrm{j}})}_{{}_{\mathrm{i}}})=\left\{\begin{array}{c}1,\hspace{0.33em}\mathrm{if}\hspace{0.33em}{({\widehat{\mathrm{C}}}_{\mathrm{j}})}_{{}_{\mathrm{i}}}={({\mathrm{C}}_{\mathrm{j}})}_{{}_{\mathrm{i}}}\\ 0,\hspace{0.33em}\mathrm{othervise}\end{array}\right.$, ${\widehat{\mathrm{C}}}_{\mathrm{j}}$ is the predicted dependent variable, C_j is the true value of the dependent variable, and N is the sample size.

Although accuracy indicates how much the model is correctly estimating on the entire data set, the differences in performance on different classes are not considered in relation (15). Therefore, other metrics may also be meaningful in the case of multi-class problems. Such a commonly employed measure is the Area Under the Receiver Operating Characteristic Curve (AUC-ROC), which quantifies the ability of the model to distinguish between each class [97]. One of the most used approaches to computing AUC for different classes was developed by Hand and Till [98]. Their AUC combines the pairwise discriminability values of classes into a global measure that can be written as follows ([98], p. 177):

$${\mathrm{AUC}}_{\mathrm{total}}=\frac{2}{\mathrm{k}(\mathrm{k}-1)}{\displaystyle \sum _{\mathrm{i}<\mathrm{j}}\mathrm{AUC}({\mathrm{c}}_{\mathrm{i}}{,\mathrm{c}}_{\mathrm{j}})}$$

(16)

where $\mathrm{AUC}({\mathrm{c}}_{\mathrm{i}},{\mathrm{c}}_{\mathrm{j}})=\frac{\mathrm{AUC}({\mathrm{c}}_{\mathrm{i}}{|\mathrm{c}}_{\mathrm{j}})+\mathrm{AUC}({\mathrm{c}}_{\mathrm{j}}|{\mathrm{c}}_{\mathrm{i}})}{2}$. Each term in the expression of AUC(c_i,c_j) is computed based on the formula developed by Hand and Till [98] for binary classes. For example, the expression of AUC(c_i|c_j) that represents the probability of a randomly chosen class c_j having a lower estimated probability of belonging to class c_i than a randomly chosen class c_i is given by ([98], p. 174):

$$\mathrm{AUC}({\mathrm{c}}_{\mathrm{i}}{|\mathrm{c}}_{\mathrm{j}})=\frac{{\mathrm{S}}_{{\mathrm{c}}_{\mathrm{i}}}-{\mathrm{n}}_{{\mathrm{c}}_{\mathrm{i}}}\cdot ({\mathrm{n}}_{{\mathrm{c}}_{\mathrm{i}}}+1)/2}{{\mathrm{n}}_{{\mathrm{c}}_{\mathrm{i}}}\cdot {\mathrm{n}}_{{\mathrm{c}}_{\mathrm{j}}}}$$

(17)

where ${\mathrm{S}}_{{\mathrm{c}}_{\mathrm{i}}}$ is the sum of the ranks of the class c_i data set points, while ${\mathrm{n}}_{{\mathrm{c}}_{\mathrm{i}}}$ and ${\mathrm{n}}_{{\mathrm{c}}_{\mathrm{j}}}$ are the number of data set points that belong to class c_i and c_j, respectively.

Since an ANN multiple ordinal classification is an iterative approach in which the network architecture is continually improved and its hyperparameters are tuned to obtain the expected output, computing time and resources are influenced by the sophistication of the ANN model [99]. In order to determine if an ANN model configuration is worth applying, the resulting accuracy versus the time consumed by the ANN model to reach it was introduced as a measure of time complexity [99].

5. Data Analysis and Results

5.1. The Ordinal Regression Analysis

Ordinal regression is sensitive to the explanatory variables that are highly correlated with each other [100]. Thus, one of the model assumptions includes the absence of multicollinearity among these variables [100]. The variance inflation factor (VIF) was employed to check that there is no multicollinearity between the independent variables, as it is a commonly employed measure to verify its absence [101]. Besides ordinal regression, detecting collinearity between independent variables is also needed in linear regression and Fisher discriminant analysis [101]. The existing statistical software (including SPSS) offers the possibility of computing the VIFs only through a linear regression procedure, although their values are also required in the logistic regression analysis. Therefore, such a procedure must be performed in SPSS to obtain the VIF values; these measures concern the relationships among the independent variables but not the relationships with the dependent variable. Moreover, to run multicollinearity analysis in SPSS, the categorical predictors must be replaced with dichotomous dummy variables [100,102]. After running the required procedure, the resulting values of the VIFs ranged from 1.606 to 3.375, which are below the value of 5. Therefore, the results do not indicate a problematic amount of collinearity [101,103], and the assumption of the absence of multicollinearity is satisfied.

The other assumption of ordinal regression is the proportionality of the odds, which states that the regression coefficients are the same across all categories of the ordinal dependent variable [104]. We examined this assumption through the test of parallel lines conducted in SPSS software, considering the $\overline{x1,x4}$ variables as covariates and x5 as a factor (the x4 variable was considered a covariate since there is an inherent order between the SME and large enterprise in terms of number of employees). We found this test as not significant (χ2 = 9.003, df = 18, p = 0.960) and we concluded that the proportional odds assumption is also held.

Next, we also employed SPSS software to fit the ordinal regression model to our data using the likelihood ratio test [103], which compares the difference in −2log likelihood of the proposed model (10) and the null model (the model with no explanatory variables). The ratio test was significant (χ²diff = 158.597, df = 6, p < 0.001), indicating that our model provides a significant improvement over the intercept-only model. The consistency between the observed data and the fitted model can be tested through the Pearson and deviance goodness-of-fit statistics [104]. We found the Pearson goodness-of-fit statistic (χ² = 308.275, df = 334, p = 0.840 > 0.05) and the deviance measure (χ² = 160.009, df = 334, p = 1.000 > 0.05) as both non-significant, and we may conclude that the observed data are consistent with the fitted model. However, the statistics of the likelihood ratio and goodness-of-fit tests are sensitive to empty cells [104,105]. When estimating models that include covariates, there are often many cells with zero frequencies reported by SPSS. This is our case, and in a situation like this, the significance values of these statistics would not be accurate, and neither of them may be considered reliable [104,105]. Nevertheless, there are several pseudo-R-square statistics that have been advised in examining the association between the dependent variable and the explanatory variables in the proposed model. The pseudo-R-square for McFadden (0.622) should be considered indicative of a very satisfactory fit of the model, as this value tends to be much lower than the R²s for the multiple regression model [106]. Therefore, this value suggests that our model with its independent variables explains a relatively large proportion of the variation among firms in their perception of universities as an external source for open innovation activities. In conclusion, the proposed ordinal regression model is most likely to be a good predictor of the perceived importance of universities in open innovation with firms.

Table 2. Parameter estimates for the ordinal regression model.

		Estimate	Std. Error	Wald	df	Sig.	95% Confidence Interval
							Lower Bound	Upper Bound
Threshold	[y = 1.00]	−3.213	0.935	11.811	1	0.001	−5.045	−1.381
	[y = 2.00]	−1.894	0.722	6.871	1	0.009	−3.309	−0.478
	[y = 3.00]	0.361	0.677	0.285	1	0.593	−0.965	1.688
	[y = 4.00]	2.899	0.765	14.340	1	0.000	1.398	4.399
Location	x1	0.361	0.079	21.146	1	0.000	0.207	0.515
	x2	−0.026	0.048	0.297	1	0.586	−0.120	0.068
	x3	0.220	0.065	11.596	1	0.0006	0.093	0.347
	x4	−0.917	0.615	2.221	1	0.136	−2.122	0.289
	[x5 = 1.00]	−0.603	0.806	0.559	1	0.454	−2.183	0.977
	[x5 = 2.00]	0.043	0.605	0.005	1	0.943	−1.143	1.229
	[x5 = 3.00]	0 a	.	.	0	.	.	.
Link function: complementary log-log.

^a This parameter is set to zero because it is redundant (see Table 1 for the codification of the firm size and industry type, i.e., the x4 and x5 variables).

presents the results of the estimation of the ordinal regression model. The interpretation of these results was mainly focused on the sign and statistical significance of the regression coefficients.

The x1 and x3 explanatory variables were found to have a significant positive impact on the importance of universities as an external source of open innovation for the industry, as their regression coefficients were both positive and statistically significant (p < 0.001). Therefore, hypotheses 1 and 3 are supported. Moreover, as the value of each of these explanatory variables increases, the probability of a higher category of the dependent variable increases, while the other predictors in the model are held constant. The x2 explanatory variable turned out to have a negative impact on the university’s perception of the importance of open innovation with industry. However, this impact was not statistically significant (p > 0.1), and thereby Hypothesis 2 was only partially supported given the insignificance of the barriers. Regarding the control variables x4 and x5, they were also found to be not statistically significant (p > 0.1).

5.2. The ANN Analysis

The x1 and x3 significant hypothesized explanatory variables from the ordinal regression were used as nodes in the input layer of the developed MLP network, which was implemented with the “Neuralnet” package of R software [107]. The resilient backpropagation algorithm (rprop) was employed because of its faster convergence and robustness [108]. More specifically, our MLP network was based on the rprop+ (the rprop with weight backtracking) algorithm with cross entropy as an error function, while its threshold and stepmax values were set as 0.1 and 1 × 10⁷, respectively. Since the sample size of our study is N = 98, the upper bound of the number of hidden nodes for a SLFN is according to relation (12) N_SLFN ≤ 98. Moreover, the upper bounds of the hidden nodes for a TLFN are (N₁)_TLFN = 33 and (N₂)_TLFN = 18, considering the integer values in relation (14). To determine the number of neurons in each hidden layer of the two networks, the dataset was split into training, validation, and test sets with a typical proportion of 70-15-15 [109] taking into account the relatively small size of the sample data.

Different models were built for the SLFN, with the neurons in the hidden layer varying from 1 to 98 (a total of 98 models). For the TLFN, the neurons in the first layer varied from 1 to 33, while in the second layer, they ranged from 1 to 18 (594 models). The training dataset was employed to fit the models in the case of both SLFN and TLFN architectures. Next, the validation dataset was used to compare these models, and the one with the best performance was identified. Finally, the best model was applied against the testing dataset to evaluate its predicting performance. The accuracy criterion given by relation (15) was employed to assess the performance of each model.

Table 3. The performance measures of the best SLFN and TLFN models.

MLP Model	Architecture	Number of Neurons	Accuracy			AUC-ROC
			Training	Validation	Test	Training	Validation	Test
SLFN	1 hidden layer	41	79.687%	70.588%	52.941%	0.894	0.794	0.796
TLFN	1st hidden layer	15	82.812%	76.47%	70.588%	0.900	0.858	0.881
	2nd hidden layer	8

presents the results for the best SLFN and TLFN models, which also include the AUC-ROC measure for each of these models. According to the results in Table 3, the performance measures of the best MLP with two hidden layers model TLFN_(15,8) are better than those of the best with one hidden layer model SLFN₍₄₁₎ for all training, validation, and test sets. At the same time, the performance statistics of the TLFN_(15,8) of the validation and test sets are relatively similar to those of the training set. Therefore, the TLFN_(15,8) network may be considered an adequate predictor of new datasets. Moreover, the 79.687% accuracy of the SLFN₍₄₁₎ model was obtained in the training time of 1.18 s. Regarding TLFN_(15,8) model, an accuracy of 82.812% resulted after a training time of 1.05 s. With this outcome, the TLFN_(15,8) model also gives a better result compared to SLFN₍₄₁₎. Figure 5 depicts the final architecture of the proposed ANN model.

Nevertheless, a k-fold cross-validation was carried out to further avoid overfitting the TLFN_(15,8) model, as the sample size of our study was relatively small. In addition, the k-fold cross-validation gives a reliable perspective on the model’s performance.

Table 4. The 10-fold cross-validation results.

Statistic	Partition	Fold										Mean	Standard Deviation
		1	2	3	4	5	6	7	8	9	10
Accuracy (%)	Training	87.500	67.045	86.364	85.393	65.909	81.818	82.022	81.818	80.682	62.500	78.105 (%)	8.796 (%)
	Test	50.000	70.000	60.000	66.667	60.000	70.000	66.667	60.000	70.000	60.000	63.333 (%)	6.146 (%)
AUC-ROC	Training	0.934	0.867	0.937	0.931	0.842	0.920	0.919	0.885	0.914	0.845	0.8994	0.0348
	Test	0.754	0.800	0.722	0.625	0.833	0.888	0.916	0.850	0.777	0.694	0.7859	0.0856

shows the accuracy and AUC-ROC statistics of a 10-fold cross-validation process. The values of the performance measures of the test set are smaller, which is usual [110], but not substantially different from those of the training dataset. Thus, the TLFN_(15,8) can be considered a robust model for predicting independent datasets.

6. Discussion and Conclusions

Opening up their innovation activities has become an important way for firms to compete in today’s dynamic and changing environment. Although industry may draw knowledge and technologies from different sources, the scientific knowledge of universities is regarded as having a great potential to improve firms’ competitiveness [111], considering its unique features compared to other sources [30]. The understanding of how industry engages in collaboration with universities in open innovation has recently seen important advances [112,113]. Accordingly, it appears that motives, barriers, and channels of knowledge transfer are important antecedents in explaining the propensity of firms to employ universities in their open innovation activities. Despite this fact, an integrative approach that investigates their impact on the perception of universities as an important external knowledge source for open innovation is still lacking. Within this context, we developed a research framework to fill the above research gap in the open innovation literature.

The findings of our analysis provided some interesting insights regarding the influence of the proposed antecedents of industry-university collaboration for open innovation and the resulting impact on the perception of the importance of universities as a source of knowledge in the open innovation context. Conceived as fundamental reasons for firms to engage in collaboration with universities, motives have been seen as anticipated benefits [114]. In collaboration with universities, firms aim to capture both their tacit and highly codified scientific knowledge using proactive and passive strategies, respectively [115]. Thus, several studies reviewed the motivations of firms to collaborate with universities in an open innovation context [50,57,58]. The inbound knowledge flow between the university and industry may each have ‘non-pecuniary’ or ‘pecuniary’ motives [116], which can be used to explain why they collaborate in open innovation. In this sense, our study included motives that belong to both the unmonetized and monetized flows of knowledge. Our findings indicated that, taken together, these motives have a significantly positive impact on the perception of universities as a knowledge source for industry in open innovation. In other words, universities are perceived as more important sources for firms that have expressed more important motives for their collaboration in open innovation. Therefore, an understanding of the nature and importance of the motives becomes essential to the successful implementation of open innovation between industry and university actors.

Since innovation is considered a complex phenomenon that is subjected to uncertainty and changes, the collaboration between industry and universities in such a setting may face significant challenges [117]. Inherent with the complexity of the open innovation context, the collaboration between the two organizations in this process is hindered by different barriers. Identifying these barriers and eventually overcoming them is also crucial for successfully implementing open innovation activities [61]. For this purpose, our study comprised barriers that can be delineated based on the same logic of pecuniary and non-pecuniary challenges to open innovation [53]. As expected, we found at an overall level that these barriers have a negative impact on the perceived importance of universities as a knowledge source for industry in open innovation. However, these barriers do not significantly influence this perception. A possible explanation of this result is related to the criterion for inclusion in the sample of only the firms that cooperate with universities. Despite the recognition of the presence of barriers to open innovation, they might learn how to gain scientific knowledge from universities, although this process requires time, resources, and extensive effort [118,119]. As a result, firms may become more open and confident in partnering with universities in order to attain their goals [120].

According to extant literature [41,58], knowledge flows through many channels during industry-university collaboration in open innovation. Different perspectives on categorizing these channels have been proposed by the scholars, such as the “dominant mechanisms employed” considered by Gilsing et al. [121] or the “dominant mode of governance” illustrated by Alexander et al. [41]. The science versus development-based regimes criterion was used in the former, while the relational versus transactional style of engagement was distinguished in the latter. Each of these transfer mechanisms shows its own unique characteristics, and it should be noted that in most situations, multiple connections are used together depending on the transfer objectives [122]. On the whole, our study involved transfer channels that belong to both perspectives described above, which can also be analyzed based on the same pecuniary versus non-pecuniary logic of knowledge transfer proposed by Dahlander and Gann [53]. Altogether, these channels have a positive and significant impact on the perception of the importance of universities as an external knowledge source for industry in open innovation. Therefore, universities are perceived as more important sources for firms that employ more channels for their collaboration in open innovation. This result supports the view of Meissner and Carayannis [122] and the findings of Costa et al. [63] regarding the diversity of knowledge sources, which pointed out that the promotion of multiple channels of collaboration rather than a single one may be more effective. Although a deep search across a wide variety of knowledge transfer channels can provide valuable resources to industry in exploiting innovation opportunities, the costs of such search efforts should also be understood and not dissipated across too many external sources [64].

The control variables also offered some insights in terms of the size of the firm and industry type. Specifically, the sign of the firm size variable in Table 2 suggested that the SMEs (coded with 1) are more likely to assign higher ratings on the importance perception of universities in their collaboration for open innovation than large firms (coded with 2). At the same time, the impact of the firms’ size on the open innovation paradigm is still under debate [123,124]. On one hand, SMEs have to draw more from open innovation initiatives than large firms, as they are generally less research-intensive, but they also possess more limited internal capabilities and resources for managing such initiatives [112]. On the other hand, large firms may be more responsive to open innovation with universities as they carry out more formal research and development activities and have greater resources and time to build their collaboration than SMEs [111,125], but they also have more capabilities to conduct internal innovation activities. With regard to the industry type, our analysis has mixed results, as shown in Table 2. Our finding suggested that the high-tech industry (coded with 1) is less likely, while the medium high-tech industry (coded with 2) is more likely to assign higher ratings on the importance perception of universities in their collaboration for open innovation than the low-tech industry (coded with 3). Although open innovation practices vary across sectors [52], this finding is in part different from other research such as that of Kanama and Nishikawa [126], which found medium- and high-tech industries as having the tendency to use more universities as an external knowledge source. It also should be noted that control variables do not offer any significant impact, which may be related to the limited homogeneity of the participants from the two industrial areas employed in our sample.

As industry searches to become more innovative, understanding the antecedents’ impact on the perception of universities as an external knowledge source to industry may help firms develop specific strategies to sustain those antecedents that have a positive effect and diminish the influence of those with a negative one. In this way, more effective and efficient employment of open innovation between the two partner organizations is expected, which may increase their collaboration in such a context through suitable policies and procedures.

6.1. Concluding Remarks

The interaction between firms and universities continues to increase in the context of the open innovation paradigm [122], where motivations, barriers, and channels of knowledge transfer have been considered as major antecedents of their collaboration [16]. However, their impact on the perception of universities as an important external knowledge source for industry has not been explored yet, which has resulted in a limited understanding of such influence. Aiming to fill this gap, we developed a research framework based on the extant literature that related the impact of the above antecedents to the perceived importance of universities as an external knowledge source to industry in open innovation. As a result of the hypotheses of our study, we provide a first perspective of which of the three antecedents significantly impact this perception and contributes to a more articulate view of the collaboration between these two actor partners in an open innovation context.

On the whole, the findings of our study expand the existing literature in the open innovation field and have both theoretical and practical implications. The main theoretical contribution consists in modeling the antecedents’ impact on the importance perception of universities as an external knowledge source to industry in open innovation based on the two-stage ordinal regression and neural network approaches. Although some studies have been conducted to adapt ANNs to learn ordinal categories [92,127], a hybrid ordinal regression and ANN modeling approach has been rarely described in the literature. Considering the order relation between categories of perceived importance, an ordinal regression approach was employed in the first stage for modeling the relationship between the response and explanatory variables of our research framework. However, such modeling presents some shortcomings related to the non-linearity of the model and data distribution. At the same time, the ANNs can overcome these shortcomings but are not suitable for hypothesis testing because of their algorithm’s difficulty in determining how the decision is made. Within this context, in the second stage, an ANN analysis was carried out that included shallow and deep ANNs. Both types of architecture employed the significant explanatory variables obtained in the first stage and the dependent variable of our study as inputs and outputs, respectively. Moreover, the CDF of the distribution used in obtaining the link function of the ordinal regression model through its inverse transformation was also employed as the activation function for each output node of the ANNs.

In the first stage, the regression coefficients of the resulting model are somewhat more difficult to interpret [81]. Therefore, our analysis was focused on their sign to examine the direction of the effect of the explanatory variables on the response variable and their statistical significance to test the hypotheses of the study. The results have shown that hypotheses H1 and H3 are fully supported, while hypothesis H2 is only partially supported, which suggests a differentiated impact of the analyzed antecedents. The sign and statistical significance of the findings of this work are relatively similar to those of other studies in which methods such as SEM were employed (e.g., [17]). Therefore, reaching such conclusions with different approaches may contribute to supporting their validity.

From a practical perspective, these findings provide an insight into the antecedents that firms should sustain to exploit open innovation opportunities, as well as those they should mitigate in their collaboration with universities. We also found in the second stage that a two-hidden-layer MLP network is more effective in modeling the complex relationship between the significant explanatory variables obtained in the first stage and the dependent variable of our study, which is consistent with the existent literature [19]. In this way, we focus on quantifying the antecedents’ impact on the perceived importance of universities as an external knowledge source and offer a practical tool to predict such an impact.

6.2. Limitations and Direction for Future Research

Like any other exploratory research, our study has several limitations that have to be acknowledged, which also points out future research directions. First, to our best knowledge, very little theoretical or empirical evidence related to the developed research framework is available in the literature. On the one hand, it is possible that some of its components might not be considered. Nevertheless, our framework has an open and flexible architecture that allows the incorporation of any new components. And future studies will have to confirm, modify, or even reject some of these findings once such components are distinguished in future research. On the other hand, the three variables of our study expressed by the relations (1), (3), and (5) are relatively simple constructs that were defined based on the seminal works in the field of open innovation [64,70]. Nevertheless, each of them has a relatively high internal consistency, a similar result to those pointed out in the literature [64,70]. However, the statistical significance of the proposed variables may be of higher relevance if they are validated by both confirmatory and exploratory factor analysis, which is expected to be performed in our future studies.

Second, the findings resulting from this study should be viewed within the described context of the two industrial areas and based on a relatively small sample size. Moreover, we employed a purposive sampling to cover as many opinions as possible [128], which also may induce some limited homogeneity among the participants. Considering its exploratory nature, this work is also intended to draw a tentative outline for future research in the analyzed field and requires replication before generalizing its findings. Hence, future additional research is required so that its conclusions can be fully useful in the practice of open innovation between industry and universities. As Villasalero [14] pointed out, future studies that involve larger samples should be conducted in different settings and different national operating environments to find the extent to which the analyzed impact is conditioned by the features existing in such contexts.

Third, the proposed activation function for the output nodes of ANNs does not guarantee the rank-monotonicity of the outputs, a desirable characteristic in prediction-making although not necessary [92]. More sophisticated modeling in the direction suggested in [127] should be performed in future studies to explore if it will lead to better performance in the context of our approach.

Lastly, this study reflects only the firms’ perspective regarding the antecedents’ impact of opening up innovation on the perceived importance of universities as an external knowledge source to industry. Future research should also address this impact from the point of view of the academic institutions open innovation with industry. Findings from comparing the perspectives of both organizational actors are expected to provide empirical evidence that can be used to improve their collaboration in an open innovation context.