Exploring Potential R&D Collaboration Partners Using Embedding of Patent Graph

Abstract

Rapid market change is one of the reasons for accelerating a technology lifecycle. Enterprises have socialized, externalized, combined, and internalized knowledge for their survival. However, the current era requires ambidextrous innovation through the diffusion of knowledge from enterprises. Accordingly, enterprises have discovered sustainable resources and increased market value through collaborations with research institutions and universities. Such collaborative activities effectively improve enterprise innovation, economic growth, and national competence. However, as such collaborations are conducted continuously and iteratively, their effect has gradually weakened. Therefore, we focus on exploring potential R&D collaboration partners through patents co-owned by enterprises, research institutions, and universities. The business pattern of co-applicants is extracted through a patent graph, and potential R&D collaboration partners are unearthed. In this paper, we propose a method of converting a co-applicant-based graph into a vector using representation learning. Our purpose is to explore potential R&D collaboration partners from the similarity between vectors. Compared to other methods, the proposed method contributes to discovering potential R&D collaboration partners based on organizational features. The following questions are considered in order to discover potential R&D partners in collaborative activities: Can information about co-applicants of patents satisfactorily explain R&D collaboration? Conversely, can potential R&D collaboration partners be discovered from co-applicants? To answer these questions, we conducted experiments using autonomous-driving-related patents. We verified that our proposed method can explore potential R&D collaboration partners with high accuracy through experiments.

1. Introduction

In the increasingly complicated global technology market, enterprises need to consider research and development (R&D) strategies for ambidextrous innovation to survive. Burnett and Williams (2014) insisted that enterprises should be prepared for innovation acceleration owing to rapid technological change [1]. Similarly, Danneels (2007), Hung and Chou (2013), and Obradović et al. (2021) stressed the need for strategies to rapidly discover novel collaboration opportunities as technology lifecycles are becoming shorter [2,3,4].

Nonaka (1994) mentioned that knowledge conversion occurred through a continuous dialog between explicit and implicit aspects of knowledge owned by enterprises [5]. He especially stressed that the knowledge of an enterprise is combined, transferred, and created through socialization, externalization, combination, and internalization. Accordingly, enterprises need to determine how to diffuse their tacit knowledge to external entities in the current era.

Brown and Eisenhardt (1997) emphasized that a cornerstone change, such as in the culture of the enterprise, was needed for a successful business [6]. Dyer and Singh (1998) mentioned that inter-organizational combinations such as enterprises, research institutions, and universities played an important role in diffusing knowledge [7]. Thus, enterprises are expected to input sustainable resources for collaborative R&D and acquire larger market value through cooperation [8,9,10]. Thus, the discovery of potential R&D collaboration partners is directly related to the survival of various organizations.

Chen and Kenney (2007), Yang et al. (2010), Gao et al. (2011), and Temel and Glassman (2013) agreed with the opinion that R&D collaborations were effective at achieving efficient innovation and improving the innovation competence of nations by promoting economic growth [11,12,13,14]. Many organizations also agree with their opinions and exchange human, financial, and technical activities to promote a win–win idea. Organizations are promoting proactive university–industry collaboration (UIC) and academia–industry collaboration (AIC) for their survival. They exchange material and technical resources consumed in R&D through UIC and AIC, saving time and cost. However, Gulati (1995) and Capaldo (2007) pointed out that although long-term collaboration between the same partners may reduce the cost of development of innovation, an excessively iterative frequency may cause negative effects [15,16].

Durmuşoğlu and Durmuşoğlu (2022) argued that patents can be used to discover opportunities for technological innovation in various fields, from traffic systems to healthcare [17,18]. Zhang et al. (2017) defined a patent as a concept that implies evidence of technological innovation [19]. Conventional studies use co-patents to discover potential R&D collaboration partners. This is because co-patents, which are jointly owned by two or more organizations, are collaborative outcomes [20,21]. In particular, co-applicants who share the ownership of co-patents may exercise technological potential in the market through R&D collaboration. Furthermore, Briggs (2015) discovered that patents whose co-applicant was an AIC had larger influence in exercising the potential than other patents [22].

Researchers have considered the diffusion of implicit knowledge for knowledge transfer, combination, and creation in the accelerating global market. The discovery of potential R&D collaboration partners through collaboration between organizations is the solution that researchers found. However, the effect has been gradually weakened because of continuous and iterative behaviors. Consequently, a novel approach is now needed to discover potential R&D collaboration partners. We focus on discovering potential R&D collaboration partners with patents owned by co-applicants such as enterprises, research institutions, and universities. The purpose of this study is to view co-applicants from the knowledge graph viewpoint and discover potential R&D collaboration partners from the pattern. To this end, we will (i) extract co-applicants from patents, (ii) convert them into knowledge graphs, (iii) embed co-applicants into a vector space, and (iv) identify potential R&D collaboration partners through similarity-based relevance algorithms.

Our contributions are as follows:

• A patent owned by co-applicants is a piece of work by collaboration. Thus, co-applicants are organizations that actively use the current business opportunities. Accordingly, this paper aims to explore the future of collaborative R&D by finding a pattern between co-applicants from a knowledge graph.
• A knowledge graph is an excellent tool that can express a complex relationship between numerous objects. Therefore, a knowledge graph is well-suited to identifying the collaboration history of co-applicants such as enterprises, research institutions, and universities. Furthermore, we aim to identify potential R&D collaboration partners by converting the co-applicants in the graph to a vector and measuring the collaborative similarity.
• In this study, the relevance algorithm that measures the collaborative similarity of co-applicants is free from specific assumptions. Therefore, the proposed method can be used regardless of the technical field and organization type.

Therefore, we ask the following questions to discover potential R&D collaboration partners in collaborative R&D: Can information about co-applicants of patents explain business opportunities? If so, can potential R&D collaboration partners be discovered from co-applicants? In order to answer these research questions, the rest of the paper is structured as follows. Section 2 reviews previous studies to discover business opportunities. Section 3 explains the preliminary theory and the methodology in detail. Section 4 presents the empirical evidence for the research question. Finally, Section 5 discusses the implications of the proposed methodology, its limitations, and future research directions to overcome them.

2. Literature Review

Internal and external collaboration R&D supports various business activities of organizations. Therefore, a novel methodology for the discovery of potential R&D collaboration partners should efficiently manage and analyze the information of organizations. In this section, previous studies on the discovery of business opportunities and measures to alleviate the limitations are reviewed.

2.1. Related Works

Kim and Inkpen (2005) and Santamaría et al. (2021) stressed the need to consider the organization’s nationality for future collaboration [23,24]. A patent map is effective in discovering business opportunities by visualizing various features of technologies. Yoon et al. (2002) and Kim et al. (2008) provided effective patent maps by clustering patents to explore business partners [25,26]. Furthermore, Lee et al. (2009) and Yoon et al. (2013) extracted various variables of patents by reducing the patent data dimension to increase the effect of the patent map [27,28]. Lee and Lee (2019) proposed a patent map that can visualize a trend of technology to identify business opportunities [29].

Yoon et al. (2015) proposed a functional framework that can discover novel opportunities from developed technologies [30]. They asserted that technology and collaborative R&D modeling were effective when discovering business opportunities. Faccin and Balestrin (2018) mentioned that as collaborative R&D changed and began depending on strategy, stock, and type of knowledge creation, a model that can explain this was needed [31]. Messeni Petruzzelli and Murgia (2020) explored the factors that can explain knowledge transfer according to the stock and type of organization [32]. Furthermore, Murgia (2021) proved that collaboration heterogeneity was affected by nationality and institutional diversity [33].

Existing studies considered the geographical distance of organizations to discover business opportunities. Maggioni and Uberti (2009), Burhop and Wolf (2013), and Drivas and Economidou (2015) analyzed the influence of geographical distance on knowledge innovation [34,35,36]. They discovered that specific organizations contributed to the diffusion of knowledge, and their influence was considerable. Capaldo and Petruzzelli (2014) mentioned that the closer the geographical distance, the larger the positive effect on collaborative R&D when selecting a business partner [37].

Golosovsky and Solomon (2017) and Ardito et al. (2018) emphasized that a knowledge graph was a suitable tool to not only explore the internal structure beyond co-patents but also measure the importance of collaborative innovation [38,39]. Centrality is a simple and effective approach for discovering potential R&D collaboration partners from knowledge graphs. Goetze (2010) attempted the extraction of major organizations from knowledge graphs for human resource management in R&D [40]. In addition, knowledge graphs using patents are also used to discover major nations and inventors in a specific technology field [41].

In recent years, studies on the use of patent texts or technology classification codes have been conducted to discover business opportunities. Lee et al. (2022) discovered novel technology-based enterprises and opportunities with technologies classified through text mining [42]. Seo (2022) developed a method using topic-based knowledge graphs and association rule mining by analyzing patent texts to explore business opportunities [43]. In addition, Wu et al. (2023) designed a multilayer network based on the international patent classification (IPC) code [44].

We reviewed previous studies and found that patent-based knowledge graphs were effective in discovering business opportunities beyond technologies. However, previous studies have some limitations. First, they could be used to visualize a relationship between technology and organization using patent maps, but the discovery of business opportunities could not properly reflect the features of organizations. Second, the functional-based framework could be used to explore factors from existing business opportunities, but it has many difficulties in discovering potential opportunities. Third, geographical distance-based approaches may be unsuitable in a worldwide pandemic situation. This was because organizations conducted collaborative R&D in this era regardless of their environment or location. Finally, knowledge graphs using patent topics or IPC codes are dependent on the main keywords of technology. Thus, this study proposes a method that uses embedding of the co-applicant graph and similarity-based relevance algorithms to improve the limitations of previous studies.

2.2. Node Embedding

This paper proposes a representation-learning-based methodology to extract features of nodes (or vertices) and links (or edges) that make up a knowledge graph. A graph is suitable for expressing ever-increasing data and a complex relationship that connects data. The graph is defined as $G=\left(V,E\right)$, where $V$ denotes the set of nodes and $E$ denotes the edge set.

The graph is a method to express data using nodes and links. Graphs in existing studies have been analyzed through approaches such as the number of edges connected to a node. In recent years, representation learning of unstructured data such as images or texts has advanced. Following this trend, several studies have proposed representation learning that can be applied to graphs. Perozzi et al. (2014) argued that a graph connected by nodes was similar to the pattern of words that make up a sentence [45]. They applied a sequence of nodes extracted from a graph to SkipGram. SkipGram, which is one of the typical language models, embeds natural language into a vector space by maintaining the co-occurrence probability of words [46,47,48].

Let us assume a sequence of words having length $T$ as $w_1,w_2, \dots , w_T$. SkipGram learns to have the maximum co-occurrence probability with neighboring words to obtain features of the center word $w_t$. The objective function of SkipGram is presented in Equation (1).

$$\frac{1}{T}{{\displaystyle \sum }}_{t=1}^T{{\displaystyle \sum }}_{-k\le j\le k,j\ne 0}\log P\left(w_{t+j}|w_t\right)$$

(1)

where $k$ is the size of the context.

DeepWalk searches for a pattern from a sequence of nodes through SkipGram [45]. First, DeepWalk uses a transition probability based on a random walk to extract a sequence of nodes from the graph $G=\left(V,E\right)$. When a random walk whose length is $l$ is extracted from the source node $u$, the selection probability of the $m$-th node can be calculated using the following equation.

$$P\left(u_m=w|u_{m-1}=v\right)=\left\{\begin{array}{cc}\frac{{\pi }_{vw}}{Z}& if \left(v,w\right)\in E\\ 0& otherwise\end{array}\right.$$

(2)

where $v$ and $w$ refer to the $\left(m-1\right)$-th and $m$-th selected nodes, and $u_0=u$. ${\pi }_{vw}$ refers to the unnormalized probability when $\left(v,w\right)\in E$, and $Z$, the sum of ${\pi }_{vw}$, refers to the normalizing constant.

Grover and Leskovec (2016) proposed the second-order random walk that can extract local and global information from graphs through the search bias [49]. Their proposed approach, Node2Vec, employed a weighted transition probability. The transition probability weight ${\alpha }_{vw}$ used in Node2Vec is presented in Equation (3):

$${\alpha }_{vw}=\left\{\begin{array}{cc}\frac{1}{p}& if s{p}_{vw}=0\\ 1& if s{p}_{vw}=1\\ \frac{1}{q}& if s{p}_{vw}=2\end{array}\right.$$

(3)

where $p$ and $q$ refer to the return parameter of the likelihood of immediately revisiting a node in the walk and the in–out parameter of differentiating between inward and outward nodes, respectively. $s{p}_{vw}$ denotes the length of the shortest path between $v$ and $w$.

DeepWalk and Node2Vec embed node $V$ in the $\left|V\right|\times D$ dimension through SkipGram. The vectorized graph node is used in various application fields such as technology forecasting, financial fraud detection, and analysis of user behavior in addition to document clustering and community detection [50,51,52,53,54].

2.3. Link Prediction

A link in a knowledge graph explains the relationship between nodes. A graph is classified into unweighted and weighted graphs according to the feature of the link. An unweighted graph refers to a graph expressed as 1 when a link connecting two nodes exists and 0 otherwise. A distance matrix is a typical example of a weighted graph. That is, the distance between two nodes is expressed by the weight of the link.

Let us assume that $E$ is $E\subseteq \left(V\times V\right)$ in graph $G=\left(V,E\right)$ and the label of the edge between two nodes is $y$. For example, in an unweighted graph, it is $y\in \left\{0,1\right\}$. Link prediction can be used to forecast a relationship between future nodes. Conventionally, a heuristic score was used to predict a link between nodes $u$ and $v$.

Let $N\left(u\right)$ be the neighborhood of node $u$. Common neighbors predict a link based on the number of neighborhood intersections $N\left(u\right)\cap N\left(v\right)$ between two nodes. Jaccard’s coefficient uses the intersection and union of the neighborhood of two nodes [55]. Jaccard’s coefficient is presented in the following equation.

$$\frac{\left|N\left(u\right)\cap N\left(v\right)\right|}{\left|N\left(u\right)\cup N\left(v\right)\right|}$$

(4)

where $N\left(u\right)\cup N\left(v\right)$ denotes the union of the neighborhood of $u$ and $v$.

The Adamic–Adar score employs a subset of the intersection of the neighborhood of two nodes [56]. The Adamic–Adar score is presented in the following equation.

$${{\displaystyle \sum }}_{w\in N\left(u\right)\cap N\left(v\right)}\frac{1}{\log \left|N\left(w\right)\right|}$$

(5)

where $w$ refers to the element in the subset of the neighborhood of $u$ and $v$.

Let $f:V\times V\to {ℝ}^D$ be the function that maps edge $E$ to dimension $D$. Then, the predictive model $h$ for link prediction is presented in the following equation.

$$\mathit{arg}\underset{h}{\mathit{min}} \mathcal{L}\left[y, h\left(f\left(V\times V\right)\right)\right]$$

(6)

where $\mathcal{L}$ denotes the loss function of predictive model $h$.

3. Discovering Collaboration Partners

This study aims to explore potential co-applicants to discover new business opportunities. The proposed method employs a knowledge graph that reflects the information of the patent’s co-applicants. Now, let $S_i$ be the set of applicants of the $i$-th patent $P_i$. The knowledge graph of the co-applicant is defined as $G=\left(V,E\right)$, where $V$ denotes the set of $\forall S_i$ and $E$ denotes the applicant–applicant edge set. In graph $G$, $\left|V\right|$ and $\left|E\right|$ are the number of nodes and edges, respectively. We introduce a methodology that embeds applicants and co-applicants through $G$.

Figure 1 shows a flowchart of the methodology for discovering business opportunities. First of all, the co-applicant is extracted from a patent, which is a scientific document. The co-applicant transforms into a knowledge graph $G$. Next, a sequence of applicants is extracted from the knowledge graph of co-applicants. SkipGram learns the relationship between nodes from the sequence of applicants and converts them into vectors. Finally, MMR discovers potential R&D collaboration partners in the vector space. The remaining subsections describe the proposed method in detail.

3.1. Preliminary Theory

Let $f: V\to X_V$, like SkipGram or Node2Vec, be a function that maps words in a sentence or nodes in a network to the $D$-dimension, where $X_V\in {ℝ}^{\left|V\right|\times D}$. Perozzi et al. (2014) discovered that nodes in a network followed a power-law distribution (or Zipf’s law) like natural language [45]. Figure 2a shows the power-law distribution of the Reuters newswire dataset. Thus, we determined whether patent applicants follow a power-law distribution for the embedding of applicants.

Figure 2b shows the power-law distribution of applicants of patents that were filed from 15 March 2001 to 8 December 2022, in the United States Patent and Trademark Office [57]. The distribution of the applicants was similar to that of natural language. Therefore, we approach the embedding of co-applicants based on this empirical evidence. Figure 2c is the power-law distribution of applicants for 873 patents in relation to autonomous driving used in the experiment in this paper. It can be seen that the data we collected are similar to the distributions in Figure 2a,b. In Section 4, Figure 2c will be explained in detail.

3.2. Embedding Collaboration Partners

This paper proposes a methodology based on co-applicants to discover new business opportunities. The co-applicants of patents can be divided into academia (university, institution) and industry (enterprise). The purposes of the proposed method are: (i) extracting latent features of co-applicants and (ii) exploring new business partners. The proposed method broadly involves three steps.

A graph can express co-applicants of patents. In the co-applicant graph $G$, the neighborhoods of node $v$ are likely to be co-applicants. Thus, our approach samples $v$ and $v$ ‘s neighbors, $N\left(v\right)$, to maintain relationships with co-applicants. Mapping function $f$ is also learned to make $v$ and sampled neighborhood $N\left(v\right)$ of $v$ closer in the low dimension. Equation (7) is the objective function of co-applicant embedding.

$$\underset{f}{\mathit{max}} {\displaystyle \sum }_{v\in V}\mathit{log}P\left(N\left(v\right)|f\left(v\right)\right)$$

(7)

The mapping function $f$ returns the vector of the applicant. However, we need the co-applicant vector to discover potential business partners. Now, let $u$ and $v$ be the co-applicants of $P_i$. That is, for $u$ and $v$ satisfying $\left(u,v\right)\in E$, $f\left(u\right)$ and $f\left(v\right)$ are $D$-dimensional vectors. The vector of the co-applicant for node (applicant) $u$ and $v$ is presented in Equation (8).

$$g_{uv}=f\left(u\right)\circ f\left(v\right)$$

(8)

where $g_{uv}:f\left(u\right)\circ f\left(v\right)\in {ℝ}^D$.

Table 1. List of operators for embedding collaboration partners.

Operator	Definition
Average	$0.5\times \left(f^d\left(u\right)+f^d\left(v\right)\right)$
Hadamard	$f^d\left(u\right)\times f^d\left(v\right)$
Weighted-L1	$\left|f^d\left(u\right)-f^d\left(v\right)\right|$
Weighted-L2	${\left|f^d\left(u\right)-f^d\left(v\right)\right|}^2$

presents Average, Hadamard, Weighted-L1, and Weighted-L2, which are typical operators of $g$ [49]. In operator, $f^d$ refers to the $d$-th element of $D$-dimension vector $f$. Average and Hadamard operators return the element-wise addition and production of two vectors, respectively. Furthermore, Weighted-L1 and L2 operators return element-wise L1-norm and L2-norm.

3.3. Exploring Collaboration Partners

The first step of the proposed method is to build a knowledge graph using co-applicants. The second step is to convert the co-applicant in the knowledge graph to a vector. Finally, the third step is to discover potential R&D collaboration partners from the co-applicant, which is converted from the vector. To achieve the goal of the third step, we use similarity-based relevance algorithms.

Carbonell and Goldstein (1998) and Bennani-Smires et al. (2018) proposed maximal marginal relevance (MMR) to prevent the combination of duplicate keywords in information retrieval [58,59]. MMR has been used in various domains because it can select keywords by reflecting semantic similarity. MMR, which is applied in this study to discover novel co-applicants, is presented in Equation (9).

$$\mathit{arg}\underset{g_{..}\in E/C}{\mathit{max}} \left[\gamma ·sim\left(g_{..},g_{uv}\right)-\left(1-\gamma \right)·\underset{g_{wx}\in C}{\mathit{max}}sim\left(g_{..},g_{wx}\right)\right]$$

(9)

For the edge set $E$ in the graph $G$, $C$ refers to a subset of candidate co-applicants. Then, MMR returns $g_{..}$, which is similar to that of $g_{uv}$ and different from that of already selected $g_{wx}$. In the equation, $\gamma$, which has a value from 0 to 1, is a parameter that controls selection diversity. The closer the gamma is to 1, the better MMR helps select diverse co-applicants. $sim$ is a function to measure the similarity of two vectors. When cosine similarity is used, $sim$ of $g_{uv}$ and $g_{wx}$ is as presented in Equation (10).

$$sim\left(g_{uv},g_{wx}\right)=\frac{{{\displaystyle \sum }}_d{g}_{uv}^d\times g_{wx}^d}{\sqrt{{{\displaystyle \sum }}_d{\left(g_{uv}^d\right)}^2}\times \sqrt{{{\displaystyle \sum }}_d{\left(g_{wx}^d\right)}^2}}$$

(10)

where $g_{uv}^d$ refers to the $d$-th element of $g_{uv}$.

The reasons for the suitability of the MMR-based search to achieve the goal of the proposed method are as follows: First, MMR does not require statistical assumptions. This is because MMR is run with simple operations in the vector space. Second, the maximum value of similar objects that can be selected by MMR is $\left|V\right|\times \left|V\right|$, which is very large. Thus, researchers can determine the number of objects that can be searched depending on the purpose of the analysis. Finally, MMR can discover new co-applicants, which are different from existing patterns. Simply put, if objects are selected in the most similar order, expected co-applicants can be extracted.

We define the novel links (co-applicants) selected by MMR as potential R&D collaboration partners. The reasons for this are as follows: First, it is highly likely that the co-applicant selected by MMR is not an existing business partner. Second, the vector of the co-applicant is selected from the node embedding. That is, the vector of co-applicants reflects the collaborative business relationship between patent applicants. Finally, we discovered empirical evidence that patent applicants follow a power-law distribution, which was the same as words in a sentence and nodes in a graph. This is an answer that clearly demonstrates the complex principles required for the applicant to show suitability for node embedding and link prediction.

4. Experimental Results

The purpose of this study is to explore latent co-applicants to discover potential R&D collaboration partners. To do this, we proposed a method where node embedding and MMR were applied to the co-applicant graph. Our experiment was conducted in the following order: data collection, co-applicant embedding, and exploring potential co-applicants. All visualizations of the graphs in the experiment are the results processed by Gephi software (version 0.9.7, accessed on 15 May 2023: https://gephi.org).

4.1. Experimental Setup

For this experiment, we collected 873 patents in relation to autonomous driving. Autonomous driving is one of the technologies that have attracted considerable attention from many nations and enterprises. Thus, we collected patents about autonomous driving technology, which were filed in China, Europe, Japan, South Korea, and the United States of America. Figure 2c shows the distribution of autonomous-driving-related patents. We determined that because applicants of autonomous driving technology followed a power-law distribution, applying the proposed method was suitable.

Figure 3 shows the co-applicant graph of the collected dataset. The size of the node denotes the number of connected links. The color of the node indicates the country in which the paper was filed. Orange represents China, green represents Europe, red represents Japan, blue represents South Korea, and gray represents the United States of America. The original names of the applicants indicated in the nodes are summarized in

Table A1. List of applicant abbreviations.

Abbreviation	Applicant	Abbreviation	Applicant
Advics	Advics Co., Ltd.	LG	LG Electronics, Inc.
Aisin	Aisin Seiki Kabushiki Kaisha	Lotte	Lotte Corporation
Audi	Audi AG	Lotus	Lotus Thermal Solution, Inc.
Axia	Axia Materials Co., Ltd.	Mazda	Mazda Motor Corp.
BMW	Bayerische Motoren Werke Aktiengesellschaft	Mitsubishi	Mitsubishi Motors
Bosch	Robert Bosch GmbH	Nissan	Nissan Motor Co., Ltd.
Chrysler	Chrysler Group LLC	P	Patents filed by individuals
Continental	Continental Automotive GmbH	PanKorea	PanKorea
Daihatsu	Daihatsu Motor Co., Ltd.	Renault	Renault SAS
Daimler	Daimler AG	Samsung	Samsung SDI Co., Ltd.
Dasan	Dasan	SK	SK Hynix, Inc.
Denso	Denso Corporation	Soken	Soken, Inc.
Doowon	Doowon	Sumitomo_Wiring	Sumitomo Wiring Systems, Ltd.
Egtronics	Egtronics Co., Ltd.	Suzuki	Suzuki Motor Corp.
EJR	East Japan Railway Co.	Tamagawa	Tamagawa Seiki Co., Ltd.
Fujitsu	Fujitsu Limited	Toshiba	Toshiba Corp.
Gate	Gates Korea Co., Ltd.	Toyota	Toyota Motor Corp.
Geely	Geely Co., Ltd.	U_Chungang	Chung-Ang University
GM	GM Global Technology Operations LLC	U_Chungnam	Chungnam National University
Hain	The Hain Celestial Group, Inc.	U_D’orleans	Université d’Orléans
Hanon	Hanon Systems	U_Fudan	Fudan University
Hawei	Huawei Technologies Co., Ltd.	U_Huazhong	Huazhong University
Hitachi	Hitachi, Ltd.	U_Korea	Korea University
Honda	Honda Motor Co., Ltd.	U_Kumoh	Kumoh National Institute of Technology
Hyundai	Hyundai Motor Company Co., Ltd.	U_Marin	Korea Maritime & Ocean University
I_GHI	Gyeongbuk Hybrid Technology Institute	U_Nagaoka	Nagaoka Univ of Technology
I_GHIM	Global Frontier Hybrid Interface Materials	U_Pukyong	Pukyong National University
I_KECA	Korea Electrical Contractors Association	U_Regents	Regents of the University
I_KICCET	Korea Institute of Ceramic Engineering and Technology	U_Seoul	Seoul National University
I_KICT	Korea Institute of Civil Engineering & Building Technology	U_SeoulTech	Seoul National University
I_KIMM	Korea Institute of Machinery & Materials	U_Soongsil	Soongsil University
I_KIST	Korea Institute of Science and Technology	U_Stuttgart	UNIVERSITÄT Stuttgart
I_Kost	Korea Transportation Safety Authority	U_Sungkyunkwan	Sungkyunkwan University
I_MultiEnergy	Global Frontier Center for Multiscale Energy System	U_Transport	Korea National University of Transportation
Infineon	Infineon Technologies AG	U_woosuk	Woosuk University
Inosuki	Inosuki	U_Yamaguchi	Tokyo Univ of Science Yamaguchi
Jaeshin	Jaeshin PowerTech	U_Yeongnam	Yeungnam University
JATCO	JATCO Ltd.	Unichem	Unichem Co., Ltd.
JSMEA	Japan Ship Machinery & Equipment Association	Unicks	Unick Corporation
KAIST	Korea Advanced Institute of Science and Technology	ValeoKapec	Valeo Kapec Co., Ltd.
Kia	Kia Motors Corp.	Vinatech	VINATech Co., Ltd.
Kyungshin	Kyungshin Corp.	Younghwa_Tech	YounghwaTech Co., Ltd.

in Appendix A. In the applicant abbreviation, “I_” and “U_” are prefixes referring to the research institution and university, respectively.

4.2. Embedding Collaboration Partners

Co-applicant embedding was conducted using Node2Vec in the experiment.

Table 2. List of hyperparameters used in the experiment.

Description	Hyperparameter
Dimensionality of node, D	16
Return parameter, p	0.5
In–out parameter, q	2
Length of walk, l	5
Number of walks per node	30

summarizes the detailed hyperparameter values. We converted applicants into a 16-dimension vector through node embedding. Furthermore, 0.5 and 2 were used as the return parameter and in–out parameter in Node2Vec. The length of the walk and the number of walks per node, which were extracted from the applicant, were 5 and 30, respectively.

Figure 4 shows the reduced result of the 16-dimension applicant embedded into two dimensions [60]. The shape of the node represents the type of applicant. A circle represents an enterprise, a triangle represents a research institution, and a square represents a university. In the figure, the shade of the node’s background refers to the decision boundary of the k-nearest neighbor algorithm, which is classified according to the applicant’s nationality. The experimental results show that the nationality of the applicant and the neighborhood relationship of the applicant is well maintained (See Figure 3).

The proposed method explores potential R&D collaboration partners through MMR from embedded co-applicants. However, we needed to quantitatively verify whether the proposed method satisfactorily maintains the information of co-applicants. Thus, link prediction was conducted to predict actual co-applicants. The applicants that are embedded in 16 dimensions are connected through 3160 links. Of these, only 77 links exist, while the remaining 3083 do not. In the experiment, the performance of link prediction according to link embedding $g$ and predictive model $h$ was compared. The link prediction performance was measured using accuracy, precision, recall, F1-score, geometric mean (GM), and area under the receiver operating characteristic curve (AUC). The predictive models used in the experiment are the adaptive boosting (AB), decision tree (DT), k-nearest neighbor (KNN), and random forest (RF) models.

The process to select optimal link embedding and the predictive model to discover potential R&D collaboration partners is as follows: First, the performance according to the hyperparameters of Node2Vec was compared. In Appendix A, Figure A1 shows the comparison results of GM and AUC according to the hyperparameter of Node2Vec. The link that connects co-applicants is highly imbalanced. Therefore, we selected the optimal combination as the case where GM and AUC were 0.6 and 0.8 or larger. As a result, we determined that the performance was the best when the return parameter and in–out parameter were 0.5 and 2, and $g$ and $h$ were weighted-L2 and KNN, respectively.

Table 3. Performance of link prediction.

Operator	Classifier	Accuracy	Precision	Recall	F1-Score	GM	AUC
Average	DT	0.976	0.500	0.217	0.303	0.882	0.606
	KNN	0.987	0.867	0.565	0.684	0.659	0.782
	RF	0.976	0.500	0.043	0.080	0.977	0.521
	AB	0.978	0.583	0.304	0.400	0.832	0.649
Hadamard	DT	0.974	0.400	0.174	0.242	0.906	0.584
	KNN	0.981	0.727	0.348	0.471	0.806	0.672
	RF	0.977	0.667	0.087	0.154	0.955	0.543
	AB	0.985	0.846	0.478	0.611	0.722	0.738
Weighted-L1	DT	0.972	0.389	0.304	0.341	0.829	0.646
	KNN	0.983	0.706	0.522	0.600	0.690	0.758
	RF	0.979	1.000	0.130	0.231	0.933	0.565
	AB	0.977	0.538	0.304	0.389	0.831	0.649
Weighted-L2	DT	0.969	0.364	0.348	0.356	0.801	0.666
	KNN	0.984	0.700	0.609	0.651	0.624	0.801
	RF	0.981	1.000	0.217	0.357	0.885	0.609
	AB	0.977	0.538	0.304	0.389	0.831	0.649

presents the performance of link prediction measured using optimal hyperparameters. The comparison results exhibited that the performances of DT, RF, and AB were comparable regardless of the operators. Therefore, we decided the final operator and predictive model based on GM and AUC. This is because GM and AUC help measure robust performance on data with imbalanced labels. Therefore, we chose wideghted-L2-based KNN as the final model, with GM and AUC greater than 0.6 and 0.8, respectively.

4.3. Embedding Collaboration Partners

Now, we know that the information of the co-applicant in the proposed method is well-preserved. Therefore, we extracted co-applicant candidates from the optimal model to discover potential R&D collaboration partners. The co-applicant candidate is a sample that (i) has no link to connect the actual co-applicant but (ii) is likely to be connected by the optimal model.

Table 4. List of candidate collaboration partners discovered by the method.

No.	Co-Applicant Candidate	No.	Co-Applicant Candidate
1	Aisin, Daihatsu	9	I_KIST, Inosuki
2	Aisin, Fujitsu	10	Kia, Lotte
3	Axia, Lotte	11	Lotte, Infineon
4	Daihatsu, Fujitsu	12	Soken, Fujitsu
5	Egtronics, Unichem	13	Soken, Sumitomo_Wiring
6	Fujitsu, Sumitomo_Wiring	14	U_Korea, Inosuki
7	I_GHI, I_KECA	15	U_Sungkyunkwan, U_Transport
8	I_KICT, I_KIST	16	Unichem, Gate

presents the list of candidate co-applicants extracted from the optimal model. The extracted 16 co-applicants are expected to have potential collaboration. “Fujitsu”, which emerged most frequently in the list, develops automotive network solutions based on fifth-generation mobile communications, so it is expected that the company will develop autonomous driving technology continuously [61].

Figure 5 shows the visualization of the candidate co-applicant graph. It is expected that several business opportunities will be discovered centered around “Fujitsu” in Japan. South Korea is projected to have continuous collaboration around institutions or universities.

In the experiment, potential R&D collaboration partners among candidate co-applicants were explored using the proposed method. The selection diversity $\gamma$, which was the hyperparameter of MMR, was set to 0.8 in the experiment. In addition, a link that has the maximum of Equation (9) was searched to discover the highest possible co-applicant.

Table 5. List of potential R&D collaboration partners.

Candidate Collaboration Partners	Potential Collaboration Partners
Aisin, Daihatsu	Renault, Toshiba
Aisin, Fujitsu	Toyota, Honda
Axia, Lotte	I_KECA, Audi
Daihatsu, Fujitsu	Daihatsu, Chrysler
Egtronics, Unichem	U_Yeongnam, U_Huazhong
Fujitsu, Sumitomo_Wiring	Tamagawa, Huawei
I_GHI, I_KECA	Jaeshin, U_Yeongnam
I_KICT, I_KIST	Egtronics, Chrysler
I_KIST, Inosuki	PanKorea, BMW
Kia, Lotte	PanKorea, U_Huazhong
Lotte, Infineon	U_Korea, Nissan
Soken, Fujitsu	Soken, Tamagawa
Soken, Sumitomo_Wiring	Toshiba, LG
U_Korea, Inosuki	Egtronics, Suzuki
U_Sungkyunkwan, U_Transport	YounghwaTech, Aisin
Unichem, Gate	Unichem, SK

presents the list of potential co-applicants derived through the proposed method. Most co-applicants were derived to have potential collaboration with applicants that had the same nationality.

Figure 6 shows the graph of potential co-applicants derived through the proposed method. The analysis results revealed that most co-applicants were discovered in the same nation. In Japan, possible sustainable collaboration is expected around “Fujitsu” and “Aisin” [62,63]. Furthermore, in South Korea, business opportunities were potentially high with institutions such as “I_KIST” and “U_Korea” in addition to promising electric vehicle battery-related enterprises such as “Egtronics” [64,65]. Furthermore, “Chrysler” is expected to collaborate with enterprises and institutions in various nations.

5. Conclusions

5.1. Discussion and Implications

In the rapidly evolving global market, enterprises are reinforcing proactive human, financial, and technological collaboration with research institutions and universities for their survival. In particular, the collaboration of organizations has reduced the time and cost that are consumed in R&D. Thus, organizations continue to maintain and manage their collaboration systems. However, it has been pointed out that this practice may have a negative impact on the development of organizations [66].

At present, organizations including enterprises have to answer the question of “where, how, and to whom to spread their knowledge” beyond socialization, externalization, combination, and internalization of their knowledge. Previous studies have answered the question through patent maps, functional-based frameworks, geographical distance-based approaches, and knowledge graphs. However, they have the following limitations: (i) they did not properly reflect the features of organizations [25,26,27,28,29], (ii) it was difficult to discover potential R&D collaboration partners using them [30,31,32,33], (iii) they needed to be adjusted according to the rapidly changing global pandemic environment [34,35,36,37], and (iv) they were dependent on specific topic or technology classification codes [43,44].

In response, we proposed a method that can discover potential R&D collaboration partners for the sustainable development of organizations. This paper represents applicants who own the patent and their relationships in a knowledge graph. Applicants, who are nodes in a knowledge graph, are embedded in a vector space through node embedding. Finally, potential R&D collaboration partners are discovered through the MMR algorithm.

We ask the following research questions:

• Can the information about co-applicants of patents clearly explain business opportunities?
• Can potential R&D collaboration partners be discovered from co-applicants?

In order to answer the research questions, we attempted to discover potential R&D collaboration partners from 873 patents. Potential R&D collaboration partners were discovered by extracting organizations that were similar to candidate co-applicants.

5.2. Limitations and Further Research

The present study has several limitations. First, the proposed method did not consider the timing of collaborative R&D. This is because collaborative R&D is conducted at the right time and right place. Second, our method did not consider the relationship with organizations when discovering potential R&D collaboration partners. The collaboration with organizations is conducted considering the relationship between nations or competitors. Consequently, advanced methodologies should consider this to discover potential R&D collaboration partners. Third, we did not present a method to determine hyperparameters such as the dimensionality of the node. Finally, the proposed methodology needs to be compared with other approaches. However, to the best of our knowledge, our study is the first work to explore R&D partners based on collaboration graphs. In the future, there is a need to compare the proposed method with a model that improves several limitations.

The plans for future research are as follows:

• Collaborative R&D will be conducted according to the plan of the human and financial scale. Thus, it is necessary to discover business opportunities that can maximize the value of a limited resource in the future.
• The potential of the discovered business opportunities will be determined by the management plan and strategy of the organization. Accordingly, advanced methodologies should consider the business feasibility of the discovered opportunities.
• Methodologies will be proposed that can be applied to various technological fields concurrently. This is because multidisciplinary business opportunities are a global trend.

This paper proposes a methodology based on a knowledge graph to discover potential R&D collaboration partners. Although our methodology has several limitations, it can answer the questions we raised. Thus, we expect organizations to discover business opportunities that can result in sustainable market value through the proposed method.