Prediction of University Patent Transfer Cycle Based on Random Survival Forest

Round 1

Reviewer 1 Report

Dear Authors,

1. The article indicates the current, crucial and original scientific problem.

2. The research gap and novelty of contribution to the science development is clearly stated. 

3. This coherent manuscript presents very strong methodology part.

4. Interesting results are properly presented.

5. I will also consider to add/separate the discussion part of the article. 

Please see my detailed comments below:

1. What is the main question addressed by the research? 

The authors describe the main research problem and the aim of the manuscript as follows: 

“Clarifying the fluctuation of patent transfer probability over time can help universities actively seek potential patent grantors in the market at an appropriate time, and then realize patent industrialization”. 

“This paper aims to incorporate the factors affecting the patent transfer cycle into the survival analysis model based on the existing research, furthermore, predicting and evaluating the individual university patent transfer cycle with the model.” 

2. Do you consider the topic original or relevant in the field? Does it address a specific gap in the field? 

Yes, I consider the topic original and relevant in the field. The authors concentrated on the literature emphasized the relevance of the topic in the introduction, namely: “Promoting patent industrialization is a key link to facilitate the close integration of science and technology with economy. As a main part of the national innovation system, Chinese universities produce generous patents every year, but the industrialization rate is low and the transfer cycle is long, resulting in a large number of scientific research resources cannot be fully utilized [1]. As the timeliness of patent is strong, if effective technology transfer cannot be carried out within a specific time span, the patent will become invalid, which would cause a huge waste of scientific and technological resources for universities, enterprises and the country [2] For one thing, as the exclusive right of developed  technology, the technology preemption of patents is essential for the development of enterprises[3]. Clarifying the time from patent’s application to authorization can prevent other enterprises from using their patented inventions for commercial purposes, thus reducing competition among enterprises [4] . For another, the cost of maintaining invention patents is relatively expensive for universities, so it is necessary for universities to sell patents to recover the investment cost of the inventions [5]. Therefore, both universities and enterprises have strong motivations to realize the patent transformation”. “Based on the above reasons, establishing a patent transfer cycle prediction model and analyzing its influencing factors are of great theoretical and practical significance for both universities and enterprises”. 

3. What does it add to the subject area compared with other published material? 

The methodology used in this paper is an original, novel approach by the author/s. 

Moreover, according to the authors “the research above has generally confirmed the impact of patent characteristics and inventor team characteristics on the university patent transfer cycle, and mainly focused on the factors affecting the university patent transfer cycle and the patent convertibility prediction, less considered the dynamic effect of patent transfer cycle over time. Nevertheless, when specifically analyzing the issue of university patent transfer, in addition to concern about whether the patent has transferred, the patent transfer cycle and opportunity are also needed to be involved. 

Compared with other published  material the article “incorporate the factors affecting the patent transfer cycle into the survival analysis model based on the existing research, furthermore, predicting and evaluating the individual university patent transfer cycle with the model.” 

4. What specific improvements should the authors consider regarding the methodology? What further controls should be considered? 

It is not clear enough what are the stages of this research.  Do you analyzed 79393 invention patents, or this one single study that was presented in part 4.3? Please, give more detailed description of the patent with application number CN102675607B. 

4.3. Case analysis part presents Figure 4 that shows the overall survival function of the OOB patent data in train set? It is not clear enough. What does the following abbreviation, namely: “the OOB patent data” mean? 

In part 4.3 the Nelson-Aalen estimator should be defined. 

“the patent with application number CN102675607B applied by Nanjing 324 University is randomly selected to analyze” remark: if this is a single case study this is not a random selection. The case was intentionally selected and based on a single case study you cannot conclude on the whole population.   

The theoretical and empirical part of this paper should be more consistent. 

5. Are the conclusions consistent with the evidence and arguments presented
and do they address the main question posed? 

Generally yes with the following comment: Based on a single case study you cannot conclude on the whole population.   

6. Are the references appropriate? 

Yes. 

7. Please include any additional comments on the tables and figures. 

Figure 4. Survival function of patents  

How many patents are presented in fig. 4? This figure was presented in the case study part of the article. It is not clear.  

Additional remark: According to IMRaD format a paper is structured by four main sections: Introduction, Methods, Results, and Discussion.  

The discussion part of the article is missing. I suggest to add the discussion part. 

To sum it up. I still consider this article as the original and valuable study and recommend its publication after minor changes suggested above. 

Good luck!

Author Response

Response to Reviewer 1 Comments

Point 1: It is not clear enough what are the stages of this research.  Do you analyzed 79393 invention patents, or this one single study that was presented in part 4.3? 

Response 1: Thanks to the reviewer for the comments, the revision of point 1 is as follows:

The 79393 patents referred to the patents which were applied by C9 League from 2002-2020 and can be retrieved from the INCOPAT patent platform,these patent data were used to establish different survival models after data preprocessing. According to the another suggestion (point 6), the picture“survival function”(which was fig.4 in the part 4.3 and now is fig.2 in part 4.1 in the revised manuscript) , shows the overall survival function of the OOB (out-of-bag) patent data, which were generated in the bootstrap process after data preprocessing and train test sets dividing. This part of data has not participated in the establishment of decision tree in the random survival forest model, which accounts for about 37% of the train set, including a total of 16,982 patent data.  And the single cases (fig.5 and fig.6) which were presented in part 4.3 are the predicted survival function and cumulative risk function of individual patent.

Point 2: Please, give more detailed description of the patent with application number CN102675607B.

Response 2: Thanks to the reviewer for their comments, the revision of point 2 is as follows:

Based on the point 5, the patent sample CN102675607B has been replaced to the other two patents that are more representative, and the detailed descriptions about the individual patents in case analysis have been added, which includes a brief introduction of the patent and which company the patent transferred to.

Point 3: 4.3. Case analysis part presents Figure 4 that shows the overall survival function of the OOB patent data in train set? It is not clear enough. What does the following abbreviation, namely: “the OOB patent data” mean?   

Response 3: Thanks to the reviewer for their comments, the revision of point 3 is as follows:

The Figure 4 (which is Figure 2 in the revised manuscript) shows the overall survival function of the OOB (out-of-bag) patent data, which were generated in the bootstrap process after data preprocessing and train test sets dividing. This part of data has not participated in the establishment of decision tree in the random survival forest model, which accounts for about 37% of the train set, including a total of 16,982 patent data. OOB data can be regarded as a validation datasets separated from train datasets, which can be used to judge the variable importance, measure the generalization ability and evaluate the performance of the model.

Point 4: In part 4.3 the Nelson-Aalen estimator should be defined.

Response 4: Thanks to the reviewer for their comments, the revision of point 4 is as follows:

The Nelson-Aalen estimator is the cumulative incidence of patent transformation calculated by the cumulative risk function, which merely includes the data of patent that has been transferred successfully in the observation period. It has been explained comprehensively in the revised manuscript.

Point 5:“the patent with application number CN102675607B applied by Nanjing 324 University is randomly selected to analyze” remark: if this is a single case study this is not a random selection. The case was intentionally selected and based on a single case study you cannot conclude on the whole population.  The theoretical and empirical part of this paper should be more consistent.  

Response 5: Thanks to the reviewer for their comments, the revision of point 5 is as follows:

This problem has been tackled in the revised manuscript. To illustrate the predictability of random survival forests in the issue of patent transfer cycle, two representative patent samples applied by Tsinghua University and Zhejiang University, which has a larger number of patent applications among C9 League, with patents application quantity both exceeding 20% of the total sample were selected for prediction and analysis. The patent CN105549647B which discloses a mobile piglet traction local culture environment monitoring system, was applied on December 15, 2015 by Zhejiang University and transferred to Hefei Shenmu Information Technology Co., Ltd. on December 9, 2021, with a lifetime of about 72 months. Additionally, the patent CN100386728C which is an planned type of medical treatment instrument, was applied on March 24, 2006 by Tsinghua University and transferred to Beijing Pinchi Medical Equipment Co., Ltd. on October 26, 2016, with a survival time of about 127 months. These two case study were predicted and analyzed by random survival forest model according to the empirical results in part 4, and the experiments in part 4 which are based on the theoretical part in part 2.

Point 6: How many patents are presented in fig.4? This figure was presented in the case study part of the article. It is not clear. 

Response 6: Thanks to the reviewer for their comments, the revision of point 6 is as follows:

The picture “survival function” (which was fig.4 in the old version and is fig.2 in the revised manuscript) has been moved from 4.3 case study to 4.1 Importance of variables when considering its implication and referencing other related representative literature [1-2], it shows the overall survival function of the OOB (out-of-bag) patent data, which were generated in the bootstrap process after data preprocessing and train test sets splitting. This part of data has not participated in the establishment of decision tree in the random survival forest model, which accounts for about 37% of the train set, including a total of 16,982 patent data.

Point 7: The discussion part of the article is missing. I suggest to add the discussion part. 

Response 7: Thanks to the reviewer for their comments, the revision of point 7 is as follows:

The discussion part has been added to the revised manuscript, the concrete content is: “Clarifying the fluctuation of patent transfer probability over time can lead universities to seek potential patent grantees in the market at appropriate times, and then realize patent industrialization effectively, facilitating the close integration of science and technology with economy. Considering that the existing relevant research was mainly focused on studying the factors affecting patent transfer cycle, less consideration has been given to the change of the probability of patent transfer over time, also, the dynamic period of patent transfer has not been predicted. Based on this background, as time-to-event outcomes can provide more information than simply whether or not an event occurred. In order to deal with these outcomes, as well as censored observations where the event was not observed during follow-up, survival analysis methods are adopted in this research.

 In an effort to select the model with the optimal prediction performance in university patent transfer cycle, Cox proportional risk model, Cox model based on Lasso penalty, random forest model and random survival forest model are compared together in predicting performance when applying the sample data of university patent transfer cycle. And it shows that the prediction performance of the random survival forest is optimal among the four survival models, which can provide suggestions for both universities and enterprises on identifying the opportunity of patent transformation, thereby shortening the patent transfer cycle and improving the patent transfer efficiency.

However, this research also has some limitations: Firstly, patents in different technical fields may have different patterns in their transfer cycles, all kinds of university invention patent data are used to modeling as a whole in this paper, which can ensure the robustness of the model to a certain extent, but the prediction accuracy of an individual patent might be reduced. Secondly,  this study simply takes the patent applied by the C9 League for the model establishment as research samples, whereas the types and levels of university patents are varied. Therefore, considering the finiteness of the sample types in this research, the scope of model application may have some limitations. In view of these deficiencies, further research is supposed to be performed in the future work.”

References

• Miao, F., Cai, Y., Zhang, Y., Li, Y., Zhang, Y.,... Bursac, Z. Risk Prediction of One-Year Mortality in Patients with Cardiac Arrhythmias Using Random Survival Forest. Computational and Mathematical Methods in Medicine, 2015,
• Tse, G., Lee, S., Zhou, J., Liu, T., Wong, I. C. K., Mak, C.,... Wong, W. T. Territory-Wide Chinese Cohort of Congenital Long QT Syndrome: Random Survival Forest and Cox Analyses. Frontiers in Cardiovascular Medicine, 2021.

Author Response File: Author Response.docx

Reviewer 2 Report

I liked the research idea, and the methodology used to reach its conclusions is sound. I appreciated in the paper the fact that the authors uncovered an interesting are of university patent transfer cycle and predict its effectiveness. The paper's methodological apparatus is appropriate to the task at hand, as well as well supported by academic literature. However, I believe that the main effects uncovered here are somewhat spurious. Also... the paper demonstrated that, among the four models tested, random survival forest is the most accurate. However, I would like to see a sound reason why the authors picked these 4 models in the first place, because many other models could fit their data. Also, the paper needs a serious revision of English phrasing, because in many sections it is cumbersome and the ideas presented are difficult to follow. Overall, I believe this is a good paper and it has a sound structure and I support its publishing in your journal, after some minor revisions.

Author Response

Response to Reviewer 2 Comments

Point 1: I would like to see a sound reason why the authors picked these 4 models in the first place, because many other models could fit their data.

Response 1: Thanks to the reviewer for the comments, the response of point 1 is as follows:

Firstly, it is determined by the research background and model characteristics. Most existing relevant researches were mainly focused on studying the factors affecting patent transfer cycle, less attention has been given to the change of the probability of patent transfer over time, also, the dynamic period of patent transformation has not been predicted. Based on this background, as time-to-event outcomes are can offer more information than simply whether or not an event occurred, to handle these outcomes, as well as censored observations where the event was not observed during follow-up, survival analysis methods should be used instead of static models such as the logistic regression, support vector machine and the decision tree.

Afterwards, when selecting frequently-used survival models which are referred in other representative literature [1-5], it is considered that the cox proportional risk model, cox proportional risk model with penalty, random forest model based on machine learning, random survival forest model are popular in the survival analysis and their performance are generally compared together. In addition, since the previous studies have shown that Lasso method can obtain better prediction results than the forward and the backward stepwise regression methods [6], Lasso method is adopted to establish a proportional risk model with penalty in this paper. Therefore, Cox proportional risk model, Cox proportional risk based on Lasso penalty, random forest model and random survival forest are selected to establish the university patent transfer cycle model, and their predicting accuracy are compared, in an effort to select the model with the optimal prediction performance.The reasons above have been explained comprehensively in the revised manuscript.

Point 2: However, I believe that the main effects uncovered here are somewhat spurious... Also, the paper needs a serious revision of English phrasing, because in many sections it is cumbersome and the ideas presented are difficult to follow.

Response 2: Thanks to the reviewer for the comments, the revision of point 2 is as follows:

The English phrasing has been improved in the revised manuscript to make the paper easier to understand. In an effort to make the article easier to follow and more reasonable, some revisions have been made to the manuscript as well: (1) The discussion part has been added to make the structure of the article more complete. (2) In order to illustrate the predictability of random survival forests in the issue of patent transfer cycle, two representative patent samples were selected for prediction and analysis in the case study part instead the one patent randomly chosen in the old version. Increasing the number and representativeness of samples makes the model more convincing to some extent.

References

• Lu, W., Pan, X., Dai, S. Q., Fu, D. L., Hwang, M., Zhu, Y. S.,... Ding, K. F. Identifying Stage II Colorectal Cancer Recurrence Associated Genes by Microarray Meta-Analysis and Building Predictive Models with Machine Learning Algorithms. JournalOf Oncology, 2021.
• Baralou, V., Kalpourtzi, N., & Touloumi, G. Individual risk prediction: Comparing random forests with Cox proportional-hazards model by a simulation study. Biometrical  Journal. 2022, 1-13.
• Chang, E., Joel, M., Chang, H. Y., Du, J., Khanna, O., Omuro, A.,...Aneja, S.Comparison of radiomic feature aggregation methods for patients with multiple tumors. Scientific Reports, 2021, 11, 9758. 
• Jeffery, A. D., Dietrich, M. S., Fabbri, D., Kennedy, B., Novak, L. L., Coco, J.,... Mion,L.C..ADVANCING IN-HOSPITAL CLINICAL DETERIORATION PREDICTION MODELS.American Journal of Critical Care. 2019, 27, 
• Qi, J, Yan, JW, Idrees, M ,Almutairi, SM ,Rasheed, RA ,Hussein, UA,...Li, C.Five EMT-Related Gene Signatures Predict Acute Myeloid Leukemia Patient Outcome. Disease Markers, 2022.
• Fang, K., N, Zhang G., J, & Zhang H., Y.. Individual Credit Risk Prediction Method: Application of a Lasso-logistic Model. The Journal of Quantitative & Technical Economics. 2014, 31, 125-136.

Author Response File: Author Response.docx