Framework for Measuring Process Innovation Performance at Indonesian State-Owned Companies

Round 1

Reviewer 1 Report

The article is a valuable study of activities in Indonesian industrial companies, using the delphi method to outline the framework for the innovation process. However, the article needs to be improved. After Introduction, select Literature Review from the extensive Material and methods section. The proportions of the selection of experts seem to be shaken, because there is only one professor and 3 representatives of the Ministry, it is worth better justifying such a selection of experts. it is probably legitimate to grasp the practical implications. Failure to assess the reliability of the interview questionnaire - what is the Cronbach's alpha value for this questionnaire? Finally, the article has many editorial flaws, some sources do not open (line 266, 271, 333, 378, 447). Figure 1 is duplicated (pages 7 and 16). The source of the drawing is also missing (probably own elaboration). The legibility of the text is hindered by numerous abbreviations of names. It seems that these errors will be easily corrected, which will add value to the article.

Author Response

Point 1: After Introduction, select Literature Review from the extensive Material and methods section.

Response 1: Thank you, we really appreciate your comments; very helpful and have been revised as needed. Revision in manuscript: line 114.

Point 2: The proportions of the selection of experts seem to be shaken, because there is only one professor and 3 representatives of the Ministry, it is worth better justifying such a selection of experts. it is probably legitimate to grasp the practical implications.

Response 2: Thank you for the beneficial riposte. According to the Delphi methods, the number of panelists recommended is not circumscribed [107] but must not be less than 8 participants [105]. Furthermore, the average of the number of panelists is between 15-20 [104]. If the sample size of a Delphi study is too small, these subjects may not be considered as having provided a representative pooling of judgments regarding the target issue. If the sample size is too large, the drawbacks inherent within the Delphi technique such as potentially low response rates and the obligation of large blocks of time by the respondents and the researcher(s) can be the result.

Point 3: Failure to assess the reliability of the interview questionnaire - what is the Cronbach's alpha value for this questionnaire?

Response 3: Thank you, we really appreciate your comments; very helpful and have been revised as needed. Revision in manuscript: Table 3—Table 12.

Point 4: Finally, the article has many editorial flaws, some sources do not open (line 266, 271, 333, 378, 447). Figure 1 is duplicated (pages 7 and 16).

Response 4: Thank you for the enlightened evaluation, and we have revised it. Moreover, Figure 1 on page 7 is a framework illustration for determining key indicators utilized to explore success factors. Next, Figure 2 on page 16 to indicate the success factors for each of the main indicators.

 

Reviewer 2 Report

I found this paper interesting.

But the text and structure of this manuscript are not scientifically strong. Written in simple sentences. Therefore, is lack of strengthening the scientific text in some places of this study.
I would like to recommend to construct the main goal of this study on the Introduction part. The research question is not sufficient. However, it would be worthwhile to reveal more about the rationale for the scientific problem. The platform for the scientific problem construct is not disclosed.

Research Instrumentation has some inaccuracies: 

first, please construct the criteria, and after then main indicators; 

second, please indicate how each criteria will be checked_ which method can help to describe this criteria; 

third, please justify the Sampling type and argumentation of sampling sample; 

fours, what is the scientific result of your study? The sounding to the others studies/other researchers/.

I would recommend strengthening the scope of the discussion by conceptualizing the essential aspects of the topic.

Author Response

Point 1:  please construct the criteria, and after then main indicators;

Response 1: We construct the main indicators based on the model produced by Detecon Consulting [15] by adding other indicators, i.e., diffusion, push factor and pull factor (explained from line 313-320). Then we give the panelists open-ended questions to explore the success factors for each of the key indicators. This step is the prefix of the Delphi method in accordance with our previous publication [112].

Point 2:  please indicate how each criteria will be checked which method can help to describe this criteria.

Response 2: We added Cronbach's Alpha value of 70% for minimal consensus achievement. Furthermore, we use the upper limit of the Q3 quartile for the average and the lower limit of Q1 for the standard deviation [113]. Revision in manuscript: Line 401-409 (first paragraph on section Result).

Point 3:  please justify the Sampling type and argumentation of sampling sample.

Response 3: According to the Delphi methods, the number of panelists recommended is not circumscribed [107] but must not be less than 8 participants [105]. Furthermore, the average of the number of panellists is between 15-20 [104]. If the sample size of a Delphi study is too small, these subjects may not be considered as having provided a representative pooling of judgments regarding the target issue. If the sample size is too large, the drawbacks inherent within the Delphi technique such as potentially low response rates and the obligation of large blocks of time by the respondents and the researcher(s) can be the result. The 12 panelists (75%) are from practitioners who have been actively involved in process innovation activities in SOEs, expected to provide practical experience. Three other panelists (from ministries and academics) were assigned as keynote speaker who validated panelists experience with the latest literature and regulations.

Point 4:  what is the scientific result of your study? The sounding to the others studies/other researchers/. I would recommend strengthening the scope of the discussion by conceptualizing the essential aspects of the topic.

Response 4: In the discussion session, we have compiled the results obtained from this research, namely 28 success factors with 33 other references/previous research. The details are as follows:

Indicator	Success Factors	Reference
Input	Organization Culture; Organization Support & Commitment; Problem Understanding	[73] [86] [88] [116] [117] [127]
Process	Innovation Strategy; Personal KPI; Benchmarking; Innovation Impact Assessment; Pilot Project; Supervision; Organization Support & Commitment	[126] [131] [132] [130] [133] [90] [134]
Output & Outcome	Cost Efficiency; Improving Business Process; Improving Competitiveness; Process Effectiveness	[9] [91] [92] [93] [95] [140]
Diffusion	Leadership; Management Commitment; Employee Involvement	[123] [124] [125]
Culture	Creative and Innovatiove Mindset; Innovation Mindset	[120]
Strategy	Continuous Improvement; Market-driven	[121] [122] [129]
Pull Factors	Customer Demand; Competitive Advantage; Excellent Performance; Sustainability in Competition	[85] [119]
Push Factors	Cost Efficiency; Productivity Improvement; Increasing Profit	[51] [95]

 

Reviewer 3 Report

Dear Authors,

While your paper has the potential contribution, I would like to suggest several improvements as follows:

1. I believe that the issue of open innovation within the SOEs is critical, however, you provide a poor transition of drafting as evident in line 30. You discuss the potential of open innovation and directly move to the conversation of the Indonesia Minister. It is highly recommended that you settle one issues/paragraph to focus the readability.
2. I believe that the paragraph 1 in the introduction would mainly introduce the issue of open innovation.
3. I also suggest the paragraph 2nd to discuss the reason why Indonesia's economy is also propelled by the SOEs, in contrary to the majority of developed countries. Some theoretical discussions in the context of SOE in the economics and current practice would be nice.
4. The third paragraph would introduce some Indonesian SOEs before pointing to their specific innovation. Each first sentence must briefly present the main idea of the paragraph. 
5. The third paragraph can also be used to emphasize why your paper is important, fresh, novel, or if possible ground breaking.
6. International readers mostly do not recognize Indonesian SOEs, so please add an information regarding the business field of the stated SOEs in a bracket.
7. MDPI employs IMRAD in the paper presentation. Thus, you can actually create a sub-section under the introduction section explaining the innovation, innovation classification, and Process Innovation. 
8. The materials and method has to solely focus on the research method aspects. Thus, you may put the subsection of framework of process innovation, and so on here. 
9. The higher agenda of research method is firstly to explain why you select one research approach, design and method over the other possiblities. Please explain why you use the detection? You also need to explain why you employ Delphi, instead of commonly understood confirmatory factor analysis, via rotation or other potential options.
10. the font size and the space needs to be adjusted according to the author guidelines in the template.
11. In the PDF version, some references are not found, see for example line 447. 
12. what is KPI in line 533 stands for?
13. The discussions must provide a concrete statement regarding why your research findings must be applied in Indonesia SOEs. This step would ensure the specific implication of your study.
14. Please add reference from JOITMC that may be related to your article.

Best

Author Response

Point 1: I believe that the issue of open innovation within the SOEs is critical, however, you provide a poor transition of drafting as evident in line 30. You discuss the potential of open innovation and directly move to the conversation of the Indonesia Minister. It is highly recommended that you settle one issues/paragraph to focus the readability.

Response 1: Thank you, we really appreciate your comments; very helpful and have been revised as needed. We have already changed the composition of paragraphs 1 and 2 to accommodate the transition process between innovation discussion and open innovation.

Point 2: I believe that the paragraph 1 in the introduction would mainly introduce the issue of open innovation.

Response 2: Thank you for the notice. We have rearranged the structure, therefore the open innovation is in paragraph 2.

Point 3: I also suggest the paragraph 2nd to discuss the reason why Indonesia's economy is also propelled by the SOEs, in contrary to the majority of developed countries. Some theoretical discussions in the context of SOE in the economics and current practice would be nice.

Response 3: Thank you for the suggestions, we really appreciate your comments; very helpful and have been revised as needed.

Point 4: The third paragraph would introduce some Indonesian SOEs before pointing to their specific innovation. Each first sentence must briefly present the main idea of the paragraph.

Response 4: Thank you for the suggestions. We have revised this.

Point 5: The third paragraph can also be used to emphasize why your paper is important, fresh, novel, or if possible ground breaking.

Response 5: Thank you for the suggestions. We have added these sentences in the last third paragraph in Introduction section: This description of success in innovating can be one of the knowledge transfer materials between SOE companies, therefore other SOEs can do the same. This is an area of study in the research that we will accomplish.

Point 6: International readers mostly do not recognize Indonesian SOEs, so please add an information regarding the business field of the stated SOEs in a bracket.

Response 6: Thank you for the suggestions. We have already provided a little explanation about several SOEs in paragraph 3.

Point 7: MDPI employs IMRAD in the paper presentation. Thus, you can actually create a sub-section under the introduction section explaining the innovation, innovation classification, and Process Innovation.

Response 7: Thank you very much. We accept your advice and also adopt papers published in the journal JOItmC.

Point 8: The materials and method have to solely focus on the research method aspects. Thus, you may put the subsection of framework of process innovation, and so on here.

Reviewer 8: Thank you for the suggestions. Thank you very much. We accept your advice and also adopt papers published in the journal JOItmC.

Point 9: The higher agenda of research method is firstly to explain why you select one research approach, design and method over the other possiblities. Please explain why you use the detection? You also need to explain why you employ Delphi, instead of commonly understood confirmatory factor analysis, via rotation or other potential options. 

Response 9: Thank you for the suggestion. We have supplemented the manuscript about the reason why we use the Deplhi Method. This is because Delphi methods that do not require too much data that are very suitable for the data collection process during the COVID pandemic. As we all know, the pandemic period requires us to reduce interaction with others, especially in groups. With these difficulties, the data to be obtained will be very limited,therefore an analysis cannot be performed with other statistical methods such as confirmatory factor analysis.

Point 10: the font size and the space needs to be adjusted according to the author guidelines in the template.

Response 10: Thank you for the suggestions. We have revised it.

Point 11: In the PDF version, some references are not found, see for example line 447.

Response 11: Thank you. We have revised it.

Point 12: what is KPI in line 533 stands for?.

Response 12: Thank you for the evaluation. We have written the acronym of KPI for Key Performance Indicator in line 432.

Point 13: The discussions must provide a concrete statement regarding why your research findings must be applied in Indonesia SOEs. This step would ensure the specific implication of your study

Response 13: Thank you for the comment. We have already described in Limitations and Further Research. “In future research, we plan to use the indicators of the success of process innovation to assess the performance of process innovation in several state-owned companies. In conclusion, SOEs can evaluate the implementation of innovation in a company to ensure the sustainability of its innovation”.

Point 14: Please add reference from JOITMC that may be related to your article.

Response 14: Thank you for the suggestion. We have cited 17 papers of JOITMC.

Reviewer 4 Report

Dear authors,

congratulations on an excellent manuscript. Examination of the success factors of process innovation using the given methodological apparatus in the context of innovation activities of state enterprises (SOE) can be considered as very beneficial for the ongoing scientific discussion on innovation performance of enterprises in knowledge-intensive industries. The manuscript is very well written; the level of the English language is sufficient. I appreciate the quality of the elaboration of the theoretical background, a depth of the Delphi method descriptions and a really very precise discussion and conclusion, including relevant suggestions for further research on topic. I would suggest just a minor changes and corrections:

(of a technical nature:)

• line 24-25 - same sentence with different source as in case of 39-40.
• reference not found – there is probably a problem with conversion to pdf in case of lines 266, 271, 333, 378, 447
• missing letter in 472
• please re-read the manuscript and unify shortcuts, as I am not sure, whether state owned enterprises are SOE, or SEO – e.g. line 235, 236.
• line 436 – in the title of the table, there should be „push“ instead of „pull“ factor

(suggestions:)

In line 440 + 441 you state: „ researchers asked the panel members to convey their experiences and knowledge related to process innovations that had been carried out in their respective companies “. I would suggest to briefly describe examples of key drivers and attitudes in case of concrete SOEs, as it could be a valuable additional information to quantitative results.

I find the titles and content of material and methods, resp. methodology chapters a bit confusing. It is not very common in scientific articles to introduce basic terms and their definitions in the chapter "Material and methods". As we do not have a goal, data and methodology in this obligatory chapter, it is necessary to modify the structuring of the manuscript.

Sub-chapters 2.1., 2.2., 2.3. and 2.4., I understand rather as a theoretical background, so I recommend to change the name of chapter 2 to „Theoretical background“.

In other ways it was a fruitful reading and I am looking towards citing your results.

Author Response

Point 1: line 24-25 - same sentence with different source as in case of 39-40.

Response 1: Thank you for the evaluation. We have made changes and improvements to paragraphs 1 and 2 due to suggestions from other reviewers and also revised the references we used. We realized there were some unintentional errors when providing reference sources.

Point 2: reference not found – there is probably a problem with conversion to pdf in case of lines 266, 271, 333, 378, 447 missing letter in 472.

Response 2: Thank you for the evaluation. We have revised it in line 504.

Point 3: please re-read the manuscript and unify shortcuts, as I am not sure, whether state owned enterprises are SOE, or SEO – e.g. line 235, 236.

Response 3: Thank you for the beneficial correction. We have revised. The correct abbreviation is SOE.

Point 4: line 436 – in the title of the table, there should be „push“ instead of „pull“ factor (suggestions:).

Response 4: Thank you for the suggestions. We have revised this.

Point 5: In line 440 + 441 you state: „ researchers asked the panel members to convey their experiences and knowledge related to process innovations that had been carried out in their respective companies “. I would suggest to briefly describe examples of key drivers and attitudes in case of concrete SOEs, as it could be a valuable additional information to quantitative results.

Response 5: Thank you, we have included panelists experience as a complement in the Discussion section, for instances:

Our experts also compare the operational unit with the higher innovation culture with more innovation proposals than other units. The ratio is approximately two to six each year. (line 495- 497)

At the beginning of employee recruitment, HR introduces a culture of innovation and innovation achievements created by the company and provides information on how innovation dramatically affects employees' career development. (line 508-510)

The leader in the work unit with the highest number of employees could become the driver of innovation. This leader could encourage all employees to innovate actively so that innovation output will be high. Usually, the most prominent company’s expenditure also comes from these work units; if they could innovate and achieve better cost efficiency, then it will have a significant impact on the company’s expenditure. (line 534-539)

Companies can create an integrated innovation management system from the early stage to the diffusion stage of innovation management as a form of commitment to innovation. All employees can contribute their innovation ideas by registering them through the system in the initial stage. The idea can be verified by the leader in each work unit and then realized with a pilot project. Through this innovation management system, work unit leaders can also directly evaluate the innovation performance of subordinates. In doing so, the leadership also actively participates in maintaining the company's innovation culture. (line 550-557)

Point 6: I find the titles and content of material and methods, resp. methodology chapters a bit confusing. It is not very common in scientific articles to introduce basic terms and their definitions in the chapter "Material and methods". As we do not have a goal, data and methodology in this obligatory chapter, it is necessary to modify the structuring of the manuscript.

Sub-chapters 2.1., 2.2., 2.3. and 2.4., I understand rather as a theoretical background, so I recommend to change the name of chapter 2 to „Theoretical background“. In other ways it was a fruitful reading and I am looking towards citing your results.

Response 6: Thank you for the examination. We have made improvements to the paper writing structure by adjusting the IMRAD that has been used as a reference by MDPI. We also accommodate input from other reviewers to provide literature review/theoretical background section in Sub Section 2.1. - 2.4.

Reviewer 5 Report

This paper intends to explore the success factors that influence the successful implementation of process innovation in Indonesian state-owned companies. For that purpose, the authors used the three stages of an online questionnaire for the Delphi method to obtain consensus from experts. The authors found 28 factors that influence the successful implementation of process innovation. Therefore, human resources and organizational factors, such as leadership, problem understanding, strategy, and culture, affected the success of process innovation in SOEs, even more than the research & development budget, support, and commitment factor of a given company.

The following are some suggestions for improvement:

• The research question of the study or hypothesis is not very clear. What is the value added provided by this paper?
• Methodology is clear, but it is not clear how the panel of experts who acted as respondents were selected. Which were the criteria for selection for this sample?
• There are some references in text not found in references list.

Author Response

Point 1: The research question of the study or hypothesis is not very clear. What is the value added provided by this paper?

Response 1: Thank you for the beneficial examination. We have made changes and improvements to the Introduction section. The purpose of our research is conveyed in paragraph 2 and at the end of paragraph 3. Companies proven to be sustainable in successful innovation can be an example for other SOEs when it comes to managing their own innovations. Next, in the Section Limitation and Further Studies, we plan to use the indicators of the success of process innovation to assess the performance of process innovation in several state-owned companies. In conclusion, SOEs can evaluate the implementation of innovation in a company to ensure the sustainability of its innovation. We plan to use the indicators of the success of process innovation to assess the performance of process innovation in several state-owned companies. In conclusion, SOEs can evaluate the implementation of innovation in a company to ensure the sustainability of its innovation.

Point 2: Methodology is clear, but it is not clear how the panel of experts who acted as respondents were selected. Which were the criteria for selection for this sample?

Response 2: Thank you for the question. We have described in the manuscript in Sub Section 3.2.1. This study involved 15-panel members consisting of 12 experts who were directors and senior managers of SOEs. We use 15 participants as panelists according to the requirements of the Delphi method which is usually involving 15-20 participants. Moreover, 75% (12/15) of the panelists are SOE practitioners, it is expected to provide a lot of practical experience when making process innovations in their companies. That will be additional information on the analysis we do. The 12 panelists (75%) are from practitioners who have been actively involved in process innovation activities in SOEs, expected to provide practical experience. Three other panelists (from ministries and academics) were assigned as keynote speaker who validated panelists’s experience with the latest literature and regulations.

Point 3: There are some references in text not found in references list.

Response 3: Thank you for the evaluation. We have revised it.

Round 2

Reviewer 2 Report

Authors reflect to my comments. Now paper is sufficient and can be published.

Reviewer 3 Report

The paper has been revised significantly. While the first paragraph in the introduction has been altered, I still believe that the mentions of Indonesia SOEs can be placed at the end of the paragraph. This would be a better transition to the second paragraph, mainly discussing the open innovation in the SOEs. However, they are acceptable in the current form.