Identifying Service Opportunities Based on Outcome-Driven Innovation Framework and Deep Learning: A Case Study of Hotel Service

Round 1

Reviewer 1 Report

I congratulate the authors for the very interesting topic of their research and for the work they invested in writing this article. 

 

I have the following recommendations for improving the quality of this paper so it will be suitable for publishing in such a high-quality journal like Sustainability:

 

a. The coherency of the paper must be improved. There should be alignment between title, abstract, introduction, body of article and conclusions. In the title business opportunities are mentioned, but the paper does not address the topic of business opportunities identification. The ODI framework is about identifying opportunities for product and/or service innovations not about business opportunities. You also mentiond in line 443 that your results present sources of service innovations.

It should be clearer in the title and/or abstract that you have conducted a case study because your data refers only to the reviews provided by the clients of a single hotel.

 

b. In the theoretical background section you should present only the frameworks and the tools that you applied in this research (for example you presented the universal job map and you argued that is not appropriated for the purpose of your research- so my question is why you wrote about it?). Also, you should stress why the framework and tools you have applied are suitable for the hotel industry given their previous uses/applications.

c. Since the 3rd section should present the method you have applied, I think that all examples provided in this section should be based on this particular research and not some random examples (e.g.-examples in line 257, 263 etc.)

d. If the purpose of your study is to propose a method than you should specify if it's a new method or what are the foundations of your method and what you brought to the method. Also, you should stress why your method is appropriate in a service context, what are the benefits and the limitations of this method.

e. You should stress what your papers brigs to the literature and practice, the limitations of your research and future research direction in the conlusion section.

f. You have to make sure that you properly acknowledged all your sources - for the framework, methods, tools, metrics, figures etc.

g. there are several spelling error and you shoul correct them.

Author Response

Response to reviewers

>> Thank you so much for providing the review. The reviews were helpful to us in improving the paper and we have addressed each concern to the best of our ability. Based on your guidance, we have tried our best to completely address your comments. Our replies (in RED) are below. The revisions are colored in RED in the revised manuscript as well.

  

Reviewer #1

I congratulate the authors for the very interesting topic of their research and for the work they invested in writing this article.

I have the following recommendations for improving the quality of this paper so it will be suitable for publishing in such a high-quality journal like Sustainability:

1. The coherency of the paper must be improved. There should be alignment between title, abstract, introduction, body of article and conclusions. In the title business opportunities are mentioned, but the paper does not address the topic of business opportunities identification. The ODI framework is about identifying opportunities for product and/or service innovations not about business opportunities. You also mentiond in line 443 that your results present sources of service innovations.

It should be clearer in the title and/or abstract that you have conducted a case study because your data refers only to the reviews provided by the clients of a single hotel.

 >> Thank you for your comment and we agree with your comment. Our paper is aiming to identify service opportunities or customer’s needs, not business opportunities. Therefore, we have replaced all ‘business opportunities’ to ‘customer’s needs’ or ‘service opportunities (Line 13, 18, 41, 48, 69, 104, 121, 123, 178, 309, 314, 317, 336, 341, 427, 428, and 518).

So, Business opportunity score (BOB) has been revised to Service opportunity score (SOS) (Line 314, 424, 434, 440, 449, 457, 459, 467, 473, 480, 509-510, 511, and Appendix A)

In addition, we have revised the title. Business opportunities’ is replaced by ‘service opportunities’ and the specific case information is added (Line 2~4).

 

2. In the theoretical background section you should present only the frameworks and the tools that you applied in this research (for example you presented the universal job map and you argued that is not appropriated for the purpose of your research- so my question is why you wrote about it?).

>> We have deleted the description on Universal job map and briefly mentioned it with Service job map (Line 107~111 and 127~135).

• For this, ODI suggested Service job map (Figure 2) [18]. Service job map can effectively visualize an overall operational process of a specific job and so it is a basis framework that enables ODI to discover desired outcomes from unexpected steps. Service job map consists of 12 steps and can fully consider the characteristic that service begins with defining service needs and accessing to service providers and ends with payment.
• There were several attempts to apply ODI process in other fields. Lim, et al. [19] proposed semi-automatic method to construct simple job map for product out of patent documents. By analyzing patent documents based on ODI’s job classification, jobs-to-be-done were easily identified. However, this method is aiming product, not service, and because the jobs were extracted from patents, which is written by product manufacturer, customers’ voice were not reflected. Joung, et al. [20] applied ODI into service domain to detect customer complaints from customer service centers. By extracting keywords and cluster them based on ODI’s job description, service provider can identify failure of their service. However, this paper aims to catch abnormal states while delivering service, not how customers evaluate to their service.

Also, you should stress why the framework and tools you have applied are suitable for the hotel industry given their previous uses/applications.

>> We have supplemented why we chose the hotel service case (Line 63~67).  

• Tourism industry is taking great economic importance and hotel sector is facing more intense competition (Lee et al., 2016). Moreover, hotel sector is heavily damaged by COVID-19 and is now facing new challenges to meet new customers’ needs during and post COVID-19 (Hao et al., 2020). Therefore, it is important to find identify customers’ needs promptly and constantly. NLP-aided ODI would be a perfect solution to service this challenge.

3. Since the 3rd section should present the method you have applied, I think that all examples provided in this section should be based on this particular research and not some random examples (e.g.-examples in line 257, 263 etc.)

>> Based on your comment, we have changed the sentence examples for NLP (Line 254 and 260).

• Mike, Alice and I went to school --> I asked for early checkin and extra roll-away
• He must run, swim and ride bicycle in triathlon --> He smiled and walked away

4. If the purpose of your study is to propose a method than you should specify if it's a new method or what are the foundations of your method and what you brought to the method. Also, you should stress why your method is appropriate in a service context, what are the benefits and the limitations of this method.

>> To specify and stress the contribution of our method, we have supplemented the description on the breakthrough of our approach in Conclusion section (Line 490~496 and 497~500).

• Using ODI and BERT-based classification method, our proposed method can reduce the time and human effort necessary in analyzing large amount of online review data in service industry to identify new service opportunities. ODI framework is powerful framework, but it requires highly trained expert in this framework, which makes service providers hard to implement it. Especially, collecting and analyzing customers’ interview are time-consuming, expert-dependent work. The proposed method has resolved this issue by suggesting BERT-based semi-automated methods and proved it with a case study.
• While the classical ODI starts from customer interview, our approach starts from customer reviews. Compared with interviews, text reviews are relatively easy to collect and process automatically. ODI framework along with deep learning techniques enables human experts to efficiently analyze large amount of text datasets to identify service opportunities.

5. You should stress what your papers brings to the literature and practice, the limitations of your research and future research direction in the conlusion section.

>> The limitations of our method were already described in Conclusion section (Line 519~530), and future research direction was supplemented in the last paragraph of the Conclusion section (Line 531~536).

• Our further work will focus on neural network structure to perform better with less training datasets. Neural network techniques such as transfer learning is being considered. By collecting data from various service industry, neural network could be trained to classify general jobs and by further train task-specific data, the model is tuned to produce desirable outcome. To utilize transfer learning, services that shares common structure for process must be clustered beforehand, which is also another future research topic. To overcome the limitation regarding outcome cluster issue, GANs(Generative Adversarial Networks)-based sentence generation [43] is a carefully considered possible approach.

6. You have to make sure that you properly acknowledged all your sources - for the framework, methods, tools, metrics, figures etc.

>> Thank you so much for pointing out this issue. We missed the source of Figure 8 and have added the reference for Figure 8 (Line 366).

• Figure 8. Structural difference between TDNN and SS-BR, redrawn from [11]

7. there are several spelling error and you shoul correct them.

>> Some errors have been corrected.

• [Line 102] framewok --> framework
• [Line 171] eight steps --> six steps

Reviewer 2 Report

Dear authors:

Congratulations for this paper and for your effort. I think that your study is of great interest and covers an important gap within the literature. 

In any case, after reading your work in depth, I have some concerns that I hope can be addressed in the next round of review. 

1.- INTRODUCTION:

This section requires more elaboration and effort. Specifically, I have two major concerns:

1.1.- First, it is necessary to justify why authors focus on “hotel service”. They only state a minor justification at the beginning of “results” section, but this explanation is not sufficient for me. In this way, authors should make a greater effort to show the relevance and importance of this choice, and justify why they focus on it.       

1.2.- Second, authors should delve into the main contributions made by their work. This is an issue of great relevance in any study. Despite the interest of the topic addressed, authors must "sell" their work, and contributions play a transcendental role in this point.    

 

2.- THEORETICAL FRAMEWORK:

I think the "just -to-be-done" approach is very well explained. In addition, the figures included in this section are very illustrative for readers –even if they are not very familiar with this approach. 

In any case, I think there is still a path for improvement:

2.1.- Need for delving into the literature review. Beyond focusing on this "just -to-be-done" approach (as well as on the different maps illustrated), authors should discuss the previous evidence related to this topic (i.e., prior evidences regarding ODI, even if they are based on a different approach). This will let readers know how the state of art is in this moment.

2.2.- In addition, I think that “Section 2.2” should be integrated into the “method” section, since it is more linked to the methodology than to the theory.         

 

3.- METHODS:

I think this section is very well defined and explained. I am not an expert in this technique, but your description has allowed me to follow the different stages and their explication. I think it is a novel methodology and this fits very well with the aim of this research. Congrats!

I only have two minor comments related to this section:

3.1.- On page 05, authors show that this procedure has 8 stages. However, only 6 stages are shown. I assume this would be a typo.   

3.2.- Are there prior studies that use this procedure and its stages in a similar vein? In this case, please, citing them.   

 

4.- RESULTS:

I have two different comments:

4.1.- I think this section should be divided into two subsections: (1) A first subsection can comprise the more “technical results” related to the method (see pages 11-19 of the current version of the paper); (2) A second subsection can comprise the results related to specific customer needs, which are identified from the previous “technical results” (see pages 20-21 of the current version of the paper).

4.2.- A greater effort is necessary in the explanation related to this second subsection.

4.3.- Table 6 could be removed from the main body and included as an appendix. 

 

5.- CONCLUSIONS:

In general, this section needs further elaboration. My recommendations are the following:

5.1.- The results obtained in this paper should be compared with those stated by prior literature –although they do not address exactly this same approach–, indicating the novel contribution that this study makes to the existing literature.          

5.2.- Theoretical and practical implications should be stated and explained; this is particularly relevant in this study due to its interesting results.          

5.3.- A greater effort should be made to identify future lines of research. This issue is explained very succinctly in the current version.         

 

I hope that all my comments are useful for improving your paper.

Good luck!

Author Response

Response to reviewers

>> Thank you so much for providing the review. The reviews were helpful to us in improving the paper and we have addressed each concern to the best of our ability. Based on your guidance, we have tried our best to completely address your comments. Our replies (in RED) are below. The revisions are colored in RED in the revised manuscript as well.

Reviewer #2

Dear authors:

Congratulations for this paper and for your effort. I think that your study is of great interest and covers an important gap within the literature.

In any case, after reading your work in depth, I have some concerns that I hope can be addressed in the next round of review.

1.- INTRODUCTION:

This section requires more elaboration and effort. Specifically, I have two major concerns:

1.1.- First, it is necessary to justify why authors focus on “hotel service”. They only state a minor justification at the beginning of “results” section, but this explanation is not sufficient for me. In this way, authors should make a greater effort to show the relevance and importance of this choice, and justify why they focus on it.

>> Thank you for your comment. We have supplemented why we chose the hotel service case (Line 63~67). 

• Tourism industry is taking great economic importance and hotel sector is facing more intense competition (Lee et al., 2016). Moreover, hotel sector is heavily damaged by COVID-19 and is now facing new challenges to meet new customers’ needs during and post COVID-19 (Hao et al., 2020). Therefore, it is important to find identify customers’ needs promptly and constantly. NLP-aided ODI would be a perfect solution to service this challenge.

1.2.- Second, authors should delve into the main contributions made by their work. This is an issue of great relevance in any study. Despite the interest of the topic addressed, authors must "sell" their work, and contributions play a transcendental role in this point.

>> To stress the contribution of our method, we have supplemented the description on the breakthrough of our approach in Conclusion section (Line 490~496 and 497~500).

• Using ODI and BERT-based classification method, our proposed method can reduce the time and human effort necessary in analyzing large amount of online review data in service industry to identify new service opportunities. ODI framework is powerful framework, but it requires highly trained expert in this framework, which makes service providers hard to implement it. Especially, collecting and analyzing customers’ interview are time-consuming, expert-dependent work. The proposed method has resolved this issue by suggesting BERT-based semi-automated methods and proved it with a case study.
• While the classical ODI starts from customer interview, our approach starts from customer reviews. Compared with interviews, text reviews are relatively easy to collect and process automatically. ODI framework along with deep learning techniques enables human experts to efficiently analyze large amount of text datasets to identify service opportunities.

 

2.- THEORETICAL FRAMEWORK:

I think the "just -to-be-done" approach is very well explained. In addition, the figures included in this section are very illustrative for readers –even if they are not very familiar with this approach.

In any case, I think there is still a path for improvement:

2.1.- Need for delving into the literature review. Beyond focusing on this "just -to-be-done" approach (as well as on the different maps illustrated), authors should discuss the previous evidence related to this topic (i.e., prior evidences regarding ODI, even if they are based on a different approach). This will let readers know how the state of art is in this moment.

>> We have supplemented the descriptions on previous ODI related studies with references (Line 127~135).

• There were several attempts to apply ODI process in other fields. Lim et al. [19] proposed semi-automatic method to construct simple job map for product out of patent documents. By analyzing patent documents based on ODI’s job classification, jobs-to-be-done were easily identified. However, this method is aiming product, not service, and because the jobs were extracted from patents, which is written by product manufacturer, customers’ voice were not reflected. Joung, et al. [20] have applied ODI into service domain to detect customer complaints from customer service centers. By extracting keywords and cluster them based on ODI’s job description, service provider can identify failure of their service. However, this paper aims to catch abnormal states while delivering service, not how customers evaluate to their service.

 

2.2.- In addition, I think that “Section 2.2” should be integrated into the “method” section, since it is more linked to the methodology than to the theory.

>> We agree that some descriptions in Section 2.2. seem to be integrated to the method section. So, the last part of Section 2.2 has been moved into Section 3.3 (Line 192~204).

• Multi-label classifications commonly perform the problem transformation where multi-label problems were transformed into one or more single-label problems, such as a multi-class classification [29]. Binary relevance (BR) is the most common method when problem transformation is performed [30]. BR changes a multi-label problem into a series of binary problems that each binary problem predicts the relevance of one of the labels. When the dependency between labels is weak, BR can achieve higher quality [31]. BR neural network is not a popular method since each isolated binary network needs training while ignoring dependencies between labels [32]. When samples are assigned to a single label, the problem is called multi-class classification. In such case, BR is not a good method to classify samples because BR can assign multiple labels at the same time. In multi-class classification problem, one-vs-all method is widely used [33]. One-vs-all method makes binary decision whether a sample belongs to one class or the others. By calculating the probability for all classes, the class with the highest probability is selected. In this way, only one class is assigned to a sample.

 

3.- METHODS:

I think this section is very well defined and explained. I am not an expert in this technique, but your description has allowed me to follow the different stages and their explication. I think it is a novel methodology and this fits very well with the aim of this research. Congrats!

I only have two minor comments related to this section:

3.1.- On page 05, authors show that this procedure has 8 stages. However, only 6 stages are shown. I assume this would be a typo.

>> Thank you. We have revised it.

• [Line 171] eight steps --> six steps

3.2.- Are there prior studies that use this procedure and its stages in a similar vein? In this case, please, citing them.

>>We have added the related literature and explained what they have done and their limitations (Line 127~135).

• There were several attempts to apply ODI process in other fields. Lim et al. [19] proposed semi-automatic method to construct simple job map for product out of patent documents. By analyzing patent documents based on ODI’s job classification, jobs-to-be-done were easily identified. However, this method is aiming product, not service, and because the jobs were extracted from patents, which is written by product manufacturer, customers’ voice were not reflected. Joung, et al. [20] have applied ODI into service domain to detect customer complaints from customer service centers. By extracting keywords and cluster them based on ODI’s job description, service provider can identify failure of their service. However, this paper aims to catch abnormal states while delivering service, not how customers evaluate to their service.

4.- RESULTS:

I have two different comments:

4.1.- I think this section should be divided into two subsections: (1) A first subsection can comprise the more “technical results” related to the method (see pages 11-19 of the current version of the paper); (2) A second subsection can comprise the results related to specific customer needs, which are identified from the previous “technical results” (see pages 20-21 of the current version of the paper).

4.2.- A greater effort is necessary in the explanation related to this second subsection.

>> Based on your comment, we have divided the chapter 4 into two sub-chapters: 4.1. technical result (Line 373) and 4.2. implications (Line 426). Technical result mainly shows the results related to the processing performance, identified information, and scores. Implications mainly shows the practical service opportunities of the identified desired outcomes. Even though ODI framework recommends the only desired outcomes having very high SOS, we set the relatively low criterion (SOS>=10) to identify as many service opportunities as possible.

 

4.3.- Table 6 could be removed from the main body and included as an appendix.

 >> Table 6 has moved to Appendix section.

 

5.- CONCLUSIONS:

In general, this section needs further elaboration. My recommendations are the following:

5.1.- The results obtained in this paper should be compared with those stated by prior literature –although they do not address exactly this same approach–, indicating the novel contribution that this study makes to the existing literature.

>> The breakthrough of this method is the use of ODI framework (particularly Service job map) and advanced machine learning technique to link job-to-be-dones to Service map step, to automatically generate the outcomes, and to automatically calculate service opportunity scores. Since it is a new approach (i.e. different input and different output), the direct comparisons with previous works seem to be difficult. Most previous works using review data for service innovation, including opportunity identification, have usually considered the keyword occurrence frequency to identify important information [40, 41, 42]. But the result shows that they could not identify some very important desired outcomes (or similar expressions) that the proposed method successfully identified. Therefore, we can say that the proposed method can be used for alternative approach for service opportunity identifications. We have supplemented this issue in the conclusion section (Line 504~508).

• Prior research on analyzing hotel service using online customer review [40] have calculated keyword frequencies and divided reviews into four groups. Other studies [41, 42] also uses keyword frequencies to analyze large amount of text data. Considering that “billing mistake” caused hotel’s internal process problems and online payment problems, keyword-frequency-based analysis have limitation in identifying such context.

5.2.- Theoretical and practical implications should be stated and explained; this is particularly relevant in this study due to its interesting results.

>> To stress our contribution, we have supplemented the theoretical and practical implications in the conclusion section (Line 490~496).

• Using ODI and BERT-based classification method, our proposed method can reduce the time and human effort necessary in analyzing large amount of online review data in service industry to identify new service opportunities. ODI framework is powerful framework, but it requires highly trained expert in this framework, which makes service providers hard to implement it. Especially, collecting and analyzing customers’ interview are time-consuming, expert-dependent work. The proposed method has resolved this issue by suggesting BERT-based semi-automated methods and proved it with a case study.

5.3.- A greater effort should be made to identify future lines of research. This issue is explained very succinctly in the current version.

>> We have supplemented the future direction of our work (Line 531~536).

 

• Our further work will focus on neural network structure to perform better with less training datasets. Neural network techniques such as transfer learning is being considered. By collecting data from various service industry, neural network could be trained to classify general jobs and by further train task-specific data, the model is tuned to produce desirable outcome. To utilize transfer learning, services that shares common structure for process must be clustered beforehand, which is also another future research topic. To overcome the limitation regarding outcome cluster issue, GANs(Generative Adversarial Networks)-based sentence generation [43] is a carefully considered possible approach.

Reviewer 3 Report

Dear Authors,

Please take into consideration my comments and suggestions below:

1. Shorten the title. It uses too many concepts.
2. Introduce in the abstract some clear conclusions. It is too generic and it shall increase the visibility of your paper.
3. Tables that are bigger than 2/3 of a page shall be put in the Appendix.
4. I would remove lines 67-69 as the content is very clear.
5. Figure 2 needs referencing.

Even if the article is strong in methodology and data analysis, I see no relevance for publishing it in a journal focused on Sustainability. Please check the aims of this journal. Moreover, there are so many journals on Marketing which would attract your targeted readers.

 

Yours faithfully,

Author Response

Response to reviewers

>> Thank you so much for providing the review. The reviews were helpful to us in improving the paper and we have addressed each concern to the best of our ability. Based on your guidance, we have tried our best to completely address your comments. Our replies (in RED) are below. The revisions are colored in RED in the revised manuscript as well.

 

Reviewer #3

 

Please take into consideration my comments and suggestions below:

 

1. Shorten the title. It uses too many concepts.

>> We have tried to shorten the title, but other reviewer recommended to add the detailed of case study in the title (Line 2~4).

 

• Outcome-driven innovation-based service opportunity identification using deep learning: A case study of hotel service

 

2. Introduce in the abstract some clear conclusions. It is too generic and it shall increase the visibility of your paper.

>> Thank you for your good point. We have supplemented some of the result statements in Abstraction section (Line 21~22).

 

• – e.g. maximizing safety to pay price/deposit, and maximizing possibility to avoid waiting at lobby –

 

3. Tables that are bigger than 2/3 of a page shall be put in the Appendix.

>> We have moved Table 6 to Appendix section.

 

4. I would remove lines 67-69 as the content is very clear.

>> We have removed the last paragraph of the introduction section.

 

5. Figure 2 needs referencing.

>> Figure 2 has been deleted because of another reviewer’s comment.

 

Even if the article is strong in methodology and data analysis, I see no relevance for publishing it in a journal focused on Sustainability. Please check the aims of this journal. Moreover, there are so many journals on Marketing which would attract your targeted readers.

>> Thank you for your advice. We will consider other journals for our further work.

Round 2

Reviewer 1 Report

I appreciate the effort of the authors to adress my observations. I think the quality of the paper has improved.

There are still 3 aspects that still need to be considered:

1. My observation was concerning the suitability of this method for the hotel industry and not to stress why you have chosen hotel industry. So, in the section dedicated to Theoretical background you should discuss, based on previous research that applied the same framework, why the framework and tools you have applied are suitable for the hotel industry. How was this method used in previous research? What were the outcomes?

2.Although you mentioned the limitations of your research and future research direction you failed to stress what your papers brings to the literature and practice and this is very important.

3. Last but not least you have to correct all your grammar and spelling errors.

Author Response

Reviewer #1

I appreciate the effort of the authors to address my observations. I think the quality of the paper has improved.

There are still 3 aspects that still need to be considered:

1. My observation was concerning the suitability of this method for the hotel industry and not to stress why you have chosen hotel industry. So, in the section dedicated to Theoretical background you should discuss, based on previous research that applied the same framework, why the framework and tools you have applied are suitable for the hotel industry. How was this method used in previous research? What were the outcomes?

>> The aim of this paper is to develop a universal method that can be applied any service sector. Therefore, other service domain can be the empirical case for this paper, and we are actually developing other case studies using the method for further research papers and the systemization of the method. Since the case study for this paper should show the advancement of the method, we now think the reason why we chose the hotel service should be more technical aspect based on the characteristics of the proposed method.

The reasons for selection are simple and clear: 1) data accessibility, 2) various additional services, such as facilities, restaurant, and room service, and 3) many interactions with customers. We have supplemented these reasons in the empirical analysis section.

• [Line 344~348] In addition, from the methodological perspective, customer review data related to hotel service is relatively easy to collect, and hotel service includes various additional services, such as facilities, restaurant, and room service and so there are many interactions with customers. Therefore, the hotel service is the case that service opportunities can be identified from various steps in the Service job-map and so ~

In addition, the previous studies using ODI framework do not use data-driven analysis (e.g. [8, 9]) or focus on product opportunities based on Universal job map (we adopted Service job map) (e.g. [19, 20]). Therefore, since the comparison to previous studies using ODI framework seems to be inappropriate, it would be better to focus on the contribution of this paper (compared to previous studies), instead of focusing on why we chose the hotel case.

• [Line 498~518] The major contribution of the proposed method is as follows. First, this paper is the first attempt to develop a data-driven ODI approach for service sectors. More specifically, previous studies using a data-driven ODI approach focus on identifying product opportunities and so they are based on Universal job map which is the eight step job map for universal purpose, so not well applicable for service sectors. However, this paper adopted Service job map, instead of Universal job map, and so most of analytic processes were redesigned. Second, the performance for matching/classifying the extracted SAOs to one of Service job map steps is improved by using BERT-based attention network. Specifically, each sentence in review data is assigned into one of steps in Service job map using BERT-based multi-class sentence classification. Since this process can directly affect the characteristics of the defined outcomes, this process is critical for the proposed method because this process can directly affect the characteristics of the defined outcomes, and this paper, in particular, adopted more complex job map, i.e. Service job map. Third, the proposed method automatically generates the outcome statements for every step in Service job map. Each jobs-to-be-done is extended to the form of customer outcome and the service opportunity score (SOS) for each outcome is evaluated based on the importance and satisfaction scores. Generating various, but reliable, outcomes from different job map steps with minimizing cost is one of the most important demands in the ODI framework. Even though previous studies [19, 20] could not deal with this issue, this paper developed the method that fulfills the demand. Fourth, since the propose method can automatically generate outcome statements, the method can assess potential opportunities of the outcomes without any expensive interview process. Therefore, this research is the first attempt to identify service opportunities based on data-driven ODI approach without interview process.

 

2.Although you mentioned the limitations of your research and future research direction you failed to stress what your papers brings to the literature and practice and this is very important.

>> We think the cause of this comment is that we could not highlight our contributions well in our previous version. We are sorry for bothering you and appreciate you pointed it out. We have supplemented our contribution in the introduction and conclusion section.

• [Line 61~67] As the first attempt to develop a data-driven ODI approach for service sectors, the proposed method has clear contributions. First, this research adopted Service job map, instead of Universal job map, to cover more complex service process. Second, the performance of classifying job-to-be-done into one of Service job map steps is improved by using BERT-based attention network. Third, the outcome statements are automatically generated from review data. Lastly, the method can assess potential opportunities of the outcomes without any expensive interview process.

 

• [Line 498~518] The major contribution of the proposed method is as follows. First, this paper is the first attempt to develop a data-driven ODI approach for service sectors. More specifically, previous studies using a data-driven ODI approach focus on identifying product opportunities and so they are based on Universal job map which is the eight step job map for universal purpose, so not well applicable for service sectors. However, this paper adopted Service job map, instead of Universal job map, and so most of analytic processes were redesigned. Second, the performance for matching/classifying the extracted SAOs to one of Service job map steps is improved by using BERT-based attention network. Specifically, each sentence in review data is assigned into one of steps in Service job map using BERT-based multi-class sentence classification. Since this process can directly affect the characteristics of the defined outcomes, this process is critical for the proposed method because this process can directly affect the characteristics of the defined outcomes, and this paper, in particular, adopted more complex job map, i.e. Service job map. Third, the proposed method automatically generates the outcome statements for every step in Service job map. Each jobs-to-be-done is extended to the form of customer outcome and the service opportunity score (SOS) for each outcome is evaluated based on the importance and satisfaction scores. Generating various, but reliable, outcomes from different job map steps with minimizing cost is one of the most important demands in the ODI framework. Even though previous studies [19, 20] could not deal with this issue, this paper developed the method that fulfills the demand. Fourth, since the propose method can automatically generate outcome statements, the method can assess potential opportunities of the outcomes without any expensive interview process. Therefore, this research is the first attempt to identify service opportunities based on data-driven ODI approach without interview process.

To know the real service opportunities (important but unmet customer’s needs) can be the most critical task for new service development or service innovation. Therefore, we think the practical implication is simple but very clear and powerful (identifying unrecognized or hidden customer’s needs and to quantitatively denote what outcomes should be priory focused on, without any expensive interview process) and it is already clearly described in the conclusion section (Line 521~523).

 

3. Last but not least you have to correct all your grammar and spelling errors. 

>> We have found some errors in this revision and revised them. Please let us know (specifically) if there still exist some language errors. 

• [Line 77] but also is stable without changing over time à but also is stable over time
• [Line 152] This corporation of previous information enables RNN à This incorporation of previous information enables RNN.

Reviewer 2 Report

Dear authors:

Thanks for addressing some of my comments. It is true that this version has improved, although I think that several concerns are not still being adequately addressed. I indicate them below:

1.- Introduction: the contributions made by this article to this research field have not been highlighted. Beyond indicating them in the “conclusion”, it is necessary to enhance this issue in the introduction. Specifically, this needs to be addressed in depth. It is not enough to reference it succinctly in the last section.

2.- Theory: The discussion of prior evidence related to your approach remains very weak. For readers, it is very useful to show more clearly what the state of the art is, how far it has come, and how far your paper wants to go. That is something that is not sufficiently addressed in this revised version.

3.- Results: your proposed division is adequate. However, I recommend to rename the second sub-section. The reference used for “technical results” is good. However, I think that the name of “implications” is not correct since it refers to those issues discussed in conclusions.  Thus, you should change the name of this second sub-section.

4.- Conclusions: this section also needs further elaboration. The effort carried out in this revised version is quite brief, and the contributions and implications (theoretical and practical) derived from the results are not well identified and discussed.

I hope my comments ca be useful for moving this article forward.

Author Response

Reviewer #2

 

Thanks for addressing some of my comments. It is true that this version has improved, although I think that several concerns are not still being adequately addressed. I indicate them below:

1.- Introduction: the contributions made by this article to this research field have not been highlighted. Beyond indicating them in the “conclusion”, it is necessary to enhance this issue in the introduction. Specifically, this needs to be addressed in depth. It is not enough to reference it succinctly in the last section.

>> Thank you for your comment. We agree that your guidance can clearly highlight the contribution and advancement of our paper. We have supplemented the contribution of the proposed method in the conclusion section and then summarized them into the introduction section.

• [Line 61~67] As the first attempt to develop a data-driven ODI approach for service sectors, the proposed method has clear contributions. First, this research adopted Service job map, instead of Universal job map, to cover more complex service process. Second, the performance of classifying job-to-be-done into one of Service job map steps is improved by using BERT-based attention network. Third, the outcome statements are automatically generated from review data. Lastly, the method can assess potential opportunities of the outcomes without any expensive interview process.

 

• [Line 498~518] The major contribution of the proposed method is as follows. First, this paper is the first attempt to develop a data-driven ODI approach for service sectors. More specifically, previous studies using a data-driven ODI approach focus on identifying product opportunities and so they are based on Universal job map which is the eight step job map for universal purpose, so not well applicable for service sectors. However, this paper adopted Service job map, instead of Universal job map, and so most of analytic processes were redesigned. Second, the performance for matching/classifying the extracted SAOs to one of Service job map steps is improved by using BERT-based attention network. Specifically, each sentence in review data is assigned into one of steps in Service job map using BERT-based multi-class sentence classification. Since this process can directly affect the characteristics of the defined outcomes, this process is critical for the proposed method because this process can directly affect the characteristics of the defined outcomes, and this paper, in particular, adopted more complex job map, i.e. Service job map. Third, the proposed method automatically generates the outcome statements for every step in Service job map. Each jobs-to-be-done is extended to the form of customer outcome and the service opportunity score (SOS) for each outcome is evaluated based on the importance and satisfaction scores. Generating various, but reliable, outcomes from different job map steps with minimizing cost is one of the most important demands in the ODI framework. Even though previous studies [19, 20] could not deal with this issue, this paper developed the method that fulfills the demand. Fourth, since the propose method can automatically generate outcome statements, the method can assess potential opportunities of the outcomes without any expensive interview process. Therefore, this research is the first attempt to identify service opportunities based on data-driven ODI approach without interview process.

 

2.- Theory: The discussion of prior evidence related to your approach remains very weak. For readers, it is very useful to show more clearly what the state of the art is, how far it has come, and how far your paper wants to go. That is something that is not sufficiently addressed in this revised version.

>> Actually, there is few researches on data-driven ODI approach. To our knowledge, there are only two previous studies [19, 20] and so that is the reason you feel insufficient literature review on data-driven ODI framework. Therefore, we have supplemented the details of [19, 20] in this revision.

• [Line 128~138] There exist several attempts to apply ODI process in other fields. Lim, et al. [19] proposed semi-automatic method to construct simple Universal job map for a specific product from patent data. By analyzing patent documents based on ODI’s job classification, jobs-to-be-done were identified. However, since this research focuses on the product opportunities using patent data, the analytic process structure and data are basically different from service-related research. Joung, et al. [20] applied ODI to detect customer complaints in end-user products. This research extracts keywords and functional information from webpages and classifying them into Universal job map steps using semantic similarity analysis. Based on the clustered information in each step, ODI experts define the complaints of the product. Even though these studies tried to develop a data-driven ODI approach, the research purpose, method structure and data are fundamentally different from the proposed method and insufficient to apply to identify service opportunities.

3.- Results: your proposed division is adequate. However, I recommend to rename the second sub-section. The reference used for “technical results” is good. However, I think that the name of “implications” is not correct since it refers to those issues discussed in conclusions.  Thus, you should change the name of this second sub-section.

>> We have revised to ‘Service opportunity analysis’ (Line ~~).

4.- Conclusions: this section also needs further elaboration. The effort carried out in this revised version is quite brief, and the contributions and implications (theoretical and practical) derived from the results are not well identified and discussed.

>> We have supplemented the description on the contribution.

• [Line 498~518] The major contribution of the proposed method is as follows. First, this paper is the first attempt to develop a data-driven ODI approach for service sectors. More specifically, previous studies using a data-driven ODI approach focus on identifying product opportunities and so they are based on Universal job map which is the eight step job map for universal purpose, so not well applicable for service sectors. However, this paper adopted Service job map, instead of Universal job map, and so most of analytic processes were redesigned. Second, the performance for matching/classifying the extracted SAOs to one of Service job map steps is improved by using BERT-based attention network. Specifically, each sentence in review data is assigned into one of steps in Service job map using BERT-based multi-class sentence classification. Since this process can directly affect the characteristics of the defined outcomes, this process is critical for the proposed method because this process can directly affect the characteristics of the defined outcomes, and this paper, in particular, adopted more complex job map, i.e. Service job map. Third, the proposed method automatically generates the outcome statements for every step in Service job map. Each jobs-to-be-done is extended to the form of customer outcome and the service opportunity score (SOS) for each outcome is evaluated based on the importance and satisfaction scores. Generating various, but reliable, outcomes from different job map steps with minimizing cost is one of the most important demands in the ODI framework. Even though previous studies [19, 20] could not deal with this issue, this paper developed the method that fulfills the demand. Fourth, since the propose method can automatically generate outcome statements, the method can assess potential opportunities of the outcomes without any expensive interview process. Therefore, this research is the first attempt to identify service opportunities based on data-driven ODI approach without interview process.

To know the real service opportunities (important but unmet customer’s needs) can be the most critical task for new service development or service innovation. Therefore, we think the practical implication is simple but very clear and powerful (identifying unrecognized or hidden customer’s needs and to quantitatively denote what outcomes should be priorly focused on, without any expensive interview process) and it is already clearly described in the conclusion section (Line 521~523).

I hope my comments ca be useful for moving this article forward.

Reviewer 3 Report

Dear Authors,

The current form of the manuscript is much clearer, except one thing: the title is very ambiguous with 'Outcome-driven innovation-based service opportunity identification' which is incorrect according to English grammar. Please rephrase it, as for example: 'Identifying service opportunities based on outcomes and innovation... 

Yours faithfully,

Author Response

Reviewer #3

 

The current form of the manuscript is much clearer, except one thing: the title is very ambiguous with 'Outcome-driven innovation-based service opportunity identification' which is incorrect according to English grammar. Please rephrase it, as for example: 'Identifying service opportunities based on outcomes and innovation...

 

>> Based on your guidance, we have renamed the title. But we still used the term ‘outcome-driven innovation’, because it is the name of the theory.

 

• [Line 2~3] Title: Identifying service opportunities based on outcome-driven innovation framework and deep learning: A case study of hotel service

Round 3

Reviewer 1 Report

Dear authors,

I would like to congratulate you for all the effort to put in addressing the observations/recommendations of all the reviewers. I think the quality of this version of your paper is significantly better than of the first version and thus is suitable for publication in Sustainability Journal.

Congratulations!

Reviewer 2 Report

Dear authors:

Many thanks for considering my comments and suggestions.

I think that our article, in its current version, can be published in Sustainability. Congratulations.

Thanks again for your effort.