A BERT-Based Multi-Criteria Recommender System for Hotel Promotion Management
Round 1
Reviewer 1 Report
In this submission, the authors present a multi-criteria Recommender System for matching potential customers to hotels, by analyzing their reviews using the BERT language model. Prior to accepting their contribution for publications, the following issues need to be resolved.
- The Theoretical Background section should be renamed to Related Work. Additionally, this section should cite similar approaches that use context-independent embeddings (Word2Vec, Paragraph Vectors, etc) like the following:
[1] Alexandridis, G., Tagaris, T., Siolas, G., & Stafylopatis, A. (2019, May). From free-text user reviews to product recommendation using paragraph vectors and matrix factorization. In Companion Proceedings of The 2019 World Wide Web Conference (pp. 335-343).
[2] Alexandridis, G., Siolas, G., & Stafylopatis, A. (2017). ParVecMF: A paragraph vector-based matrix factorization recommender system. arXiv preprint arXiv:1706.07513.
- The extensive analysis regarding the BERT model in lines 183-202 is redundant and should be omitted. It is sufficient to say that BERT is among the-state-of-the-art neural language models.
- Do you plan to make the crawled dataset publicly available so that other researchers might use it and compare with you?
- line 246: ReLU is an activation function and not a layer. This point here needs further clarification
- line 256: The Mean square loss is well known therefore Eq. 2 is redundant.
- line 269-271: In essence, you are describing the recommendation process here. You should extend this description and make it more clear.
- In Figure 5, the training results indicate an overfit model. You should better explore the hyperparameter space (lines 292-296).
- line 314: removing users/hotels with few ratings makes the dataset artificially dense and is an indication that the model you propose cannot address the cold-start problem (an important issue in recommender system design). You should at least explicitly discuss how are you going to deal with the cold-start problem.
- In the experiments, you miss comparison with other state-of-the-art methods.
- Finally, proofread your manuscript and correct grammatical and syntactical mistakes
Other minor corrections:
l. 13: remove the second “reviews”
l.91: It is a subjective opinion to characterize TripAdvisor as the best travel website. I would suggest using more neutral wording here, like “one of the largest”
l. 101 “is recommended to the hotel” -> “are recommended to the hotel"
l. 134 “have been started to be conducted” -> “have been conducted”
l 144 “In the similarity” -> “The similarity”
l. 150 “[31]proposes” -> “[31] proposes” (and similarly on line 155)
l. 153 a full-stop is missing
l.222 “by” -> “on”
Author Response
Please refer to the attached file for details
Author Response File: 
Author Response.docx
Reviewer 2 Report
My first remark relate to using the data which is owned by TripAdvisor. On the page of TripAdvisor (https://developer-tripadvisor.com/content-api/terms-and-conditions/, see section 6 Ownership) one can read: “TripAdvisor shall own and retain all right, title and interest in and to any, documents, techniques, know-how , specifications, plans, notes, drawings, designs, pictures, inventions, data, information and other Content provided by TripAdvisor and any derivative works hereof in connection with this Agreement (“Work Product”), including any and all intellectual property rights therein”. Collecting data by web scrapping on TripAdvisor webpage seems to be illegal and non-ethical. Thus, I strongly advice the authors to change the data source for the analysis. You might easily buy a data presented in Google Maps (both about reviews and ratings) from Google. On the other hand, I understand that mentioned Google Maps service does not collect the rating for every particular attribute of hotel. It is difficult situation you should address from both legal, ethical, and research perspective.
Second, there is no information about language of reviews. You decided to choose New York City as a research area. This is one of most international destinations in the world. Thus, there is high probability that reviews posted on TriAdvisor include those which are delivered in English by native speakers, in English by non-native speakers, and in other languages. Detection of sentiments in multilingual environment is extremely complex. Even using one language like English by both of mentioned groups (native speakers, and non-native speakers) in analysis is complicated. The language is part of the culture which significantly influence our perception of the world. Thus, investigation of sentiments should cover the problem of cultural differences. And the problem should be deeply discussed in the literature review, and the section of methodological framework.
Finally, with no doubts, in the paper the authors were focused on the method. However, I strongly advice to prepare the section like managerial applications. The goal of the social scientists is not only to research anything, but also to share the research with non-academic stakeholders. Thus, one paragraph in the section of conclusions is definitely not enough. My suggestion is to improve section of experiments and results, and to interpret your results, not only present the numbers. Second, deliver section of discussion, and compare your achievements with findings of other authors you already mentioned in literature review. Finally, deliver additional section of managerial applications, or merge this section with discussion.
Author Response
Please refer to the attached file for details
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
In this revised version of the original manuscript, the authors addressed virtually all of the points raised in my previous review, therefore I suggest the submission to be accepted for publication. Nevertheless, the manuscript should still be checked for grammatical & syntactical mistakes.
Reviewer 2 Report
Dear Authors. Much appreciate all your efforts to consider all my remarks and comments. I can only confirm that you have already meet all my expectations to make your paper to be accepted for publication. Hope, that the Editor-in-Chief will agree this opinion.
I was also thinking about TripAdvisor. I suppose that some kind of open access platform, not oriented on mercantile purposes, is truly demanded. Or maybe some additional feature for Open Street Map? In fact the content published in TripAdvisor is customer generated content. As thus, we - customers, tourists - should keep our legal rights to this.
Good luck with your research!
