An Assessment Methodology about the Effectiveness of Mobility IT Solutions: Application to Six Demo Sites

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper(sustainability-2815271) introduces a systematic assessment approach to quantify the alignment of specific innovative technologies developed under the IP4 Shift2Rail program with travelers and Transport Service Providers' (TSPs) needs. This study applies a quantitative assessment methodology across six demo sites; a 5 module toolbox and assessment methodology are presented to more broadly evaluate the Travel Companion APP/IP4 ecosystem. However, there are some organizational issues in the structure of the content, making it challenging to read. Additionally, this study may lack an enough deep workload in its methodology.

 

The comments for authors’ considerations (also can see the Attach).

 

1. Page2, line 86-88, This paper mentions that the evaluation method used is referred to in a previous paper. Compared to the previous method, what are the differences in the evaluation method employed in this paper? Or, in other words, what innovations does this paper contribute in this regard?

 

2. This paper does not include a literature review section. Why is that? It is recommended to supplement the paper with a literature review section.

 

3. Page2, line 71-77, The paper mentions that these problems can be solved, so what is the meaning of solving these problems?

 

4. Page3, line 114-118, Steps 1 and 2 both involve definitions and appear to be overly cumbersome. It is suggested to combine them into a single step.

 

5. Figure 1 is too arbitrary. The legends and information are unclear for reading and understanding. Thus, the graphical efforts are not enough to help explain. It is recommended to revise.

 

6. Equation (1) may contain an error. express the number of questions applicable to all the profiles measuring the satisfaction with the functionality “j” offered by the TSP “k”. So why add ?

 

7. Equation (4) is difficult to read. There appears to be a formatting or typography error. Please modify.

 

8. In the extended part of the methodology, why did you choose the methods AHP analysis, Regression analysis, Bayesian Network analysis, and ANOVA test? What are the advantages over other methods?

 

9. Page6, line 203-204, The figure titles mentioned in the paper do not correspond to the actual figure titles, please check.

 

10. Table 2 is designed to be very difficult to read, with a large number of repeated words.

11. The fourth part of the paper contains a large number of repeated phrases and words. For example, lines 308-310 and 362-364, this situation occurs in many places.

 

 

12. Check lines 249-250 for errors.

Comments for author File: Comments.pdf

Comments on the Quality of English Language

The language level is clear. 

Author Response

Dear esteemed reviewer, We would like to express our sincere gratitude for taking the time to review our manuscript and providing us with your valuable feedback. Your insightful comments have helped us improve the quality of our work. We have carefully considered all of your suggestions and have made the necessary revisions to ensure that the manuscript meets the standards. We hope that our revisions meet with your approval. Please find the revised version of the manuscript and our answer to your comments attached. We look forward to any further feedback you may have. Thank you once again for your valuable contribution to our work. Best regards, 

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

In this manuscript, the authors propsed an assessment methodology based on the calculation of the “Effectiveness” of IT solutions with respect to these users’ needs and  expectations, which are developed in another work.  Here, they presented some results of an intersectional  assessment by applying this methodology with data collected in six demo sites  including Athens, Barcelona, Liberec, Osijek, Padua and Warsaw). 

 

The idea could be interesting, but the structure is not well organzied and the manuscript is a little tedious. In addition, there are also some problems to be addressed as follows

1) the structure could be re-organized;

2) There are too many tables, which can be greatly reduced or combined;

3) Figure 1 is too vague, and also not necessary.

 

Comments on the Quality of English Language

In this manuscript, the authors propsed an assessment methodology based on the calculation of the “Effectiveness” of IT solutions with respect to these users’ needs and  expectations, which were developed in another work.  Here, they presented some results of an intersectional  assessment by applying this methodology with data collected in six demo sites  including Athens, Barcelona, Liberec, Osijek, Padua and Warsaw). 

 

The idea could be interesting, but the structure is not well organzied and the manuscript is a little tedious. In addition, there are also some problems to be addressed as follows

1) the structure could be re-organized;

2) There are too many tables, which can be greatly reduced or combined;

3) Figure 1 is too vague, and also not necessary.

4) The words and expressions need to be futher polished, such as in Abstract, "An assessment methodology based on the calculation of the “Effectiveness” of IT solutions with respect to these users’ needs and expectations was developed in a previous work" is too long.

Author Response

Dear esteemed reviewer, We would like to express our sincere gratitude for taking the time to review our manuscript and providing us with your valuable feedback. Your insightful comments have helped us improve the quality of our work. We have carefully considered all of your suggestions and have made the necessary revisions to ensure that the manuscript meets the standards. We hope that our revisions meet with your approval. Please find the revised version of the manuscript and our answer to your comments attached. We look forward to any further feedback you may have. Thank you once again for your valuable contribution to our work. Best regards, 

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The authors present in the paper the application in six demo sites of the methodology for evaluating the effectiveness of IT solutions for travelers mobility.

The authors present in the paper the application in six demo sites of the methodology for evaluating the effectiveness of IT solutions for passenger mobility.

The methodological aspects have already been published in another paper (reference 20).

Although the research methodology is well described and the results are relevant, I suggest improving the paper in terms of structure. 

The inclusion of all the information, for all six sites, makes the paper hard to read. The number of tables (45), with the same information, but referring to another site, loads the paper with irrelevant information. 

Therefore, in order to improve the form of the paper, I suggest summarising the information in a reduced number of tables.

Author Response

Dear esteemed reviewer, We would like to express our sincere gratitude for taking the time to review our manuscript and providing us with your valuable feedback. Your insightful comments have helped us improve the quality of our work. We have carefully considered all of your suggestions and have made the necessary revisions to ensure that the manuscript meets the standards. We hope that our revisions meet with your approval. Please find the revised version of the manuscript and our answer to your comments attached. We look forward to any further feedback you may have. Thank you once again for your valuable contribution to our work. Best regards, 

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

The issue studied in this article is a very interesting topic, but the authors still need to conduct some in-depth academic research in order to provide more convincing quantitative conclusions.

(1) Are the quantitative calculation formulas (1) and (2) proposed in this article applicable to the author's research? The author did not engage in in-depth discussion.

(2) The conclusions obtained in the paper are mostly qualitative language descriptions, without providing more convincing quantitative conclusions and trend analysis.

(3) Why did the author not use analysis methods such as Bayesian networks?

(4) The paper spent a lot of space presenting information from six websites. What is its purpose?

(5) What is the basis for selecting these six sites? What is the representativeness? Is the conclusion obtained universally applicable?

Comments on the Quality of English Language

No

Author Response

Dear esteemed reviewer, We would like to express our sincere gratitude for taking the time to review our manuscript and providing us with your valuable feedback. Your insightful comments have helped us improve the quality of our work. We have carefully considered all of your suggestions and have made the necessary revisions to ensure that the manuscript meets the standards. We hope that our revisions meet with your approval. Please find the revised version of the manuscript and our answer to your comments attached. We look forward to any further feedback you may have. Thank you once again for your valuable contribution to our work. Best regards, 

Author Response File: Author Response.pdf

Reviewer 5 Report

Comments and Suggestions for Authors

Frankly, I think this paper discussed an interesting topic. But there are some unreasonable points in the paper. For five reasons I cannot accept the paper. The reasons include as follows.

1) There are too many key words in the paper.

2) The explanation of the two variables  and  seems to be missing from the paper.

3) The elaboration of the research questions in the thesis is somewhat confused and disordered. It is suggested that the research questions be reconceived and accurately expressed so as to make the theme of the thesis more prominent and orderly.

4) The definition and presentation of "effectiveness" in the paper seems to be inappropriate.

5) There are too many concepts and methods jumbled together in the paper, which is a bit cluttered and disjointed.

Comments on the Quality of English Language

Overall the language in the paper is not too problematic. However, some sentence structure could be slightly adjusted to improve fluency. For example, in the abstract, "It has become unavoidable to promote the culture of using Mobility as a Service (MaaS) among travellers to address climatic challenges, especially the global warming phenomenon. " could be changed to "Promoting the culture of Mobility as a Service (MaaS) among travelers has become unavoidable to address climatic challenges, particularly the global warming phenomenon.".

Author Response

Dear esteemed reviewer, We would like to express our sincere gratitude for taking the time to review our manuscript and providing us with your valuable feedback. Your insightful comments have helped us improve the quality of our work. We have carefully considered all of your suggestions and have made the necessary revisions to ensure that the manuscript meets the standards. We hope that our revisions meet with your approval. Please find the revised version of the manuscript and our answer to your comments attached. We look forward to any further feedback you may have. Thank you once again for your valuable contribution to our work. Best regards, 

Author Response File: Author Response.pdf

Reviewer 6 Report

Comments and Suggestions for Authors

The paper presents a comprehensive methodology for evaluating the effectiveness of IT solutions in the context of Mobility as a Service (MaaS). This paper develops an assessment methodology that measures IT solutions' effectiveness based on user needs and expectations. The following comments are recommended to improve the quality of the paper.

(1)  Can you provide more detailed information on the exact implementation steps of the assessment methodology?

(2)  The layout of page 5 should be adjusted.

(3)  How does the proposed methodology perform under different urban settings or demographic variations?

(4)  What are the limitations of your methodology, and how might they impact the study's findings?

(5)  Could you provide more details on the data collection process and the analysis methods used?

(6)  Enrich your references with recent related articles which might help the authors to develop future research. For example, “The vulnerability of distributed state estimator under stealthy attacks” in Automatica.

(7)  Proofread the paper again to correct typographical errors.

Comments on the Quality of English Language

Moderate

Author Response

Dear esteemed reviewer, We would like to express our sincere gratitude for taking the time to review our manuscript and providing us with your valuable feedback. Your insightful comments have helped us improve the quality of our work. We have carefully considered all of your suggestions and have made the necessary revisions to ensure that the manuscript meets the standards. We hope that our revisions meet with your approval. Please find the revised version of the manuscript and our answer to your comments attached. We look forward to any further feedback you may have. Thank you once again for your valuable contribution to our work. Best regards, 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

I can see the authors have revised the paper according to the comments. Thanks for the revisions, well done. I am happy to positively support its acceptance. It is a good work. 

Comments on the Quality of English Language

The language level is clear. 

Author Response

Dear reviewer,

I sincerely appreciate your thoughtful review of our paper.

Kind Regards, 

Reviewer 2 Report

Comments and Suggestions for Authors

The current revision is satisfactory, and it can be accepted in the present form.

Author Response

Dear reviewer,

I sincerely appreciate your thoughtful review of our paper.

Kind Regards, 

Reviewer 4 Report

Comments and Suggestions for Authors

(1) Please ask the author to supplement and improve the applicability analysis of the methods used in this article.

(2) Please explain the promotional value of the methods used in this article.

Comments on the Quality of English Language

no.

Author Response

Dear reviewer, 
We are thankful for your valuable comments.
Below, we tried to address and answer your comments in the revised version of the manuscript. 

(1) Please ask the author to supplement and improve the applicability analysis of the methods used in this article.

Answer:  Thank you so much for this comment. In the revised version of the manuscript, in lines 858 to 866, we mentioned that, to ensure that a specific intervention in a research study is applied effectively to the population of interest in "real-world" conditions, it is essential to carefully analyze the methods and modules being used based on their characteristics. This methodology can also be applied to evaluate other ICT innovations and assess their functionalities, value, and acceptability levels. To achieve this, a combination of methods such as AHP, BN analysis, regression analysis, and ANOVA test can be used to evaluate the functionalities of future innovations and establish correlations and predictions about their improvement.


(2) Please explain the promotional value of the methods used in this article.

Answer: We are thankful for this comment. In the revised version of the manuscript, in lines 127 to 131, we noted that It is important to note that the methods, modules, and mathematical models utilized in this study, including AHP, Bayesian Network analysis, regression analysis, and ANOVA test, possess the ability to analyze quantitative data, hierarchize factors, and make accurate predictions. As a result, these methodologies are highly valuable and have been implemented in this study to achieve favorable results.


Many thanks for your efforts and your appropriate comments on this review. Your contributions have been invaluable and greatly appreciated.

Warm Regards,