Application of the DEA Method for Evaluation of Information Usefulness Efficiency on Websites

Round 1

Reviewer 1 Report

1.          The topic of this paper is interesting and practical; however, the structure of the paper needs great improvements.  It is confusing as to the purpose of the research and how it relates to the background of the study. The materials should contain data collection and methods for data analysis. The results, implication, and contributions of the paper should be included in Introduction.

2.          There are many long paragraphs in Introduction but the link between the concepts is not well explained. It is not clear how these concepts lead to the research problem, purpose of the research, and how this paper fills the research gap.

3.          The literature review session should be between Introduction and Method.

4.          The paper needs to follow the common forms

l   Introduction: background, research gap, research purpose, method, results, contribution, and structure of this paper.

l   Literature review: missing. Need to compare this study with prior studies and explain similarities and differences.

l   Materials and methods: data collection, and method for data analysis.  The authors did not explain how the research data is selected. What is being analyzed should be in the Materials and Method section.

l   Data collection method should be included in the Materials section, not the Result section.

l   The authors should explain the key questions on the survey and how the questions were drawn from the literature.

l   Results and discussions seem to be mixed in this paper.  The results and discussions should be stated separately.

l   Results should contain only the results.

l   The discussion should contain managerial implications, and how benefit from the results of the study.

l   Conclusion: should be brief and include research purpose, method, results, discussion, and conclusion.  The conclusions did not clearly show these elements.

 

5.          Most of the references should be more up to date, for example, within the last 5 to 10 years. 

Comments for author File: Comments.pdf

Author Response

Dear Reviewer,

We would like to thank you very much for your review, opinions and all of the remarks.

The following changes have been made to the article in accordance with the comments:

1) modification of the substantive content structure as part of the entire article,
2) improving the formatting of the content as part of the entire article,
3) adding a description and literature on sustainability in the context of conducted research (initial part of the content under section: “1. Introduction”),
4) adding a description on research gap in the context of conducted research (final part of the content under section: “1. Introduction”),
5) adding additional items to the literature review (middle part of the content under section: “1. Introduction”),
6) adding a summary description of the entire content of the article (final part of the content under section: “1. Introduction”),
7) adding a description of the research procedure and adapting the entire structure of the article to this procedure (middle part of the content under section: “2. Materials and Methods”),
8) adding a description of the two scientific methods used in the study (middle part of the content under section: “2. Materials and Methods”),
9) adding thematic subtitles as part of the description and presentation of results as part of the entire section: “3. Results”,
10) removing section “4. Discussion” and moving the contained content and analysis results to section “3. Results” and section “4. Conclusions”,
11) adding additional items in the section “References” (initial and middle part of the content under section: “References”),
12) supplementing several literature items in the section “References” with newer editions,
13) adding one acknowledgment item in the section “5. Patents".

 

Best regards
Michał Nowakowski and Dariusz Porębski

 

Reviewer 2 Report

The authors use a complex and neat statistics to represent the subject of the research. The authors demonstrated the use of DEA method for a practical data analysis, a performing calculation for creating the ranking of the analysed websites and criteria groups, by presenting data interpretation in terms of information usefulness and did efficiency evaluation as well.

I suggest this article for the publication.

Author Response

Dear Reviewer,

 

We would like to thank you very much for your review, opinions and all of the remarks.

 

Best regards
Michał Nowakowski and Dariusz Porębski

 

Reviewer 3 Report

 Name of the Paper:  Application of the DEA method for evaluation of information usefulness efficiency on websites

The paper uses data envelopment analysis (DEA) in evaluating the information usefulness and efficiency of various websites. It provides various analyses and subsequent evaluations of the provided website information usefulness efficiency for websites of secondary and high education and shopping websites. It also provides basic methodological assumptions, the research evaluation procedure of information usefulness efficiency, forms of informational content, and various forms of information presentation and visualization. The respondent’s data have been analyzed to determine the factors that have the greatest impact on the perception of information used on websites by users.

General Observation:

1)      The beginning of each paragraph looks intentionally highlighted, which may be changed to suit the journal format.

2)      Literature review skips many website-related articles, for instance:

ü  Li, Z., Li, J. and Li, X., 2009, September. A study on website operation efficiency evaluation: Based on DEA model. In 2009 International Conference on Management and Service Science (pp. 1-6). IEEE.

ü  Muhammad, A., Siddique, A., Naveed, Q.N., Khaliq, U., Aseere, A.M., Hasan, M.A., Qureshi, M.R.N. and Shahzad, B., 2021. Evaluating usability of academic websites through a fuzzy analytical hierarchical process. Sustainability, 13(4), p.2040.

ü  Liu, Z., 2012, December. Diagnosing E-commerce website quality based on DEA. In Proceedings of 2012 2nd International Conference on Computer Science and Network Technology (pp. 762-765). IEEE.

3)      To use DEA, the evaluated websites must have the following characteristics (Li et al., 2009):

(i) have similar objects and assignments;

(ii) have a similar outer-environment;

(iii) have the same input and output index.

The author may provide more clarity on how they dealt with these constraints.

Li, Z., Li, J. and Li, X., 2009, September. A study on website operation efficiency evaluation: Based on DEA model. In 2009 International Conference on Management and Service Science (pp. 1-6). IEEE.

4)      The author may provide a research framework and table explaining input and output variables/groups used to provide more clarity on the research process.

5)      Please refer to “Symbols of the main criteria (MC-314 01-O, … , MC-16-O and MC-01-I, … , MC-15-I) and symbols used in Table 1’ are not the same.

6)      Please refer to “Group III. Knowledge increase aspects in the content of the article. Authors need to justify the role of websites in increasing knowledge content in the group -III. The “knowledge content” depends on the quality of the article and not on the website.

7)      Please refer to “For this purpose, eight groups of criteria, related to different aspects of information usefulness, were built” the basis of the selection of sub-criteria is unclear reference [61--63] are not in English.

8)      Some of the terms used in Group I-VIII are difficult to interpret how the accuracy of the feedback to the questionnaire was ensured.

9)      The article is based on the data collected using a questionnaire survey, but the information regarding the questionnaire, its accuracy, validity, response group, sample, population is missing.

 

10)  Figure 1 and Figure 2 are not helping readers to understand. Hence, Group I to Group VIII may be suitably renamed to represent the group characteristics.

Author Response

Dear Reviewer,   In response to the detailed comments of the review, I would like to additionally highlight the following aspects related to the research and the article:   1) the highlights of each paragraph have been removed, 2) the selected articles are very interesting items, which I will want to read in the future, 3) selected websites had the same subject, similar type of website, similar scope of content, similar functionality, similar external environment and similar input and output indexes, 4) input and output variables / groups largely coincided with Table 1, 5) the symbols of the main criteria in Table 1 appear repeatedly and in different order, which may give the impression of errors, 6) all of the main criteria in “Group III. Knowledge increase aspects in the content of the article” refer to the quality of the content and information, not the quality of the website, 7) other models of effectiveness evaluation from the literature review used a wide range of criteria, from a few to even a dozen, which was to justify the selection of 8 criteria (groups of main criteria), 8) the survey form was written in a similar, but easier language, and also had additional explanations of the most important concepts and issues, 9) more detailed information on the survey can be found at the beginning of the section “2. Materials and Methods”, 10) this is a very interesting point that I will try to keep in mind in the future.   Best regards Michał Nowakowski

Reviewer 4 Report

Abstract

▪ Make spelling and grammar corrections in the text; standardize the spelling of the purpose of the text in the abstract and introduction; indicate the analytically based theory that supported the research.

 

Introduction

 The introduction lacks an argument showing clearly and convincingly the importance of the chosen topic, the need to study it and the positioning of the article in the field of knowledge. These tasks require a synthetic review of the literature in support of a compelling argument that your article meets a research need and will make a contribution to the field of knowledge in question.

 Make it clear what type of article this is. Is it empirical research? Is it a reflection article? Is it a point of view? Is it a case study? Because this lack of initial clarity can leave a little open that the work will walk along a path that, in the end, is not verified by being executed in a different way. It is suggested to make as clear as possible, from the beginning, the execution strategy to be adopted.

 

Theory

To what extent does the research problem contain and reflect categories/constructs/variables that come from the tourism-specific literature review itself and how this connects with problem solving of an explicit/implicit agenda of issues in the subarea.

I have a paper to indicate, although not in English, that can help in website evaluation: http://geplat.com/rtep/index.php/tourism/article/view/635

▪ Include a summary table (replicate the same category table that appears and guides the analysis), at the end of this section, highlighting the main theoretical categories of the study and its theoretical basis.

 

Methodology

Also on the summary table, from the discussion carried out, indicate the theory or author defined to support the analysis.

▪ Which categories / constructs / variables were analyzed? Why these variables? Where did they come from/were they taken from? How do they connect with theory?

▪ What are the data collection, processing and analysis techniques?

▪ What and how are the limitations of the study?

 

Analysis - Results

▪ Style recommendation – structure the analysis by periods and standardize the opening paragraph and each one of them, as in some it mentions the distribution of resources and in others it does not.

▪ Describe data tabulation, indicates the use of descriptive analysis. However, which theory/author supported the analyses?

▪ Reorganize and deepen a little the logic of the discussion in the analysis, producing greater tying/systematization between categories, the theory, and the specific theory of tourism, because as it is, the analysis seems to be based on connections in the form of free association, without leaving it tied and explicit, what categories (indicate which categories – topics of discussion – were used to structure the analysis, as they seem to be only implicit), data and analysis techniques were used.

 

Conclusion

▪ Based on the discussion carried out, point out the progress/differentiation of the work in relation to the existing ones on the subject.

▪ More robust conclusions, supported by the confrontation of the results with the elements of the theoretical foundation section and that explain relevant contributions to the field of knowledge, as well as recommendations for future studies.

Author Response

Dear Reviewer,   In response to the detailed comments of the review, I would like to additionally highlight the following aspects related to the research and the article:   1) I tried to show the importance and relevance of the chosen research problem by analyzing the literature, but maybe it will be better next time, 2) I wanted to produce an empirical article; the problem with the appropriate selection and approach to research literature probably resulted from the lack of similarity to other studies and the selection of an appropriate center of gravity in relation to what is and is not a similar area in relation to mine, 3) the list of analyzed variables largely coincides with Table 1, 4) a spreadsheet tool was used to collect, process and analyze data, 5) I don't know the limitations of the study, it depends on the context, 6) this is a very interesting points that I will try to keep in mind in the future.   Best regards Michał Nowakowski

Round 2

Reviewer 1 Report

The authors revised the paper satisfactorily. 

Author Response

Dear Reviewer,

We would like to thank you very much again for your satisfactory review.

In conclusion, we decided not to make additional corrections to the article.

Best regards
Michał Nowakowski and Dariusz Porębski

Reviewer 3 Report

Except for the first suggestion, the authors have neither modified the manuscript on the suggested comments for the improvement of the manuscript nor suitably defended them.

Author Response

Dear Reviewer,

Thank you very much again for a possibly satisfactory review.

Due to the lack of more detailed comments on the improvement of the linguistic aspects, we decided to stick to the current language version of the article.

In conclusion, we decided not to make additional corrections to the article.

Best regards
Michał Nowakowski and Dariusz Porębski

Reviewer 4 Report

Congrats

Author Response

Dear Reviewer,

Thank you very much again for a possibly satisfactory review.

Due to the lack of more detailed comments on the improvement of the linguistic aspects, we decided to stick to the current language version of the article.

In conclusion, we decided not to make additional corrections to the article.

Best regards
Michał Nowakowski and Dariusz Porębski