Augmented Reality Applied to Identify Aromatic Herbs Using Mobile Devices

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

1. Clarification of Problem Statement: The authors should clearly articulate the specific problem they aim to address. It is important to provide a detailed explanation of the challenges or gaps in the current methods of identifying aromatic herbs that this research intends to solve.

2. Description of Solution: The manuscript would benefit from a thorough description of the proposed solution. The authors should elaborate on the methodology used, detailing the augmented reality techniques and mobile device functionalities employed to achieve herb identification.

3. Quantitative and Qualitative Data: To enhance the credibility and depth of the research, the authors should include quantitative data or qualitative insights. For instance, they could provide metrics on the accuracy, efficiency, or user satisfaction of their augmented reality system compared to traditional methods.

4. Survey Content and Results: The article mentions the use of questionnaire; however, it would be more informative if the authors included the survey results. Detailed analysis and discussion of these results would provide valuable insights into user feedback and the effectiveness of the application.

Incorporating these suggestions will significantly strengthen the paper by providing a clearer understanding of the research problem, a comprehensive explanation of the solution, and supporting data to validate the findings.

 

Author Response

Comment 1 - Clarification of Problem Statement: The authors should clearly articulate the specific problem they aim to address. It is important to provide a detailed explanation of the challenges or gaps in the current methods of identifying aromatic herbs that this research intends to solve.

 

Response 1: We agree with the suggestions and improved the article by inserting the challenges and gaps in the research; see lines 53 to 83 of the text.

 

 

Comment 2 - Description of Solution: The manuscript would benefit from a thorough description of the proposed solution. The authors should elaborate on the methodology used, detailing the augmented reality techniques and mobile device functionalities employed to achieve herb identification.

 

Response 2: We agree with the suggestions and have included a description of the architecture of the proposed solution, detailing the methods used with augmented reality techniques and the functionalities of the mobile device. We complement the phases of the methodology used in the text. Check between lines 283 to 341.

 

Comment 3 - Quantitative and Qualitative Data: To enhance the credibility and depth of the research, the authors should include quantitative data or qualitative insights. For instance, they could provide metrics on the accuracy, efficiency, or user satisfaction of their augmented reality system compared to traditional methods.

 

Response 3: To select images from the dataset, a scale with five levels was used to rank image quality based on the Vuforia algorithm for applying computer vision, as detailed in Table 1 and between lines 457 and 481.

          For user satisfaction with the application, we detail the form and method used with users with more information. Please check lines 505 to 531.

          To measure the user’s satisfaction with the augmented reality application compared to the current method, a meeting was held with experts at CEAGESP to evaluate, validate, and compare the scenarios.

 

Comment 4 - Survey Content and Results: The article mentions the use of questionnaire; however, it would be more informative if the authors included the survey results. Detailed analysis and discussion of these results would provide valuable insights into user feedback and the effectiveness of the application.

Response 4: To comply with the user satisfaction with the application, we detailed the form and method used with users with more information, checking lines 539 to 565.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Dear author:

       Thank you for offering the opportunity to review your manuscript. In the article, an application program for identifying and classifying aromatic herbs based on Design Science Research (DSR) method is designed. Totally speaking, it is an interesting research with promised application sceneries. The experiment is detailed and logical. However, it is difficult to evaluate the performance of the designed application due to the shortage of the method introduction and result presentation. The author needs to revise the following questions and consider acceptance after a major revision.

 

1.     More experimental results are needed to exhibit the performance of the application.

2.     Instead of the method description of a research paper, section 3 is more like an instruction book. Please be more specific about the classification method.

3.     Though the authors emphasized that their application is based on an augmented reality (AR) method, the effects achieved in the manuscript doesn’t seems to meet the definition of AR.

4.     Minor grammatical errors need to be revised.

5.     Please compare the performance with more existing applications with similar tasks.

Comments on the Quality of English Language

The author's English level is fair, but there are still a few grammatical mistakes.

Author Response

Comment 1 – More experimental results are needed to demonstrate the performance of the application.

Response 1: To demonstrate the performance of the ARomaticLens application, the manuscript provides detailed experimental results and validation carried out with the help of experts from CEAGESP. Below are the main experimental results highlighted in the manuscript:

          Image Collection: The research team collected 600 images of 18 aromatic herbs. These images were taken in various scenarios, including open-air markets, supermarkets, and CEAGESP distribution points.

          Classification Accuracy: Images were processed and classified using Vuforia 10.6 software. Classification accuracy was assessed by assigning star ratings (0 to 5 stars) based on recognition time and accuracy.

          Performance metrics: Recognition time and image star rating.

          The application was tested to ensure it worked effectively without an internet connection, resolving CEAGESP's connectivity issues.

          Four experts from CEAGESP tested the application and provided feedback through a questionnaire. The results of this assessment are as follows:

          Ease of use: All four experts found the app easy to use.

          Data Collection Effectiveness: The app collected data effectively in the field.

           Real-Time Results: The application delivered real-time results as expected.

          Experts rated the usability of the interface 8 out of 10.

          The integration of augmented reality and computer vision in the ARomaticLens application demonstrated 100% accuracy in identifying the 18 types of aromatic herbs studied when associated with the application's local database. Experimental results show the performance of the ARomaticLens application in accurately identifying and classifying aromatic herbs. The combination of augmented reality and computer vision technologies provided a robust solution that met the specific needs of the CEAGESP environment. Expert validation and high usability scores further affirm the app's effectiveness and ease of use.

Comment 2 - Instead of describing the method of a research paper, section 3 looks more like an instruction book. Please be more specific about the classification method.

Response 2: The classification methods in this section have been changed and detailed, and the check lines between 283 and 341 have been checked.

Comment 3 - Although the authors emphasize that their application is based on an augmented reality (AR) method, the effects achieved in the manuscript do not seem to meet the definition of AR.

Response 3: To check whether the effects achieved in the manuscript meet the definition of augmented reality (AR), we compared the declared functionalities and results of the ARomaticLens app with the standard definition of AR.           Augmented Reality is a technology that superimposes computer-generated images, sounds, or other data onto a user's view of the natural world, thereby improving their perception of their environment. Some characteristics of AR can be cited as an overlay of digital information: AR involves the overlay of digital information (such as images, text, or 3D models) in the real-world environment. Real-Time Interaction: The technology must allow real-time interaction with overlapping information. Contextual Awareness: AR systems must recognize real-world context and appropriately align digital information with physical objects.

          Based on the ARomaticLens application, the following features can be observed as Overlay of digital information: The ARomaticLens application uses the smartphone's camera to capture images of aromatic herbs. It then processes these images using computer vision algorithms. It overlays relevant information (such as the herb's name, nutritional properties, and usage methods) onto the real-world view of the herb. Real-time interaction: The app provides real-time herb identification, displaying results instantly or within a few seconds, depending on image quality and recognition time. Users interact with the app by pointing their smartphone cameras at herbs, and the app responds by displaying information about the identified herb in real-time. Contextual Awareness: The app uses Vuforia software to track images in real-time and compare them to a local database of images of known herbs. This ensures that digital information (herb details) is contextually relevant to the physical herb viewed by the user.

          The functionalities and results described in the project involving the ARomaticLens application align well with the standard definition of augmented reality.

Comment 4 – Minor grammatical errors need to be revised.

Response 4: A grammatical review was carried out to remove errors.

Comment 5 - Compare performance with more existing applications with similar tasks.

Response 5: A topic of related work was created that addresses similar existing applications.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The research topic is certainly relevant and has both theoretical and practical significance. However, the submitted manuscript requires significant improvements. The authors do not present at all the methods of segmentation and classification of aromatic herbs that were used. There is also no comparative analysis with existing methods that have been used in similar problems. There is no description of processing and preparation of the dataset. Since we are talking about mobile devices, it is also necessary to present limitations on the use of complex architectures, etc., the speed or size of the models matters.

Author Response

Comment 1: The authors do not present at all the methods of segmentation and classification of aromatic herbs that were used.

Response 1: The methods of segmentation and classification of aromatic herbs used in the manuscript are now thoroughly detailed. We describe the use of various techniques and software to achieve accurate identification and classification of aromatic herbs. Here is a summary of the methods presented:

    Computer Vision (CV) and Augmented Reality (AR): The manuscript uses Vuforia 10.6 software integrated with Unity 2022.3 to develop the ARomaticLens application. This application uses image tracking, pattern recognition, feature tracking, and depth tracking to overlay virtual information on real-world images of herbs.

    Image Database Creation: A comprehensive database of 600 images of 18 aromatic herbs was created. These images were classified and stored in Vuforia 10.6, which categorized them based on shapes and contrasts.

    Image Processing and Recognition: The CV algorithm processes the images captured by the smartphone camera, comparing them with the local database. The application can identify herbs based on visual patterns and geometric features.

    Offline Capability: The application is designed to function offline, ensuring that users can access it without an internet connection. This is achieved by embedding the database and processing capabilities within the app.

    User Interface and Functionality: The application features an intuitive user interface, providing clear instructions and easy navigation. It also includes technical information about each identified herb, such as its scientific name, nutritional properties, and culinary uses.

    Evaluation and Validation: The application was tested and validated by experts at the CEAGESP fair, demonstrating 100% accuracy in identifying the 18 types of aromatic herbs studied.

In conclusion, we have presented detailed methods for segmenting and classifying aromatic herbs using advanced technologies like CV and AR, supported by a robust database and designed to work offline to address specific challenges at CEAGESP.

Comment 2: There is also no comparative analysis with existing methods that have been used in similar problems.

Response 2: We provided some context and mentioned related works that deal with similar problems of identifying aromatic herbs using different methods. See “Related works topic”

Comment 3: There is no description of processing and preparation of the dataset.

Response 3: We included a description of the processing and preparation of the dataset used for the ARomaticLens application. Please find below the relevant details:

1. Image Database Creation: The manuscript explains that a comprehensive image database was created with 600 images of the 18 most sold aromatic herbs at the CEAGESP warehouse. These images were captured in various scenarios to ensure robustness, including open-air markets, supermarkets, and CEAGESP distribution points.
2. Classification and Storage: The images were classified and stored using Vuforia 10.6 software. The classification process involved categorizing the images based on their shapes and contrasts to facilitate accurate identification.
3. Integration with AR and CV: The images were then integrated into the Unity 2022.3 software for use in the ARomaticLens application. Custom C# scripts were developed to manage image processing and user interactions, ensuring that the images could be used effectively within the application.
4. Star Rating System: The images were also subjected to a star rating system based on their recognition accuracy by the computer vision system. Images were rated from 0 to 5 stars, with 5-star images being recognized instantaneously and 0-star images not being processed successfully. This rating system helped optimize the database by selecting high-quality images to ensure accurate and efficient herb identification.
5. Technical Validation: The processed and prepared dataset was validated by experts at CEAGESP through practical tests, confirming the application's accuracy and usability in identifying aromatic herbs.

Comments 4: Since we are talking about mobile devices, it is also necessary to present limitations on the use of complex architectures, etc., the speed or size of the models matters.

Response 4: We added a topic to the text about Limitations on the Use of Complex Architectures in Mobile Devices. Please check between lines 143 to 165 and 603 to 622.

 

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

This article discusses an innovative augmented reality (AR) application called ARomaticLens for recognising and classifying vanilla. The study was divided into five phases through the Design Science Research (DSR) methodology, ranging from investigating the CEAGESP problem situation to validating the application through practical tests and experiential questionnaires conducted by CEAGESP experts. The results of the study showed that when associated with the application's local database, the application achieved 100 per cent accuracy in identifying the 18 vanilla types studied, without the need to use an Internet connection. In addition, users rated the usability of the interface as 8 on a scale of 0 to 10. The application has the advantage of being available offline. The article also describes the design and development process of the application, including the creation of an image library, the creation of an AR application related to computer vision, the provision of technical information, and the conversion of the herbs into condiments.

The paper is logical and the experimental setup is sound, but there are still some problems. Deficiencies and possible directions for improvement of this article include:

1.In subsection 3.1, the specific process of each stage of the study is described in detail for the reader's understanding; the five steps can be visualised by drawing pictures.

2.In subsection 3.1, a brief explanation of the choice of ranges for the variables time and temperature for Phase V should be given.

3.In subsection 4, in line 300, the number 4 in question four should be changed to IV, corresponding to the other questions.

4.In Figure 7 of subsection 4 experts validated this application in the Cagesp site can add validation results to confirm the usability of this procedure.

5. Whether images and captions should be centred in the paper.

Overall, the article could be improved by addressing these shortcomings and providing more comprehensive validation and comparison with existing methods.

Author Response

Comentário 1 - Na subseção 3.1, o processo específico de cada etapa do estudo é descrito em detalhes para compreensão do leitor; as cinco etapas podem ser visualizadas por meio de desenhos.

Resposta 1 – Obrigado pelo seu comentário. Concordamos com as observações e as dividimos em cinco novas subseções de acordo com as etapas detalhadas entre métodos e resultados.

 

Comentário 2 - Na subseção 3.1, deve ser dada uma breve explicação sobre a escolha dos intervalos para as variáveis ​​tempo e temperatura para a Fase V.

Resposta 2 - Concordamos com as observações e prosseguimos com os detalhes do estudo, que podem ser vistos no texto entre as linhas 408 e 418.

 

Comentário 3 - No inciso 4, na linha 300, o número 4 da questão quatro deve ser alterado para IV, correspondendo às demais questões.

Resposta 3 - Concordamos com as observações e reescrevemos esta etapa do resultado como uma redação abrangendo as perguntas e respostas, apresentadas nas linhas 560 e 561.

 

Comentário 4 - Na Figura 7 da subseção 4 os especialistas validaram esta aplicação no site do CEAGESP podem adicionar resultados de validação para confirmar a usabilidade deste procedimento.

Resposta 4 - Concordamos com as observações e inserimos o texto entre as linhas 539 a 565.

 

Comentário 5 - Se imagens e legendas devem ser centralizadas no artigo.

Resposta 5 - Concordamos com as observações e verificamos o modelo fornecido pelo periódico. As imagens devem ser alinhadas à esquerda do início do parágrafo quando sua largura for menor que a largura total do texto.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have adequately addressed the review comments, and the additional content makes the manuscript more complete and persuasive. I believe this paper now meets the standards for publication.

Author Response

Comment 1 - The authors have adequately addressed the review comments, and the additional content makes the manuscript more complete and persuasive. I believe this paper now meets the standards for publication.

 

Response 1: We would like to express our thanks for your comments and suggestions during the review process of our manuscript. Your careful and detailed analysis was fundamental to improving the quality of our work.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

accept

Author Response

Comment 1 - Accept

 

Response 1: We would like to thank you for your comments and suggestions during the review process of our manuscript. Your careful and detailed analysis was fundamental to improving the quality of our work.

Reviewer 3 Report

Comments and Suggestions for Authors

The authors revised the submitted manuscript, expanded certain sections, and added a description of the dataset. However, in general, the comments remain the same, they have not been eliminated.

The work does not describe the methods that were used for segmentation and classification. CV and AR are not methods. There is also no comparative analysis with other existing methods; only a section with an expanded overview of methods has been added.

In its current form, the work, in fact, considers to a greater extent the use of ready-made solutions for creating a mobile application and its testing by an expert group. If the authors wanted to present the mobile application they developed and its capabilities, then it was nevertheless necessary to at least briefly outline the methods of segmentation and classification, and present in detail the development process (software stack, rationale for choosing tools, etc.).

Author Response

Comment 1 - The work does not describe the methods that were used for segmentation and classification. CV and AR are not methods. There is also no comparative analysis with other existing methods; only a section with an expanded overview of methods has been added.

 

Response 1: We agreed with the observations and improved the manuscript by inserting the method used through the DSR framework. Please see lines 260 to 507 of the text.

 

Comment 2 - In its current form, the work, in fact, considers to a greater extent the use of ready-made solutions for creating a mobile application and its testing by an expert group. If the authors wanted to present the mobile application they developed and its capabilities, then it was nevertheless necessary to at least briefly outline the methods of segmentation and classification, and present in detail the development process (software stack, rationale for choosing tools, etc.).

 

Response 2: We agree with the observations and improve the article, inserting a brief outline of the segmentation and classification methods between lines 417 to 464.

The development details involving justification in the choice of software used can be seen between lines 388 to 415.

Author Response File: Author Response.pdf