The Development of an Autonomous Vehicle Training and Verification System for the Purpose of Teaching Experiments

Round 1

Reviewer 1 Report

The manuscript presents an experimental test for teaching purposes. The proposal is interesting, however, you do not indicate, beside the title, anything how it will impact within educational institutions. I recommend you to:

- Detail how these experiments are related to teaching purposes

- The elements do not look "low cost", please, compare it with standard elements.

- There are not step-by-step procedures to be replicated for teaching experiments, give more details for a real reproducibility.

- How is the percentage of throttle controlled by the user? 

Author Response

- Detail how these experiments are related to teaching purposes

Thank you for your questions. In the revised manuscript, we have included a comprehensive flowchart that illustrates the sequential steps involved in implementing the course. Within this system, students have the opportunity to engage in hands-on activities such as assembling the hardware components of the autonomous vehicle, configuring the software, and selecting or modifying the neural network model during class sessions. Subsequently, they can gather the necessary data for model training purposes. Lastly, the system's testing and scoring mechanism enables students to assess and validate the performance of the autonomous vehicle.

The primary objective of these experiments is to familiarize students with the complete process of implementing the course, while also enabling them to learn from the outcomes of their experiments. By engaging in vehicle assembly activities, students can avoid repeating prior design errors and gain a deeper understanding of the correct concepts. This experiential learning approach empowers students to develop more sophisticated neural network models for autonomous vehicles in the future.

- The elements do not look "low cost", please, compare it with standard elements.

Thank you for your questions. In the revised manuscript, Section 3.4 presents a comparison in Table 4 among various platforms, including AWS DeepRacer, Udacity's Autonomous Car Simulator, CARLA Simulator, Donkey Car, and the vehicles designed in this study. 

The vehicle design in this study is relatively cheaper compared to other platforms, allowing students to have their own autonomous car for implementation in group projects. Although the hardware for the test and scoring system is more expensive, it can be shared during the course for cost efficiency. In addition, the testing track for the vehicles in this study can be customized by combining output posters, allowing for flexibility in adapting to different classroom sizes. Furthermore, the unique design of the auto scoring and verification system in this paper allows for more concrete and efficient evaluation of the performance of the trained autonomous vehicles.

- There are not step-by-step procedures to be replicated for teaching experiments, give more details for a real reproducibility.

Thank you for your questions. In the revised manuscript, a Figure 13 has been added in Section 2.5 to depict the flow chart of the course execution process. In this course, students learned how to assemble an autonomous car, install and configure the necessary software, and perform a series of tests to ensure that the car is functioning properly. 

By the end of the process, students learned how to assemble and configure an autonomous car, collect training data, and train and test a neural network model for autonomous driving.

- How is the percentage of throttle controlled by the user? 

Thank you for your questions. In the revised manuscript, a Figure 13 has been added in Section 2.5 to depict the flow chart of the course execution process. In step 4 of the flow chart, detailed instructions are provided on how to set the throttle value of the vehicle and remotely control its movement.

Users utilized two different methods provided by the system to operate the vehicle and collect training data. The first method entailed manually driving the vehicle and collecting data by inputting the command "python manage.py drive". Before commencing, they could access a menu by entering "http://vehicle-IP:8887" in a web browser on a computer or mobile device, allowing them to set the value for "Max Throttle" and select "User(d)" in the "Mode & Pilot" menu to configure the parameters for remote control of the vehicle.

The second method involved using an Xbox racing wheel and pedals to control the vehicle via Bluetooth by entering the command "python manage.py drive --js". The vehicle's direction could be controlled using the steering wheel, while the throttle could be adjusted using the pedals to increase or decrease the throttle value.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper is not well-written, but the considered topic is interesting.  

    The main weaknesses of the paper are:

    (1)The readability of the paper must be improved. To this end, the authors must improve the "flow" of the ideas presented in the whole paper. The paper must become easy to read and understood at first reading.

   (2) More technical analysis is required in Section 3. Experiment and Result. For example, the authors should explain/argue why they selected (for testing) vehicles driving with 35% throttle with different steering mechanisms on 320 other tracks for 10 laps (Table 1) .

    (3)     The English language must be corrected.

 

 

 

Author Response

(1) The readability of the paper must be improved. To this end, the authors must improve the "flow" of the ideas presented in the whole paper. The paper must become easy to read and understood at first reading.

Thank you for your questions. In the revised manuscript, modifications have been made in accordance with your suggestions, including adjustments to the overall structure of the paper. Additionally, a detailed explanation of the course execution process has been added in Section 2.5. Furthermore, the experimental methods and conditions have been clarified in Section3.2 Lastly, a comparison table has been included to compare the different methods used in Section 3.4.

(2) More technical analysis is required in Section 3. Experiment and Result. For example, the authors should explain/argue why they selected (for testing) vehicles driving with 35% throttle with different steering mechanisms on 320 other tracks for 10 laps (Table 1) .
Thank you for your questions. In the revised manuscript, the Section 3.2, the experimental conditions were described in detail. During the training phase, the car was operated by personnel, who faced challenges in accurately operating the car with a throttle value set greater than 50% on the short and varied track. Conversely, setting the throttle value to less than 20% tended to result in the car getting stuck at S-curve turns. As a result, for this particular experiment, the throttle value was set between 20% and 50%, with testing conducted using a mid-range value of 35%. The testing procedure involved running autonomous vehicles A and B on both Track A and Track B for 10 laps each, and recording the results.

 (3) The English language must be corrected.

Thank you for your questions. The English portion of the manuscript has been thoroughly reviewed and revised by a professional to ensure accuracy and clarity in the revised manuscript.

Author Response File: Author Response.pdf

Reviewer 3 Report

The authors of this paper proposed an autonomous driving training and verification system for teaching experiments, which can reduce the cost of constructing an autonomous vehicle training and test environment and conducting initial training and verification of autonomous vehicle technologies on campus. My review is as under: -

1.       A generic diagram might be placed in the introduction section for improved reader understanding.

2.       Although an excellent literature review has been presented, it would be great to add a comparison table.

3.       The design technique is lacking; Please include any of the following diagrams: block diagram, flow chart, data flow diagram, and so on.

4.       Simulation detail is missing, kindly include with reference.

5.       Please include more recent references.

Author Response

1. A generic diagram might be placed in the introduction section for improved reader understanding.

Thank you for your questions. In the revised manuscript, we have included a comprehensive flowchart that illustrates the sequential steps involved in implementing the course. Within this system, students have the opportunity to engage in hands-on activities such as assembling the hardware components of the autonomous vehicle, configuring the software, and selecting or modifying the neural network model during class sessions. Subsequently, they can gather the necessary data for model training purposes. Lastly, the system's testing and scoring mechanism enables students to assess and validate the performance of the autonomous vehicle.

The primary objective of these experiments is to familiarize students with the complete process of implementing the course, while also enabling them to learn from the outcomes of their experiments. By engaging in vehicle assembly activities, students can avoid repeating prior design errors and gain a deeper understanding of the correct concepts. This experiential learning approach empowers students to develop more sophisticated neural network models for autonomous vehicles in the future. The English portion of the manuscript has been reviewed and revised by a professional for accuracy and clarity.

2. Although an excellent literature review has been presented, it would be great to add a comparison table.

Thank you for your questions. In the revised manuscript, Section 3.4 presents a comparison in Table 4 among various platforms, including AWS DeepRacer, Udacity's Autonomous Car Simulator, CARLA Simulator, Donkey Car, and the vehicles designed in this study. The vehicle design in this study is relatively cheaper compared to other platforms, allowing students to have their own autonomous car for implementation in group projects. Although the hardware for the test and scoring system is more expensive, it can be shared during the course for cost efficiency. In addition, the testing track for the vehicles in this study can be customized by combining output posters, allowing for flexibility in adapting to different classroom sizes. Furthermore, the unique design of the auto scoring and verification system in this paper allows for more concrete and efficient evaluation of the performance of the trained autonomous vehicles.

3. The design technique is lacking; Please include any of the following diagrams: block diagram, flow chart, data flow diagram, and so on.

Thank you for your questions. In the revised manuscript, Figure 1 has been updated to include detailed descriptions of each system block, along with explanations of the corresponding software environment setup and communication methods. Additionally, a process flowchart outlining the implementation process of the course has been added in Section 2.5, providing comprehensive insights into the course implementation process.

4. Simulation detail is missing, kindly include with reference.

Thank you for your questions. In the revised manuscript, the reference to "Udacity's Autonomous Car. Available online: https://www.udacity.com/course/autonomous-car-engineer-nanodegree--nd0013" has been updated in the revised manuscript to "https://github.com/udacity/self-driving-car-sim" as the previous URL is no longer available.

5. Please include more recent references.

Thank you for your questions. In the revised manuscript, a new reference from 2022 by Terapaptommakol et al. [15] has been added, which proposes the use of a deep Q-network method in the CARLA simulator to develop an autonomous vehicle control system that achieves trajectory design and collision avoidance with obstacles on the road in a virtual environment. Additionally, another new reference from 2022 by Gutiérrez-Moreno et al. [16] has been included, which presents an approach for intersection handling in autonomous driving. This approach utilizes a deep reinforcement learning approach with curriculum learning and evaluates the effectiveness of the Proximal Policy Optimization algorithm in inferring desired behavior based on the behavior of adversarial vehicles in the CARLA simulator.

Author Response File: Author Response.docx