Analyzing Performance Effects of Neural Networks Applied to Lane Recognition under Various Environmental Driving Conditions
, Sousso Kelouwani 1,*, Muhammad Zeshan Alam 1, Alvaro Uribe-Quevedo 3, Ali Amamou 1, Patricia Paderewski-Rodriguez 2 and Francisco Gutierrez-Vela 2
Round 1
Reviewer 1 Report
Lane detection is important for autonomous vehicles. This paper compare the performance of six varied and commonly used network architectures in lane detection. However, there are some problems that need to be discussed:
1. It should be based on the convergence of the model for comparing the performance ability in different scenarios. But it can not guarantee that each model has converged when there are only 50 iterations. Therefore, the effectiveness of the comparative experimental results is insufficient.
2. As for mini batch, GPU can give better performance to batch with power of 2. It often performs better when it is set to 16, 32, 64, 128. But 10, 45 and 27 are selected in the paper. The author should give an explanation of this choice.
3. In Table 2, the effect of 50 iterations is significantly reduced compared with 30 iterations under the same network and the same hyperparameters. But, why?
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
1、 3.1.3 is the image size changed by clipping or down sampling? (whether too many details will be lost by downsampling, resulting in the decline of network accuracy, which will affect the subsequent network comparison); The processing description in the second paragraph is not easy to understand. It is suggested to attach a diagram for explanation.
2、 It is suggested to add network structure diagram to 3.3.2-3.3.5
3、 4.2.1-4.2.5. It is better to add GT chart in the analysis part. For the comparison of extreme weather, it is difficult to visually see the difference without GT value
4、 Table 2 why there is no data for vgg16 and nasnet?
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
This paper creates a benchmark and analyzes the performance of widely used neural networks on lane detection under various driving conditions. While this paper is not proposing a new method, the reproduction study and the detailed comparison are valuable and of interest to readers. In addition, the article is well-written and presents the results in an easy-to-digest manner.
A couple of suggestions:
1. While the authors have already explained the accuracy and the RMSE results, it would be helpful to further put these numbers into the larger context of autonomous navigation. What does the accuracy drop mean for AV planning in various driving conditions? Does continuous recognition on a stream of video increase tolerance?
2. More details about how the fine-tuning is done in the experimental setup are worth adding. E.g. Are the last few layers fine-tuned separately from the previous layers? Is there any learning rate decay applied for individual layers?
3. Some more high-level discussion into potentially how to recover some of the accuracies drops in difficult environments can be helpful. E.g. Will other types of sensors in addition to 2D cameras be required? Will geometric-based methods help? How about ensemble models?
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
