Brake Light Detection Algorithm for Predictive Braking

Round 1

Reviewer 1 Report

In the version I reviewed the references are missing. Anyway, the main weakness of the paper lies on the lack of comparison with other methods in the literature. Such a comparison will certainly improve the quality of the paper, while highlighting its original contribution.

Author Response

Concern 1: 

In the version I reviewed the references are missing. Anyway, the main weakness of the paper lies on the lack of comparison with other methods in the literature. Such a comparison will certainly improve the quality of the paper, while highlighting its original contribution. 

 

Response:  

Thank you for your feedback. We do acknowledge and agree that by comparing our method with other methods it would certainly improve the quality. However, while we were researching the state of the art on brake light detection we contacted multiple different researchers about their work, with no response. This would then mean that we should advise the comparison algorithms from scratch and compare it with different dataset than it was done the first time. This would be extremely time consuming and since many of the other methods were hardware sensitive it would make our comparison results ambiguous.  

 

Modification: 

We clarified the explanation about table 6 found in results chapter with the following: 

Line:321 

 

“Direct comparison of the presented algorithm to other state-of-the-art algorithms would be beneficial, yet unfortunately the source codes of relevant algorithms are not available. However, Table 6 presents main operation conditions and key parameters of each study respectfully. Results are presented from each study and compared to the result achieved in this study. Unambiguous comparison of the accuracies is difficult as each study has utilized different datasets, specific hardware, different weather conditions and distances to the vehicles in front.” 

 

When advising our research we applied as transparent and straighgtforward approach by providing all the data and program code for open source usage. In future we aim to continue to do so, and hope that our work will be compared against other algorithms and in optimal case other research work will enchance our dataset and methods. 

Reviewer 2 Report

The research aims to present a detection brake lights algorithm to predict braking of moving vehicules. The study has taken about two years as is it can be deduced from the authors mention in the paper "The recording took place in real traffic 165 conditions in Helsinki metropolitan area during February and March 2020". I think this a very interesting research, it presents new results that will be useful for road safety and can help reduce road accidents. Simulation and tests were conducted in differents environment, testing results were also provided respecting the scientific approach.

However I have some questions to authors, to understand and to help theim improve this system if possible:

--Tests are done in Helsinki metropolitan area, Helsinki is caracterised by its cold weather in general and little sun, so what about algorithm efficiency in sunny areas like african and middle east countries (Saoudi Arabia, Israël, Algeria...)?

--Tests are also done in straight road, in such roads the vehicule is in front of the camera directly, so brake lights are in the field of view of the camera. What about tests in roads with sharp bends?

Author Response

Concern 1: 

Tests are done in Helsinki metropolitan area, Helsinki is caracterised by its cold weather in general and little sun, so what about algorithm efficiency in sunny areas like african and middle east countries (Saoudi Arabia, Israël, Algeria...)? 

 

Response: 

The dataset includes day time driving, including some sunny photos. In Finland sun shines ussually from quite low angles, this actually creates really demanding situations for machine vision. It is true that sun affect to most of the machine vision based solutions, when taken into account sun glares and direct sunlight. In final implementation in production vehicle the system would be fused with radar sensor data.  

 

Modification: 

We clarified the manuscipts discussion section with the following: 

Line:385 

 

“While conducting the experimental tests, it was noted that the algorithm suffered from some issues common to most machine vision algorithms, such as glares and reflections caused by sunlight. Future tests could include more sunny data with sun being at different heights. The angle of the sun affects the glares and reflections captured by the camera, thus it could affect the accuracy of the algorithm.” 

 

Concern 2: 

Tests are also done in straight road, in such roads the vehicule is in front of the camera directly, so brake lights are in the field of view of the camera. What about tests in roads with sharp bends? 

 

Response: 

Excellent point. Road conditions such as sharp bends do have an effect to the system. Camera and radar based systems both do have the same problems when it comes to mentioned conditions. Future studies could be done to investigate this problem, one solution could be to feedback steering wheel angle to the algorithm to move the region of interest to compensate bends. This type of method would improve the characteristics of the system but would not work with really sharp bends. 

 

Modification: 

We modified the manuscript by adding the following section to discussion section: 

Line: 403 

 

“The conducted experiments were measured among real traffic, on relatively straight roads. Sharp turns or steep inclinations were not featured in the gathered dataset. In all of these cases the detection can be affected since the vehicle ahead may move out of the region of interest in the camera view. In future research these types of scenarios could be recognized utilizing sensor data of the steering angle or the vehicle accelerometer. This type of approach would solve such cases to some extent.” 

Reviewer 3 Report

Interesting paper. Proposed by the Authors system uses a camera and YOLOv3 object detector to detect the bounding boxes of the vehicles ahead of the ego vehicle. The bounding boxes are preprocessed with L*a*b colorspace thresholding. Thereafter, the bounding boxes are resized to a 30x30 pixel resolution and fed into a random forest algorithm. The novel detection system was evaluated using a dataset collected in the Helsinki metropolitan area in varying conditions. Presented in the paper experiments revealed that the new algorithm reaches a high accuracy of 81.8 %. For comparison, using the random forest algorithm alone produced an accuracy of 73.4 %, thus proving the value of the preprocessing stage. Furthermore, a range test was conducted by the Authors. Then, they found that with a suitable camera, the algorithm can reliably detect lit brake lights even up to a distance of 150 meters.  In my opinion, the paper can be published, after taking into account the following remarks:

• in the keyword section, the keyword "transportation" should be added (because this paper was submitted into section "Transportation and Future Mobility"),
• there is a lack of a Reference section, as well as in the paper text there is a mistake with references, like [???],
• the text should be changed to impersonal form (not "we have presented 'like e.g. in line 40 but " presented "," done "," analyzed ", etc.),
• the Authors refer to the concept of "driving comfort level" in the article. In the Introduction section, the Authors described that the level of driving comfort is influenced by Advanced driver-assistance systems (ADAS), such as adaptive cruise control (ACC) and collision avoidance (CA), etc. It is true and very good that the Authors wrote about it. However, the Authors did not describe that the level of driving comfort is also influenced by many other factors, first of all the way roads are designed and individual geometric solutions, the use of solutions for the segregation of road users, which is just as important as the elements of vehicle equipment increasing the driver's comfort. Authors should describe this fact referring to the latest literature in this regard, e.g. in the field of design and use of safe road intersections such as roundabouts "Roundabout entry capacity calculation-A case study based on roundabouts in Tokyo, Japan, and Tokyo surroundings", DOI 10.3390 / su12041533; whether crossing with traffic lights "A back-of-queue model of a signal-controlled intersection approach developed based on analysis of vehicle driver behavior" DOI 10.3390 / en14041204 or other solutions. One paragraph in the Introduction section will be enough,
• at the end of the Introduction section, the Authors should shortly write what was the aim of the paper as well as what was contained in each paper section,
• lines from 149: The Authors at the end of the Introduction section should clearly state what the purpose of the article is. There is no need to repeat the purpose of the article in the "State of the Art" section. The same is in the next section (line 162) and others similar,
• how look like the weather condition during the measurements? It should be described,
• in the Conclusion section, the Authors should also describe the future plans/research/implementation of the presented solution.

Author Response

Concern 1: 

in the keyword section, the keyword "transportation" should be added (because this paper was submitted into section "Transportation and Future Mobility"), 

 

Modification: 

 We modified the manuscript keyword section with the following 

Line: 14  

 

“Advanced cruise control; Collision avoidance; Machine vision; Machine learning; Transportation” 

 

Concern 2: 

there is a lack of a Reference section, as well as in the paper text there is a mistake with references, like [???], 

 

Response: 

Refrence section missing from the automatically created pdf in the portal. The original Latex file includes all the references normally. We will add one additional pdf to make sure all the references are easily accessible. 

 

Concern 2: 

the text should be changed to impersonal form (not "we have presented 'like e.g. in line 40 but " presented "," done "," analyzed ", etc.), 

 

Modification: 

Text changed to impersonal form 

 

Concern 3: 

the Authors refer to the concept of "driving comfort level" in the article. In the Introduction section, the Authors described that the level of driving comfort is influenced by Advanced driver-assistance systems (ADAS), such as adaptive cruise control (ACC) and collision avoidance (CA), etc. It is true and very good that the Authors wrote about it. However, the Authors did not describe that the level of driving comfort is also influenced by many other factors, first of all the way roads are designed and individual geometric solutions, the use of solutions for the segregation of road users, which is just as important as the elements of vehicle equipment increasing the driver's comfort. Authors should describe this fact referring to the latest literature in this regard, e.g. in the field of design and use of safe road intersections such as roundabouts "Roundabout entry capacity calculation-A case study based on roundabouts in Tokyo, Japan, and Tokyo surroundings", DOI 10.3390 / su12041533; whether crossing with traffic lights "A back-of-queue model of a signal-controlled intersection approach developed based on analysis of vehicle driver behavior" DOI 10.3390 / en14041204 or other solutions. One paragraph in the Introduction section will be enough, 

 

Response: Good point. We modified the manuscripts Introduction section as follows: 

 

Modification: 

Line: 33 

 

“The cruise controller of a vehicle could act more robustly and with more reaction time if the system could perceive braking or stopped vehicles from afar.  

By utilizing machine vision, in this case a brake light status indicator, the control system could use the intention of the preceding vehicle as one input. When approaching a vehicle that is at a standstill, the decision of when to start braking in order to come to a full stop needs to be as robust as possible. More information about stopped and slow-moving vehicles could be used in intersection areas in which a large portion of fatal vehicle accidents are known to occur [1,2]. In addition to in-vehicle systems, careful consideration in traffic infrastructure and road design play an important role in overall traffic flow and driving comfort, especially in intersection areas [3,4]” 

 

Concern 4: 

at the end of the Introduction section, the Authors should shortly write what was the aim of the paper as well as what was contained in each paper section, 

 

Response:We modified and clarified the manuscript Introduction section with the following (modifications made in the next remark strongly linked to this one aswell): 

 

Line: 57 

 

“This article is structured as follows. Next, a state-of-the-art review is given of the most related scientific research. Then, the utilized dataset, and the constructed hardware and software are described in the methods section, which is followed by a description of the experiments. Experimental results are demonstrated which highlight the accuracy of the proposed brake light detector on the gathered dataset. Finally, discussion regarding the impact of the acquired results is presented.” 

 

Concern 5: 

lines from 149: The Authors at the end of the Introduction section should clearly state what the purpose of the article is. There is no need to repeat the purpose of the article in the "State of the Art" section. The same is in the next section (line 162) and others similar, 

 

Response: Excellent point. Repetision removed and the Introduction and State of the art sections clarified 

Line: 43 

 

“ In this paper, a novel brake light detection algorithm is presented. The presented method is designed to help a vehicle  detect decelerating vehicles in front earlier than by relying on a radar system alone. The method is mainly intended for use in ACC systems, but can also be applied in other driver assistance systems. The concept of Brake Light Detection Algorithm for Predictive Braking is illustrated in Figure 1. A camera is installed in the top center of the windshield inside the research vehicle. An existing machine vision algorithm YOLOv3 [5] is applied to detect any vehicles straight ahead, however this could be replaced with any vehicle detector. Then, a novel algorithm is developed to classify the brake light status of the vehicles in front. In addition to the color information used to detect the lights, the final brake light status is classified using a random forest algorithm [6]. A clear triangle shape identification can be seen in the weights learned by the algorithm. This highlights the transparency of the implemented machine learning model. The brake light status is then used for anticipatory braking for passenger comfort, which will be implemented in a separate study.” 

 

Line: 155 

“Despite earlier research in the field of brake light detection, not a single available open source implementation exists. Moreover, previous daytime studies have focused on detecting brake lights from clear pictures captured of vehicles close to the research vehicle, in high traffic situations, mostly at a standstill. The detection method presented here is designed to detect the brake lights of moving vehicles at distances ranging from 5 m to 150 m. The approach is benchmarked, and proven to function in a multitude of scenarios, accurately performing at far distances. The presented approach is designed to be used in all daytime weather conditions, yet it too has its drawbacks common to camera-based systems, such as lens flares caused by direct sunlight. The proposed algorithm and the gathered dataset will be published in an online repository to facilitate further research on the topic.” 

 

Concern 6: 

how look like the weather condition during the measurements? It should be described, 

 

Response: Good point. We clarified the Methods section accordingly: 

 

Line: 167 

 

“A self-gathered dataset was used to test and validate the brake light detection algorithm. Detailed specification of the videos used for constructing the dataset can be seen in Table 1. The equipment used to gather the dataset is introduced in detail in next section. The dataset was recorded during multiple days at different times, thus including variations in weather conditions such as rain, clear sky, cloudy sky and indirect sunlight. The recording took place in real traffic conditions in Helsinki metropolitan area during February and March 2020, and the real traffic conditions made it possible to collect data from different distances. The dataset includes image material of vehicle rears with brake lights on or off. The recorded dataset captured during this study will be publicly available after the publication. Therefore, this dataset can be used for algorithm training and testing operations in the future.” 

 

Concern 7: 

in the Conclusion section, the Authors should also describe the future plans/research/implementation of the presented solution. 

 

Response: We clarified the manuscripts conclusion section with the following: 

Line: 422 

 

“Future research will include utilizing the communication interface of the research vehicle for tuning the adaptive cruise controller with the output of a real-time implementation of the brake light detection algorithm. The goal is to improve the ACC performance in real world field tests by improving the longitudinal control and more accurately anticipating the preceding vehicle intentions than the current radar-based system installed in the research vehicle.” 

Round 2

Reviewer 1 Report

The paper has been improved. However,

- Line 33: Are the authors referring to the adaptive cruise control?
- The authors should additionally quantify the accuracy of their algorithm over the range 0-50 m (besides doing it over the range 0-100 m) and comment the result. This would definitely improve the paper presentation and constitute a sort of comparison with the quoted results in the literature.

 

Author Response

Concern 1: Are the authors referring to the adaptive cruise control?

Response:  Excellent point. This was a clear mistake in our end.

Modification: 

We modified the introduction chapter with the following: 

Line:33

“The adaptive cruise controller of a vehicle could act more robustly and with more reaction time if the system could perceive braking or stopped vehicles from afar.”

Concern 2: The authors should additionally quantify the accuracy of their algorithm over the range 0-50 m (besides doing it over the range 0-100 m) and comment the result. This would definitely improve the paper presentation and constitute a sort of comparison with the quoted results in the literature.

Response:  Accurate range measurements were not available for the samples of the test set, as no supplementary information was recorded in addition to the images. However, based on bounding box widths, range estimates were generated for all the images in the testing set. These estimates showed that roughly 90% of the vehicles in the images were at a distance of over 50 m. This focus on long-range brake light detection is what separates our work from previous works, as we present novel methodology and experimental results for a task that has not been previously studied in the existing literature.

We agree that a short-range comparison to the results found in the existing literature would certainly be an interesting addition to our manuscript. Unfortunately, we cannot satisfy this excellent idea in the given time-frame of five days, as additional data gathering and processing would be necessary.

Modification: 

We modified and clarified the manuscript with the following: 

Line:172

“The recording took place in real traffic conditions in Helsinki metropolitan area during February and March 2020, and the real traffic conditions made it possible to collect data from different distances, mostly over 50m. Longer distances were emphasised due to the intended application of predictive braking.”

Page: 6

Table 6 now includes note about the emphasised range.

“4 Mostly over the distance of 50m”