SaBi3d—A LiDAR Point Cloud Data Set of Car-to-Bicycle Overtaking Maneuvers

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Review Data | Manuscript ID: data-3036701

 

 

Thank you very much for the opportunity to review the paper:

 

Manuscript ID: data-3036701

Type of manuscript: Data Descriptor

Title: SaBi3d - A LiDAR Point Cloud Data Set of Car-to-Bicyle Overtaking

Maneuvers

Authors: Christian Odenwald, Moritz Beeking *

Spatial Data Science and Digital Earth

 

This paper introduces the Salzburg Bicycle 3d (SaBi3d) data set, with the purpose of informing traffic researchers on how to mitigate risks for cyclists related to motorized vehicle maneuvers. Data is made available at https://osf.io/k7cg9/.

 

The paper is well written and of interest to the journal. A critical point for me is understanding why this dataset is really the one, which would help, more than the others, make city planners understand dangerous overtaking maneuvers. The motivations given do not seem strong enough, and so, major work on this needs to be done before considering acceptance. Also, the paper seems to be not completely self-contained and a little too short in the comparisons performed and in the discussion section, as well. Major improvements on these aspects are required.

 

Here are some comments:

 

• I think the title might be missing a letter.

• The repository seems to be still some sort of work in progress. It is very hard to use the data, replicate the algorithms, and so on… At least with respect to the more common github repositories.

• My understanding is that the comparison is mostly done with the dataset nuScenes. It is unclear to me if there are more datasets available of the same quality or if there are only two competitive ones: SaBi3d and nuScences. It’s unclear to me why KITTI and Waymo are not discussed in detail, as there seems to be only a sentence talking about them. I believe more comparisons are needed.

• The 3d object recognition algorithms tested need to be extended. I understand that VISTA was used for nuScences, but if you just use that, the paper ends up risking having a structure too similar to what was already seen for nuScences. The fact that metadata generation is in a format identical to nuScences contributes to this feeling too.

• I think that the descriptions of Subsections 2.2 and 2.3 need some visualizations. It’s hard for those not already very familiar with LiDAR, SUSTechPOINTS, etc… to understand the data collection and processing part of your work.

• Which other object detection algorithms use the nuScences devkit? More explanations would be helpful.

• I think the paper needs more details about the VISTA algorithm, because, as it is, the paper is not self-contained and it’s hard to understand what is going on in the Results section.

• I think the discussion is minimal and looks more like a conclusion section than a discussion section. The discussion really needs to be expanded.

• The paragraph “Future work might…” makes the referee wonder: “Why then don’t we wait for the complete dataset before publishing the paper?”

 

I believe the paper needs at least a major revision.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The research addresses a topical issue, enhancing data accessibility and enabling large-scale analysis of overtaking manoeuvres involving cyclists. This study is noteworthy for its application of emerging technology with the objective of improving cyclist safety.

The platform utilized for data sharing is OSF (Open Science Framework), is an open-source online tool designed to facilitate data sharing, management, and interoperability with other services and the use of APIs. OSF is freely accessible to researchers, actively promoting open science. Given its comprehensive features and support for open research practices, it appears to be an appropriate and effective tool.

The data set appears to be correct.

Some suggestions for improving the study are presented:

·         It is recommended to read the paper available at https://doi.org/10.1080/01441647.2022.2122625 , which reviews the evolution of technology related to smart bicycles. This paper may provide valuable insights and ideas that could help enhance the summary.

·     In the explanation of the data collection (page 3), more detailed information should be provided about the four locations where the data were collected. Specifically, the following aspects need to be clarified: the width of the lanes, the presence of one or two lanes for both bicycles and motor vehicles, and whether there is a segregated cycle lane or cyclists ride in the same lane that cars.To thoroughly analyse the dataset from a cycling safety perspective, it is essential to have comprehensive information about the infrastructure at each site where an overtaking manoeuvre occurred.

·         Was the paper https://doi.org/10.1080/17489725.2024.2307969 developed based on the same data as the present study? What is the difference between these two data sets?

·         Revise the format of all references. For example, the paper 28 lacks information.

·         Page 2, line 78: specify "autonomous driving data sets of motorised vehicles" or similar.

·         All overtaking manoeuvres analysed in this study involve a single cyclist in an urban area. This raises the question of whether the same methodology can be extended to characterize the overtaking of sport cyclists, who often ride in groups, on rural or suburban roads. The urban scope of the study should be specified. In the summary or data collection section, it should be specified what kind of overtaking is being collected, whether it is a cyclist riding alone and what type of infrastructure is being used.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you very much for working on this review. I very much appreciate the changes implemented. The authors preferred to not implement some of the recommendations and I believe they risk to miss some opportunities. For example:

* Comment 2: It is always a good idea to include the code, even if the algorithm is not new. I feel the purpose of a journal publishing data must include full replicability. Furthermore, how can one appreciate the value of the data if part of the paper describing the data cannot be replicated?

* Comment 4: I am still not too convinced about the answer. I agree that the focuse needs to be on the dataset, but if the algorithm used is identical and nothing more is added on that component, then the feeling is that the work is a replication of what was already seen, but on the new data. If the data can be analyzed well with already available methods, the novelty/interest for ML researchers ends up being minimal.

* Comment 9: The answer of the authors seems in contradiction to what was written in the first version of the manuscript:

"Future work might include expanding this data set using different LiDAR setups for ease of usage by other research groups. Another consequent step could be adding a tracking component to the benchmark system and calculation of safety metrics to allow end-to-end assessments of overtaking maneuvers. And last, the data set and benchmark system could be expanded to cover other interesting maneuvers, e.g. at intersections".

Now they state: 

"The presented data set is complete and usable as-is".

I am confused about how the two statements agree with each others.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript has been improved

Author Response

No further suggestions were made; thank you for taking the time to review our manuscript.