Research on Assessing Driving Ability of Older Drivers Based on Cognitive Tests: A Case Study of Beijing, China

Round 1

Reviewer 1 Report

The manuscript talks about research on cognitive tests for the older drivers. The research topic is interesting and important. However, the discussion section is too simplistic to provide more contents for readers.

 

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 2 Report

The paper presents an analysis on the relationship between the cognitive test of older drivers and their driving record. I have some comments the authors may consider:

In terms of formatting, the authors place the reference number after the sentence full stop.  I believe this should be before.

In section 2.3, I would suggest the authors talk a little more as to why they use the homogeneity of variance and the brown Forsythe correction coefficients. 

In the results, why is the test threshold 80 points?

I do not understand how scores are calculated. Specifically, those presented in tables 2 and 3.

State the null and alternative hypotheses. In table 2, the last "Sig" column is not clear to me. Specifically, what is the statistical test that results in this p-value.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 3 Report

Overview:

The authors test the potential multicomponent test of driver capacity focusing on memory, reaction, and judgement capacity to discriminate between poorer older drivers (with a crash history) and safer older and younger drivers. The study may be a good preliminary investigation into another way to identify older drivers who should consider driving self-regulation or retirement. However, the authors need to better justify the various components of the test with reference/linkage to their relationship to driving and older drivers specifically. While the results indicate good potential for discrimination, the authors could better treat this as a preliminary test of the survey and do more explicit evaluation of its utility after modification (i.e., using ROC analysis after removing non-significant/non-discriminatory questions). The introduction and conclusion should be revised to be more explicitly connected to the methods and findings of the study. Specific comments and areas of improvement are below:

 

 

Larger issues:

The authors did a decent job of reviewing spatial tests often used in reviewing the cognitive capacity of older adults, but they need to explain and expand the literature review of both reaction and judgement ability given their use in the questionnaire. What do those terms mean here? How have they been studied regarding older drivers? Make the case for the reaction ability test and judgement ability tests used in the study. Need this before introducing “reaction ability and judgment ability test” line 82.

 

The final paragraph of the introduction does not help me fully understand the gap or how the study will fill it. Please explain the potential utility of this comprehensive test and how it will benefit science/older drivers. Consider how in the study being an older driver with a recent history of a crash appears the most discriminant factor and pretty simple. Explain what the comprehensive test offers to add scientific value to the research presented.

 

The methods are not clear in the abstract or in the paper. Did you use sum scores in the one-way ANOVA? The reason for the proportions of the test is very unclear and lack citations. A test threshold (80 points) is mentioned in the results – needs to be fully explained in the methods.

 

There is a paragraph in the discussion devoted to age-related results [starting line 217 ] that are not included in the methods/results. These methods/results need to be described and shown. The authors need to at least conduct a t-test between the two older groups showing are significantly different. It would be better to do a linear test with continuous ages and the scores rather than older/younger to demonstrate these relationships.

 

The authors might investigate evaluating diagnostic ability of their test using an ROC curve/ROC analysis to describe its discrimination ability more formally.

 

The conclusion should not be restating the study or its findings in such detail. I suggest summarizing the first conclusion paragraph in one or two sentences without specific reference to numbers. Per previous comment, at this point, the results do not fully support the conclusion about age. [line 253] The policy implications are not clearly linked to the findings of the study [starting lines 254]. In the abstract and conclusion, it’s not clear how the research supports the need to “ guide the older drivers to understand traffic elements and rules” (abstract). No statements about physical conditions of older drivers can be made given the content of the study [line 260]. No new data should be brought up in the conclusion [line 270-271 about nighttime driving]. This should be a part of the data collection/discussion to be referenced here or deleted altogether.

 

Minor issues:

The title says the study is a “case study”. Case studies are generally reserved for describing studies with very small samples. This might be better described as a cross-sectional observational design.

 

How did you identify those with recent crash records? How did you define “recent”?

 

Citations are needed for the #s in the intro regarding the crash statistics of older Chinese drivers [lines 27-35, 36-38].

 

What is the relevance of the studies citations 4 and 5? Some references here include older drivers with dementia specifically. The authors may want to speak to that separately or compare to persons without dementia.

 

Note the sample size of younger and older group in the first paragraph graph of the results. Where/how were participants recruited? Where were the tests conducted? (In a university lab? Community space?) How were they incentivized or compensated for their participation?

 

Please give examples and references for the proportion of traffic crash causes for reaction, judgement, and operation processes [lines 133-135]. These proportions do not appear to align with the percent dedicated to each section. The final makeup of the test (Lines 135-140 and Table 1) appears to ignore the previous proportions outlined.

 

The physical survey/question procedure is unclear regarding the use of a computer versus then by doing the survey “by themselves”. Both appear to be by themselves. [Lines 143-145].

 

It would be informative in Table 2 to indicate when scores are significantly different from each other using post-hoc comparisons in addition to the omnibus test. Use superscripts to denote differences. This will show whether older safer group is different from safe younger group too.

 

Consider deleting the non-discriminant questions (e.g., short-term memory question 4, figure recognition) for future revised survey building and testing.

 

The note in Table 2 says “significance at 1%, 5%, and 10%.”. Use p<.01, etc. instead.

Author Response

Please see the attachment

Author Response File: Author Response.docx

Round 2

Reviewer 3 Report

Overview:

This reviewer thanks the authors for their responsiveness to many of the reviewer comments. The methods, discussion, and results have been improved.

 

However, the foundation of the tool components is still very questionable, with several components being mentioned for the first time in the measures section with no justification for them. The lack of discrimination of (p>0.05) for half of the items shows a need for better reason for their inclusion/selection in the first place. A major revision of the introduction/background needs to help explain the choices made.

 

It still isn’t clear what this survey is adding to the scientific literature or identification of unsafe drivers given how the caution older group, defined by a recent history of a major traffic crash, mostly scored below 80. The authors should consider - why do we need this test when we could ask if an older driver has had a recent crash? What does this survey add?

 

Specifics and additional comments are outlined below:

 

 

Introduction:

Concentration, anti-interference, and object recognition are first brought up in the measures section. They should be brought up and explained in the literature review to justify their inclusion in the test. For example, where does the color judgement question fit and how is it relevant?  

This needs citations and to be in the introduction “Considering that the older drivers' driving process is mainly based on concentration, anti-interference, short-term memory, object recognition and spatial judgment ability…” [lines 139-140]

 

The introduction/literature review doesn’t use the term “judgement” to lead us into these types of questions analyzed. The literature review has paragraph on “visual perception and visual spatial abilities”. Needs consistent terminology and to be introduced early.

 

Given the newness of the composition of the test, the authors could position the survey as a novel approach that they were trying out. Couching it as exploratory work could help justify the inclusion of some of the questions that turned out to be non-significant. However, as emphasized earlier, their original inclusion still needs to be explained/justified.

 

Methods

What else can you tell us about the participants beside their ages? Do they have any other health disorders that might impact driving and the cognitive tests used? How did you find out about their crash history? (What was the exact question?) How did you define “recent” and “major” traffic crashes?

 

This is a good revision on analysis section. Please, give another sentence explaining the ROC analysis and what ideal results would be. What does a good ROC outcome look like?

 

The discussion says “cognitive tests revealed that memory, reaction and judgement ability were all associated with the age of the drivers” [line 263]. Confirm in the analysis and results sections whether you looked at each of these components separately or only together (one score). Results have only one correlation suggesting you only tested the sum score and so you should not claim each one here is associated with age (it’s just the overall score).

 

Results:

Move the ROC results [line 162-168] to not separate the average score results [line 163, lines 168-171] and explain it a little more. Explain sensitivity/specificity and what it means for the test.

 

Even the discriminatory items seem rather easy for a cognitively in tact person (counting dots ‘calculation”, traffic sign recognition). Did the unsafe group have other documented mental health issues that should be noted? (e.g., dementia) What else can you tell us about the participants beside their ages?

 

Several test discrimination significance values are not significant at p<0.05 even though they are marked/described that way with bold and * (e.g., p=0.062, p=0.061, p=0.057)

 

 

Discussion:

Justify what the survey adds to the literature beyond a shorter test time than another study. Given how the caution older group, defined by a recent history of a major traffic crash, mostly scored below 80, why do we need this test when we could ask if an older driver has had a recent crash? What does this survey add?

 

“The goodness of the fit was considerably better than that achieved in other studies.” [Line 229] Explain and add citations. What goodness of fit test? Better than what other studies?

 

The additional discussion content is confusing. [lines 240-247] What is being said about the non-discriminatory questions? And what does it mean for future work on this survey?

 

“Simultaneously, the questions are specially designed based on the characteristics of traffic environment, including road traffic signs, signal lights and road scenes, which could also guide the older drivers to understand traffic elements and rules potentially.” [276-279] This can’t be concluded from the study. The test isn’t designed to help participants learn those elements but test their memory featuring those objects.

 

Minor issues:

“With the decrease in mental and physical functioning, older drivers are more susceptible to injuries.” [Lines 42-43] doesn’t logically follow. You might say the physical vulnerability of older drivers makes them more susceptible to injuries.

 

What was the “Information processing and reasoning judgment ability” question exactly?

 

This sentence is confusing: “While the caution older group showed bad performance, 2 of 19 older drivers score more than 80 points, where the average score is 71.05 (SD=13.62) and the passing rate is 10.5%.” [Line 171-172]

 

For p-values, they do not equal 0.000, they are p<.001.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 3

Reviewer 3 Report

This reviewer thanks the authors for their responses to the criticism and concerns.