We read with interest the article “Accuracy of the DriveABLE cognitive assessment to determine cognitive fitness to drive” by Dr Dobbs,1 which appeared in the March issue of Canadian Family Physician, and disagree with his interpretation of the findings. Dr Dobbs’ conclusion that the DriveABLE In-Office cognitive assessment is highly accurate in identifying drivers with suspected or confirmed cognitive impairment who would pass or fail the DriveABLE On-Road Evaluation is based, incorrectly, on overall cell percentages. In his article, he stated the following: “For the total sample, only 1.7% of the patients who received an In-Office pass outcome received a mismatching DORE [DriveABLE On-Road Evaluation] outcome of fail. The errors for the fail outcome were somewhat higher but still low (5.6%).”1
Rather than presenting overall percentages, Dr Dobbs should have reported the actual cell counts and the row percentages, as these are far more relevant (a revised version of Table 1 is available from the corresponding author). The row percentages show that 62 of the 504 individuals who passed the In-Office assessment (12.3%) failed the On-Road test; and 204 of the 1474 who failed the In-Office assessment (13.8%) passed the On-Road test. These numbers are very different than the 1.7% and 5.6% presented in the article.
Finally, the overall raw agreement between the In-Office assessment and the On-Road test is only 50.4%. The 2 approaches would agree by chance alone 33% of the time, and a statistic to denote “chance-corrected” agreement should have been presented. We calculated such a statistic (Cohen weighted κ with quadratic weights) and it was also far from impressive. The κ value for these data is 0.432 (95% CI 0.406 to 0.459), which, based on accepted guidelines, represents only fair-to-moderate agreement—well below the required minimum value to support Dr Dobbs’ conclusions.2,3 Landis and Koch suggested that κ values of 0.61 to 0.80 represent substantial agreement, while values of 0.81 to 1 designate almost perfect agreement.2 Fleiss characterized κ values greater than 0.75 as excellent.3
Therefore, the correct interpretation of the data is that there is only fair-to-moderate agreement between the In-Office and On-Road outcomes. We do not agree with Dobbs’ conclusion that these “findings provide the evidence physicians need to be confident in using the recommendations from the DriveABLE In-Office cognitive evaluation to assist them in making accurate, evidence-based decisions about their patients’ fitness to drive.”1 Dobbs’ interpretation and the editor’s key points need to be revised to accurately reflect the results.
Footnotes
-
Competing interests
All authors are members of Candrive, a research team funded by the Canadian Institutes of Health Research that supports safe driving in older adults. One of the main objectives of Candrive is to develop a tool to aid clinicians in assessing older drivers’ fitness to drive.
- Copyright© the College of Family Physicians of Canada