Mild traumatic brain injuries (mTBI) constitute 70–90% of all head injuries.1 The Glasgow Coma Scale (GCS)2 3 is an important part of the clinical assessment of these patients in prehospital settings and emergency departments. The GCS measures level of consciousness via assessment of eye opening, response to commands and orientation to time, place and person. Designed to assess level of consciousness, the GCS contains no measure of the ability to lay down new memories, although it does test aspects of remote memory. This is of relevance as new learning has been demonstrated to be one of the best predictors of outcome after TBI.4 The Westmead Post-traumatic Amnesia (PTA) Scale (WPTAS) was designed to assess the ability of patients with TBI to lay down new memories over a 24 h period.5 6 It contains brief measures of orientation to time, place and person as well as the ability to remember new information from one day to the next. By measuring both orientation and capacity to form new memories, the WPTAS has been found to be clinically useful in populations with TBI who may be in PTA, having a high degree of inter-rater reliability as well as concurrent and predictive validity.7^–9 For patients with milder injuries, preliminary research has shown that a modified version of this scale can be used to reliably assess the capacity to lay down new memories over shorter time periods,10 allowing assessment of PTA soon after the injury has occurred.

Although the GCS has proved useful as an initial measure of severity of brain injury and is in common use, it has been said to have limitations when used to assess patients with minor injuries.11 One of its senior developers, Professor Sir Graham Teasdale, has said that the definition of mild or even minor head injury in a patient with a GCS score of 13–15 is most unsatisfactory.12 One of the shortcomings, he has argued, is that patients with a coma score of 15 make up, by far, the overwhelming number of patients classified in this group. Therefore, inclusion of all patients with a coma score of 13–15 in the same category underestimates the true severity of injury of patients with scores of 13 or 14. He has stressed that this classification also gives an impression of undue seriousness to those with a coma score of 15. A further concern with the use of the GCS is that there is a subgroup of patients with short or negligible coma and prolonged PTA.13 14 Despite these limitations, given the widespread adoption of the GCS in the assessment of patients with TBI, it is unlikely health workers will cease using it,3 as it gives important baseline data in patients who may deteriorate.

It appears that only four studies have been published in recent years in which PTA has been recorded prospectively using a standardised test in patients with mild to moderate head injury. In the first, Schwartz et al tested patients on the Galveston Orientation and Amnesia Test (GOAT) and a three word and three picture recall task.15 However, this was done on a 24 h recall basis and has not been tested over shorter time periods.

The second study was conducted by Van Der Naalt and colleagues.16 However, the scale used in the study was predominately made up of orientation questions and did not test new learning ability. This highlights a need to develop a scale that adequately measures both orientation and new learning ability over shorter time periods.

The third study, by Ponsford and colleagues,10 demonstrated that a revised version of the WPTAS (R-WPTAS) could be used for assessing adult patients with mTBI in an emergency department at hourly intervals. Scores from the R-WPTAS were found to correlate with GCS scores and estimated duration of PTA. However, Ponsford et al recommended that further studies would be required using their revised scale on an independent sample of patients with mild TBI.

The fourth study was that by De Monte and colleagues17 who investigated whether a single determination of orientation and retrograde amnesia using eight orientation questions from the GOAT could be sufficient to provide an index of severity of mTBI within the first 24 h. As with the Van Der Naalt et al study, new learning ability was not measured. Results revealed that 17% of the non-head injured orthopaedic control group were misclassified as being in PTA.

The present study involved a prospective neuropsychological evaluation of consecutive patients presenting to an emergency department with a history of mTBI as well as a matched control group of patients in the emergency department who presented for reasons other than a TBI. During the acute treatment phase, all patients were assessed on the GCS and the R-WPTAS. In the research assessment conducted shortly prior to discharge, the GCS, R-WPTAS, ImPACT Trauma computerised neurocognitive test, Concussion Symptoms Checklist, Westmead Selective Reminding Test (WSRT) and GOAT were administered. The aim was to determine whether administering the R-WPTAS, in addition to the GCS, would increase diagnostic accuracy in identifying cognitive impairment in patients with mTBI. This study is also the first independent study to validate the R-WPTAS. We hypothesised that (i) patients with mTBI would have lower scores on all measures than control patients; (ii) the R-WPTAS would have greater concurrent validity with the neuropsychological measures than would the GCS scores; and (iii) the diagnostic accuracy of the R-WPTAS in identifying significant cognitive disability in patients with mTBI would be greater than that of the GCS.

METHOD

Participants

Participants were adults, aged between 18 and 61 years, who presented for treatment to the emergency department at John Hunter Hospital, a level 1 trauma centre and tertiary referral hospital in New South Wales, Australia. For inclusion in the mTBI group, patients had to meet the American Congress of Rehabilitation Medicine criteria,18 with revision by the World Health Organization,19 except that patients with intracranial lesions were excluded.

Exclusion criteria were a skull fracture, transfer out of the emergency department for further management of the TBI, acute cognitive impairment secondary to drug or alcohol abuse (based on clinical judgment by the research psychologists), any coincident medical/neurological illness and an inability to use the dominant hand (because of the computerised tests). Numbers and reasons for those excluded based on unsuitability are listed in fig 1.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 Participants consecutively presenting to the emergency department following mild traumatic brain injury (mTBI) from February to September 2005. ED, emergency department.

A consecutive series of patients with mTBI were approached while research assistants trained in neuropsychological test administration were on duty in the emergency department between February and September 2005. This represented 81% of all mTBIs who presented. GCS and R-WPTAS data were only analysed for those patients who agreed to become research participants and complete the neuropsychological research battery.

Control participants were matched by age, education and gender to participants with mTBI. They had presented to the emergency department with injuries other than head injuries and did not require admission to hospital. Overall, 156 control patients were approached; 56 declined and 100 agreed to participate in the study. Exclusion criteria were acute cognitive impairment secondary to drug or alcohol abuse (based on clinical judgment by the research psychologists), any coincident medical/neurological illness and an inability to use the dominant hand.

Patients with mTBI and control subjects were not excluded on the basis of a history of alcohol abuse, previous head injury, neurological disorder, psychiatric problems or learning difficulties as these factors are known to influence outcome following mTBI and removing such participants would be likely to bias the results.10

The study was approved by the ethics committees of the Hunter New England Area Health Service, Newcastle, the Central Sydney Area Heath Service and Macquarie University, Sydney, Australia.

Measures

Revised Westmead PTA Scale

The R-WPTAS (table 1) consists of a combination of the Best Verbal Response Questions (BVRQs) of the GCS and the modified version of Ponsford et al’s R-WPTAS.10 The BVRQs of the GCS assess the injured person’s orientation to time, place and person. Since some of the BVRQs of the GCS and R-WPTAS overlap, these questions were asked only once. The possible range of scores for trial 1 was 0–9 and for subsequent trials, 0–12.

Glasgow Coma Scale

Acute GCS scores for patients with mTBI were obtained from their medical files.2 For the control patients, GCS scores were computed from the BVRQ scores of the GCS (overlapping with the R-WPTAS) obtained at the research assessment. To this score 10 points were added (range 3–15), based on the assumption that if control patients who were ambulant and alert and were capable of appropriately responding to orientation questions, they would be assumed to obtain a maximum score on the Best Eye Opening component (a score of 4) and Best Motor Response component (a score of 6) of the GCS.

ImPACT Trauma

ImPACT Trauma is a computer program, developed to assess patients following concussion. It measures multiple aspects of cognitive functioning and yields five summary scores which are: verbal memory, visual memory, reaction time, visual motor speed and impulse control (a validity index).20

Concussion Symptoms Checklist

ImPACT Trauma also includes a Concussion Symptoms Checklist (CSC) which measures the degree of the patient’s post-concussion syndrome based on the patient’s self-reported severity of post-concussion syndrome symptoms, the possible range being 0–132.

Westmead Selective Reminding Test

Participants are asked to remember as many words (out of 10 standard items) as they can over 10 repeated learning trials. Participants’ consistent long term retrieval was scored according to the WSRT protocol, representing a count of the number of words consistently retrieved over the 10 trials, the possible range being 0–100.9

Galveston Orientation and Amnesia Test

GOAT is a quantitative rating scale that measures disorientation and amnesia during the recovery process following TBI. It consists of 16 questions assessing orientation to time, place, person and memory for events preceding and following injury. The possible range in scores is 0–100. A score greater than 75 on at least two consecutive occasions indicates emergence from PTA.21

Procedures

mTBI participants

Acute treatment phase

The R-WPTAS was initially administered by ambulance officers at the scene of the accident or on route to hospital to 16% of patients with mTBI. In the emergency department, patients were administered the R-WPTAS by clinical staff every hour over the 4 h routine observation period. If a patient gave incorrect or no answers, clinical staff immediately presented the correct answer to the patient for memory storage and later recall. In respect to the picture card task, at the first neurological observation, patients were shown a set of three pictures and told to remember them. At the second and every consecutive neurological observation, patients were required to recall or recognise the pictures presented in the previous trial. If successful, a new set of three pictures was presented. If patients were not successful, the same set of pictures was presented again for memory storage and later recall. At each hourly interval patients were also assessed with the GCS, receiving a conventional GCS score.

Research assessment

Once patients with mTBI were cleared for discharge, they were formally invited to take part in the research and consent obtained. Participating patients with mTBI were breathalysed and their alcohol reading recorded. Demographic information, a brief medical history and any history of learning difficulties and drug and alcohol abuse was obtained. Patients were administered ImPACT Trauma, CSC, WSRT and GOAT.

Control participants

Research assessment

Control participants were approached for research participation while in the emergency department and consent obtained. They were given the same measures as participants with mTBI at the research assessment described above. However, the R-WPTAS was only administered twice, providing one measure involving recall after approximately 1 h.

Statistical analysis

Group differences in demographics, learning difficulties, psychiatric and TBI histories, drug and alcohol use, blood alcohol level, and morphine administration were assessed using Mann–Whitney U tests or Fisher’s exact tests, as appropriate. Group differences on amnesia and neuropsychological measures and their concurrent validity were analysed using multivariate analysis of variance (MANOVA) on SPSS V.15 based on the results from the time of the research assessment. Comparison of diagnostic accuracy (diagnosis of cognitive impairment in patients with mTBI) was conducted by receiver operating curve (ROC) analysis using MedCalc22 and web based software from the Centre for Evidence Based Medicine, University of Toronto,23 based on results obtained at the second neurological observation because this was the earliest point at which new learning could be assessed.

RESULTS

The majority of participants were male (77%). Mean age was 30.74 years (SD 11.34; range 18–61), and mean years of education was 11.65 (SD 2.13; range 7.0–20.0). In the mTBI group, the main cause of mTBI was road traffic accidents (43%), followed by assault (27%), sports injury (15%), fall (10%) or hit by an airborne object (5%). In the control group, the major cause of injury was sports injury (34%), followed by fall (25%), other causes (25%), motor vehicle accident (9%) and assault (7%). On presentation to the emergency department, participants with mTBI had a mean GCS score of 14.67 (SD 0.63; range 13–15).

Twenty-one research participants (9 mTBI and 12 control patients) were excluded from the analyses as they achieved a score of 20 or higher on the Impulse Control measure of ImPACT Trauma, thus raising questions regarding the reliability of their results.24 A further research participant was excluded as his injury did not meet mTBI criteria. Final sample size was 82 patients with mTBI and 88 control participants (see fig 1). Not all tests were completed by all participants, marginally reducing the sample size for some analyses. No significant differences were found between the groups in terms of gender, age, education, learning difficulties, psychiatric problems, alcohol use and drug use (table 2). Participants with mTBI received significantly more morphine administrations (p<0.001), had significantly higher blood alcohol readings at the time of the research assessment (p<0.001) and reported more previous head injuries than controls (p<0.001).

In the mTBI group, the proportions of patients at the second neurological observation with GCS scores of 13, 14 and 15 were 2%, 11% and 87%, respectively. In the control group, the proportions were 0%, 2% and 98%, respectively.

Group comparisons

MANOVA results after controlling for blood alcohol level, number of previous head injuries and morphine administration showed significant group differences on all measures except impulse control. ImPACT Trauma’s summary scores including CSC, R-WPTAS, WSRT, GOAT and GCS scores obtained at the research assessment were the dependent variables. The independent variable was patient group. Wilks’ lambda revealed a significant multivariate effect of patient group on all measures of cognitive performance and concussion symptoms (F(10,151) = 8.83, partial eta²  = 0.369; p<0.001). When the results for the dependent variables were considered separately, all measures, except impulse control (p = 0.207), reached significance. Effect sizes are presented in table 3 (amnesia tests) and table 4 (neuropsychological measures). To obtain a reduced set of dependent variables with each contributing to differentiate the groups at p<0.01, a stepwise discriminant function analysis (DFA) with the five ImPACT Trauma summary scores was performed. The DFA identified verbal memory and reaction time as significant factors. Based on the DFA results, the verbal memory and the reaction time summary scores were used as dependent variables for subsequent analyses.

Concurrent validity

To investigate the concurrent validity of the GCS and R-WPTAS on cognitive functioning, as assessed by ImPACT Trauma, two MANOVAs were carried out in the mTBI group only. The independent variable in the first MANOVA was the GCS and in the second MANOVA the R-WPTAS. In this within group analysis, only blood alcohol level emerged as a covariate and was controlled for subsequently. Results for GCS were not significant, explaining less than 1% of variance in cognitive functioning (F(2,76) = 0.334, p = 0.717, partial eta² = 0.009). Results for R-WPTAS revealed a significant multivariate effect, explaining 28% of variance in cognitive functioning (F(2,76) = 14.82, p<0.001, partial eta²  = 0.281).

Comparison of diagnostic accuracy

ROC analysis was used to determine optimal cut-off points of group discrimination for the GCS and R-WPTAS (maximising sensitivity and specificity) in terms of diagnosis of cognitive impairment. Based on the GCS score at the second neurological observation (the earliest point at which new learning could be assessed), a score of ⩽14 was considered “disease positive”. A score of 15 was considered “disease negative”. Based on the R-WPTAS score at the second clinical neurological observation, a score of ⩽11 was considered “disease positive”. A score of 12 was considered “disease negative”. The sensitivity, specificity, positive predictive value and negative predictive values are presented in table 5. The R-WPTAS had a sensitivity of 60% and the GCS 13%. The R-WPTAS had a larger area under the curve than the GCS (95% CI 0.69 to 0.82 and 0.49 to 0.64, respectively (p<0.001)).

DISCUSSION

The results of this study suggest that the R-WPTAS is more accurate than the GCS in identifying adult patients with mTBI who may be cognitively impaired in an emergency department.

A consecutive sample of patients with mTBI was shown to have significantly lower scores on cognitive measures than a control sample matched for age, education and gender. The R-WPTAS showed greater concurrent validity with the neuropsychological measures from ImPACT Trauma than the GCS. A significantly greater amount of variance was explained by the R-WPTAS than the GCS on these measures, suggesting a much stronger relationship between the R-WPTAS and neuropsychological function than that shown by the GCS. This has been shown previously in patients with more severe traumatic brain injuries using the original WPTAS,8 confirming the utility of this measure across a range of patients with differing brain injury severity. At the second neurological observation, while the GCS only misidentified 2% of the control participants and the R-WPTAS 9%, the GCS only correctly identified 13% of the participants with mTBI whereas the R-WPTAS correctly identified 60%. Although the positive predictive values for both measures were similar, the confidence interval for the R-WPTAS was much narrower, thus increasing the accuracy of the test. ROC analysis confirmed superior diagnostic accuracy on the part of the R-WPTAS. Given the high base rate in this study (48%), predictive values will be lower in centres with a lower base rate than that in the present study.

The R-WPTAS consists of items taken from Ponsford and colleagues’ version of the WPTAS10 and the BVRQs from the GCS. At the time of their second neurological observation, 60% of participants with mTBI could not lay down simple new memories, yet 87% of these had been assigned a GCS of 15. Although the GCS was not designed to measure memory (except some aspects of remote memory), in everyday use in prehospital settings and emergency departments the normal score of 15 is usually taken to imply normal brain function (which implies normal new learning capacity). The present study suggests this is not the case. It can thus be argued that the use of the R-WPTAS would contribute significantly to the accuracy of monitoring patients with mTBI to ensure that they are either not discharged from hospital or referred on for further assessment after discharge while still suffering significant cognitive impairment. While it misclassified 9% of emergency department control participants, this is lower than the 17% reported by De Monte and colleagues.17 By the time the GOAT was administered at the research assessment, patients were scoring well above the cut-off of 75, which suggested that they were out of PTA. This would indicate a lack of sensitivity on the part of the GOAT as the R-WPTAS mean score was still in the PTA range.

The operational definition of PTA for moderate to severe TBI states that “a person is said to be out of PTA if they can achieve a perfect score on the [original] Westmead PTA Scale for 3 consecutive days” (Marosszeky and colleagues6 p16). Recent research on cases of more severe head injuries has suggested that patients have probably emerged out of PTA when they first score 12/12 on the WPTAS.26 Results from the current study suggest that for patients with mTBI a perfect score on their second neurological observation, which is the first occasion on which new memory is assessed, may be enough to show that they have emerged from PTA. Further studies, however, will be required to validate the reliability of this finding in an independent sample of patients with mTBI while a concurrent “gold standard” cognitive test is administered. In this study it was a limitation that the concurrent neuropsychological tests were only administered at the time of the research assessment once patient consent to participate had been obtained. To overcome this problem, the timing of research assessments will need to be negotiated with ethics committees if the optimal time for identification of amnesia is to be determined. Ethical constraints precluded collection of demographic data on patients who declined to become participants. Future studies would also need to attempt negotiation with ethics committees so these data can be collected.

It should be noted that the R-WPTAS procedure is only appropriate for patients with mTBI and therefore the original WPTAS procedures which involve assessment on a daily basis will remain appropriate for patients with more severe injuries.

The issue of whether or not patients in emergency departments have had a brain injury is redundant as this is determined by the presenting clinical history. The issue to be addressed is: does the patient have ongoing symptoms of brain impairment which will influence management decisions? The evidence from this study, which confirms previous studies, is that use of the R-WPTAS as an adjunct to the GCS can significantly assist in rapidly identifying those patients with mTBI who may need further management.

Administration of the R-WPTAS takes less than 1 min and is thus time efficient and cost effective. This is a distinct advantage over other well established neuropsychological test protocols used for the accurate assessment of patients with mTBI. These other test protocols are not only time consuming but also can usually only be administered by trained neuropsychologists, making such assessments impractical for routine use in acute situations. The ease of administration of the R-WPTAS makes it a practical measure that can be routinely used by a range of health professionals, including ambulance officers, nurses and medical staff, leading to greater confidence in assessing patients with mild head injuries without trading time for clinical accuracy.