Enhanced identification of eligibility for depression research using an electronic medical record search engine

https://doi.org/10.1016/j.ijmedinf.2009.05.002Get rights and content

Abstract

Purpose

Electronic medical records (EMRs) have become part of daily practice for many physicians. Attempts have been made to apply electronic search engine technology to speed EMR review. This was a prospective, observational study to compare the speed and clinical accuracy of a medical record search engine vs. manual review of the EMR.

Methods

Three raters reviewed 49 cases in the EMR to screen for eligibility in a depression study using the electronic medical record search engine (EMERSE). One week later raters received a scrambled set of the same patients including 9 distractor cases, and used manual EMR review to determine eligibility. For both methods, accuracy was assessed for the original 49 cases by comparison with a gold standard rater.

Results

Use of EMERSE resulted in considerable time savings; chart reviews using EMERSE were significantly faster than traditional manual review (p = 0.03). The percent agreement of raters with the gold standard (e.g. concurrent validity) using either EMERSE or manual review was not significantly different.

Conclusions

Using a search engine optimized for finding clinical information in the free-text sections of the EMR can provide significant time savings while preserving clinical accuracy. The major power of this search engine is not from a more advanced and sophisticated search algorithm, but rather from a user interface designed explicitly to help users search the entire medical record in a way that protects health information.

Introduction

Documentation and information management are fundamental aspects of patient care. Health information technology (Health IT) such as electronic medical records (EMRs), decision aids, computerized order entry and electronic prescribing have rapidly become part of daily practice for many physicians [1]. A 2006 Cochrane review concluded that Health IT has been shown to significantly improve quality by increasing guideline adherence, enhancing disease surveillance, and decreasing medication errors [2]. While data on the impact of EMRs on clinical care is increasing, there is less information investigating the use of EMRs in clinical research [3].

The University of Michigan Health System (UMHS) has stored clinical information in CareWeb [4], an in-house developed electronic documentation creation and viewing system. Since 1998, UMHS has utilized CareWeb as a unified EMR. Patient encounters, problem lists and medication data are largely encoded as free text. This allows clinicians to easily and rapidly enter data, either by dictating or typing, without the constraints of a controlled medical vocabulary or predefined document structure. In 2007 alone, over 108 million lines of text representing 2.6 million clinical documents were entered into CareWeb. These patient data are not coded, making extraction of this information challenging.

Given the time and effort required for manual chart review, attempts have been made to apply search engine technology to the EMR [5], [6], [7]. Existing approaches are often not optimal given concerns with: (1) impracticality of use for patient data containing hundreds of documents (e.g. search in MS Word); (2) presentation of results in manner that makes it difficult to efficiently review medical records and keep information for each patient distinct (e.g. text processing applications such as jEdit); and (3) inability to search for words based on the case sensitivity associated with many medical terms, as well as security concerns arising from creation of an external data index (e.g. Lucene). The electronic medical record search engine (EMERSE) was created at UMHS to provide a secure and efficient way to utilize the EMR for research and clinical data abstraction [8]. EMERSE offers an intuitive user interface for searching the EMR. Search results from EMERSE are shown in a format consistent with the organization of the EMR, segregated by individual patients and separate categories for demographics, the problem summary list, clinical notes, and pathology and radiology reports. One of its most powerful features is the ability to perform batch searches across multiple patients. “Bundles”, or groups of search terms, can be created to perform standardized searches of patient lists. EMERSE search bundles enable the user to search for or ignore phrases, as well as include case-sensitive searches and wildcard matches. Using the bundle, the system searches the record for these terms, and produces context-sensitive search results or “hits”. EMERSE functions to protect health information; unlike manual chart reviews, records that are not of interest are ignored, not appearing as “hits”.

Currently EMERSE is being used in more than 150 research studies in a variety of medical departments across our entire health system of 3 hospitals, 30 health centers and 120 clinics. Our study team is utilizing EMERSE for several research studies on later-life depression for two main purposes: (1) to quickly and efficiently screen thousands of patients for study eligibility in a manner that protects personal health information; and (2) to search or “mine” the free-text medical record to examine patient health variables that could not previously be studied in large-scale administrative data.

While we and many other study teams at our institution have found EMERSE to be invaluable, the standard in clinical research remains manual chart reviews performed by trained chart abstractors. Methods created for EMR information extraction need to be of similar accuracy and faster than manual review of medical records in order to add value.

The purpose of this study was to compare the clinical accuracy and speed of eligibility chart reviews performed using EMERSE with those done manually through the EMR. Based on our experience with this tool in several studies, we hypothesized that using EMERSE would be faster than manual chart reviews while maintaining clinical accuracy.

Section snippets

Setting

As the first step in a NIMH and IRB-approved study, we use our EMR to screen for eligibility by identifying patients who attended appointments in participating family medicine and internal medicine clinics in the prior week. Eligibility participation criteria include (1) age 60 and older, (2) primary care provider recommendation of new depression treatment within the previous month, (3) White or African American race, and (4) no history of dementia, bipolar disorder or schizophrenia. These

Results

For all raters, use of EMERSE resulted in a considerable time savings regardless of level of experience (see Table 1); chart reviews using EMERSE were significantly faster than traditional manual review (p = 0.03). The time savings were greater for the raters with more experience (raters 1 and 2), yet even the most inexperienced rater (rater 3) saved over an hour by using EMERSE.

In terms of valid eligibility decision-making, the percent agreement of raters with the gold standard using either

Discussion

In this study, use of a medical record search engine, EMERSE, was shown to be as clinically accurate and significantly faster than manual chart abstraction for the purposes of eligibility chart reviews. Rater experience did not adversely impact level of clinical accuracy between the two methods.

These results indicate that a medical record search engine such as EMERSE has tremendous utility for reducing the time required for chart screening in clinical research both for prospective subject

Conclusions

Using technologies such as an electronic medical record search engine to augment manual chart reviews for patient eligibility determination can result in significant time savings while preserving the ability to make valid decisions. The major power of EMERSE is not from a more advanced and sophisticated search algorithm, but rather from a user interface designed explicitly to help users search the entire medical record in a way that protects health information.

Summary points

“What was already

Acknowledgements

This research was supported by an NIMH grant (R21MH073002) awarded to Dr. Helen Kales. Portions of this manuscript were presented as a poster presentation at the American Association for Geriatric Psychiatry Annual Meeting, Orlando, FL, March 2008.

References (13)

There are more references available in the full text version of this article.

Cited by (0)

View full text