Skip to main content

Main menu

  • Home
  • Articles
    • Current
    • Published Ahead of Print
    • Archive
    • Supplemental Issues
    • Collections - French
    • Collections - English
  • Info for
    • Authors & Reviewers
    • Submit a Manuscript
    • Advertisers
    • Careers & Locums
    • Subscribers
    • Permissions
  • About CFP
    • About CFP
    • About the CFPC
    • Editorial Advisory Board
    • Terms of Use
    • Contact Us
  • Feedback
    • Feedback
    • Rapid Responses
    • Most Read
    • Most Cited
    • Email Alerts
  • Blogs
    • Latest Blogs
    • Blog Guidelines
    • Directives pour les blogues
  • Mainpro+ Credits
    • About Mainpro+
    • Member Login
    • Instructions
  • Other Publications
    • http://www.cfpc.ca/Canadianfamilyphysician/
    • https://www.cfpc.ca/Login/
    • Careers and Locums

User menu

  • My alerts

Search

  • Advanced search
The College of Family Physicians of Canada
  • Other Publications
    • http://www.cfpc.ca/Canadianfamilyphysician/
    • https://www.cfpc.ca/Login/
    • Careers and Locums
  • My alerts
The College of Family Physicians of Canada

Advanced Search

  • Home
  • Articles
    • Current
    • Published Ahead of Print
    • Archive
    • Supplemental Issues
    • Collections - French
    • Collections - English
  • Info for
    • Authors & Reviewers
    • Submit a Manuscript
    • Advertisers
    • Careers & Locums
    • Subscribers
    • Permissions
  • About CFP
    • About CFP
    • About the CFPC
    • Editorial Advisory Board
    • Terms of Use
    • Contact Us
  • Feedback
    • Feedback
    • Rapid Responses
    • Most Read
    • Most Cited
    • Email Alerts
  • Blogs
    • Latest Blogs
    • Blog Guidelines
    • Directives pour les blogues
  • Mainpro+ Credits
    • About Mainpro+
    • Member Login
    • Instructions
  • RSS feeds
  • Follow cfp Template on Twitter
Research ArticleResearch

Can you use a sequential sample of patients as a substitute for a full practice audit?

Study of mammography screening rates in 20 family practices in Ontario

Graham Swanson and Janusz Kaczorowski
Canadian Family Physician October 2008, 54 (10) 1436-1437.e5;
Graham Swanson
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: swansong@mcmaster.ca
Janusz Kaczorowski
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF
Loading

ABSTRACT

OBJECTIVE To compare rates of mammography screening among women in family practices, based on a sequential sample of eligible women presenting to the practices during an 8-week period, with rates found in a full audit of all eligible patients.

DESIGN Chart review.

SETTING Twenty community-based family practices in south-central Ontario.

PARTICIPANTS Family physicians and their female patients 52 to 71 years old who had had at least 1 visit to the office during the past 3 years.

INTERVENTION Eligible patients were sampled by 2 approaches: sequential sampling of patients coming for appointments during an 8-week period and a full practice audit of all eligible women.

MAIN OUTCOME MEASURE Mammography rates found using the 2 approaches.

RESULTS The mean time-appropriate rate of mammography screening based on the sequential sample was 66.4%. The mean time-appropriate rate of mammography screening for the full practice audit was 58.8%. The sequential sample rate was higher than that of the full audit by 7.6%; differences ranged from −6.5% to 24.9% among practices. Regression analysis indicated a positive and significant correlation between rates based on the data generated by the 2 different approaches (r2 = 0.50).

CONCLUSION A rate of mammography screening based on a sequential sample can reasonably approximate the actual rate of mammography screening that would be found based on a full practice audit.

Quality assurance activities in family medicine are recommended by the College of Family Physicians of Canada.1 Quality assurance involves reviewing and reflecting on the way we practice and trying to improve it. Several authors have studied various ways to measure family physicians’ performance of selected preventive maneuvers and to provide feedback to improve the quality of care.2,3 There are at least 2 recurring problems that deter implementation of quality assurance in family practices: getting physicians involved in the quality assurance activities and the amount of time and energy it takes to perform data extraction.2 This last is particularly a factor for practices that do not use fully electronic medical records.

In an attempt to make data extraction easier and less time-consuming, 20 physicians participated in an audit of mammography screening rates in family practice using an approach designed to overcome these problems. Instead of conducting a full practice audit or an audit based on a random sample of patients representative of the practice, a sample was generated from consecutive eligible patients who were booked for appointments during a predefined data-collection period.4 As these women’s charts were already pulled for the appointments, the relevant information was easily available and was collected by the physicians or their office staff.

Once collected, the data were faxed to a research assistant who collated them weekly. Anonymous individual and group summaries were prepared and shared with all participants weekly.

This very inexpensive process stimulated an enthusiastic response from participants as they collected their own data and compared them with the data of their peers. Before further studies of the effect of using a sequential sampling technique to collect data for an audit were to be undertaken, it was thought important to assess how well the data generated using this approach represented data from the whole practice. This paper reports on a study in which the results of an audit of mammography screening rates among all women in a practice eligible for screening were compared with the results of an audit of a sample of eligible women from the same practice chosen by a sequential sampling technique.

METHODS

Twenty family physicians in south-central Ontario volunteered to participate in the study. Seventeen of these physicians had been involved in a previous audit that had been part of a faculty development project in the Department of Family Medicine at McMaster University in Hamilton, Ont. The other 3 were colleagues of 1 of the authors (G.S.) who recruited them for the study.

Data were collected using 2 sampling strategies. One approach was based on a full practice audit of all eligible patients in a practice, and the other collected data from a sample of eligible women from the same practice who booked appointments during the 8-week study period. Eligible patients were defined as women 52 to 71 years old who had had at least 1 visit to the office during the past 3 years. The age group chosen corresponded to the recommended age group for screening with 2 years added to allow for the first mammograms to be done within the recommended period (50 to 69 years old).5 All women were included even if mammography was ordered for diagnostic rather than screening purposes.

Each practice was responsible for generating a list of eligible patients for the complete audit. The same research assistant visited each practice and reviewed the charts of all patients on the list. Physicians or office staff were responsible for collecting data from the sequential sample of eligible patients who had appointments during the study period. Data sheets were faxed to the research assistant who entered the data into an Excel spreadsheet. The practices were assessed in 4 blocks of 5 physicians to allow the full and sequential audits to occur simultaneously.

The research assistant collected data on age, date of most recent mammogram, and whether patients had participated in the Ontario Breast Screening Program. Questionnaires were sent to physicians before and after the audit, asking about practice profiles and the effect of the processes of the study on office function. In 1 practice, a second reviewer audited a random sample of the charts of eligible patients to ensure validity of the extraction process.

We hypothesized that there would be a 5% to 10% difference between rates measured by the 2 audit techniques with the sequential approach yielding a higher rate of mammography than the full practice approach.

Data were analyzed using SPSS software for Macintosh (version 9.1). Descriptive statistics were calculated, and correlational analyses were conducted using the difference between the mammography rates based on the 2 sampling strategies as the dependent variable to examine associations with physician and practice variables. All statistical tests were 2-sided and a P value of > .05 was considered statistically significant. Ethics approval was received from the Ethics Review Board of Hamilton Health Sciences.

RESULTS

The audits were carried out between September 2003 and June 2004. Seventeen of the family physicians were affiliated with McMaster University Department of Family Medicine (part-time clinical faculty) as part of a group of community physicians who teach family practice residents in their offices; 3 family physicians were community physicians with no university affiliation (Table 16). Table 2 shows a comparison of the sociodemographic characteristics of eligible patients by sampling method. Mammography rates for each data sampling method are compared in Figure 1. Data were collected by physicians in 10 practices, by nurses in 6 practices, and by other members of staff in 3 practices. The mean time-appropriate mammography rate for the full practice audit was 58.8% and for the sequential sample was 66.4%. The sequential sample approach resulted in a higher rate in all but 4 practices. Data were analyzed at various intervals using a grace period of 6 or 12 months after the time-appropriate 2-year mark. In the full practice audit, the rate of mammography increased to 70.0% during the study, while in the sequential sample, the rate rose to 76.2% by the end of the 12-month grace period.

Figure 1
  • Download figure
  • Open in new tab
Figure 1

Mammography rates based on full practice audit and on the sequential sample, by practice

View this table:
  • View inline
  • View popup
Table 1 Characteristics of physicians participating in the study compared with characteristics of physicians in the 2004 National Physician Survey (NPS)

Year of graduation ranged from 1965–1991 (median 1980) in the study and the mean year of graduation was 1982.5 in the NPS; practice sizes in the study ranged from 900–3200 patients, compared with a mean size of 2000 patients in the NPS.

View this table:
  • View inline
  • View popup
Table 2

Characteristics of eligible patients by sampling method

Figure 2 shows a scatter diagram as well as the regression equation of full audit versus sequential sampling where the cutoff date was 24 months from date of data collection to date of most recent mammogram. Regression analysis, using ordinary least squares linear regression, indicated a positive and significant correlation between mammography rates using the 2 different sampling techniques (r2 = 0.50). Regression analysis indicated that the rate of mammography in the sequential group could be used as a reasonable approximation of the actual rate of mammography based on the full practice audit by employing the following equation: y (actual mammography rate) = 14% + (sequential rate) × 0.66. For example, for a practice with a mammography rate of 50% based on sequential sampling, the full practice rate would be estimated to be 47% (14% + 50% × 0.66 = 47%) and for a practice with a 65% rate, the full practice rate would be estimated to be 56.9%.

Figure 2
  • Download figure
  • Open in new tab
Figure 2 Scattergram of mammography rates based on a full practice audit and on the sequential sample

Also shown is the regression equation of the full audit vs the sequential sampling.

There was no significant difference in mammography rates for women younger than 60 years and women older than 60 years by either sampling technique. Figure 3 summarizes physicians’ perceptions of the effect of the study on their practices.

Figure 3
  • Download figure
  • Open in new tab
Figure 3

Physicians’ perceptions of the effect of both audit processes on their practices: N =19.

DISCUSSION

The mean rate of mammography screening was 7.6% higher for the sequential sample than for the full audit. Although the difference in the mean rate was less than 10%, as originally postulated, the amount of variability was greater than expected as was the direction of the difference, which ranged from −6.5% to 24.9%. In 11 practices, mammography rates based on the 2 approaches differed by less than 10%. The anticipated pattern of sequential mammography rates being systematically higher than rates based on a full practice audit was confirmed.

While a rate based on a sequential sample seems to reasonably approximate the actual mammography rate, similar audits need to replicate this study design at other sites and cover other topics to assess its validity and reliability. The fact that the mammography rate based on the sequential sample was higher was anticipated because patients who have office visits are more likely to have more frequent visits and thus are more likely than those who do not have office visits to have mammograms ordered. The regression equation describes a mathematical relationship between mammography rates based on the sequential sample and rates based on a full audit. For physicians who do not have easy access to a full practice audit or who do not have fully computerized medical records, the sequential sampling strategy offers an alternative method of monitoring the care they provide.

Mammography rates based on sequential sampling run the risk of missing women who do not make appointments during the study period. The sequential sampling approach was intended to generate an approximate rate of mammography for a practice, not to find women who had not had time-appropriate mammograms.

In Ontario, incremental financial bonuses are offered as performance incentives to physicians practising under certain new payment models (rostered practices with prescribed rates). The mammography rates we found were similar among practices whether physicians were eligible through rostering for these incentives or not.

Exploratory analysis of factors thought to influence mammography screening rates, such as practice size, age of physician, sex of physician, availability of electronic records, method of payment, and percentage of eligible patients in the practice, did not reveal any notable patterns. One practice identified only 16 eligible patients during the entire study period. This was considerably fewer that the 6 to 10 patients expected to be eligible in each practice each week. As a result, the estimated rate for that practice had a very wide margin of error. The effect of the small numbers was not likely to account for the large difference in rates. Most physicians did not find the sequential audit technique an onerous experience or one that interfered with routine office function. One physician, who did find data collection onerous, had computerized medical records so found a manual data collection process relatively cumbersome. The process generated little anxiety among physicians or their staff, and almost half of them said they were willing to undertake another audit.

Changes in practice and in rates of mammography, while anticipated, were not measured in this study. Some participants commented that it had been a good experience. One said, “I was alerted on how history-keeping on the front of the chart was extremely important,” and another commented, “Recording at our end needed change.” Others said, the audit was “Worthwhile, surprising to me and adjustments gladly made,” and, “[It] provided me with an easy way to be aware of women in my practice needing mammogram.”

Limitations

There are several limitations to this study. In the sequential approach, data collection was self-recorded and, as such, was subject to selection bias. Collection of data by physicians’ staff, although not reported as an onerous task, might have been forgotten some days, thus reducing the number of patients in the sequential sample. Lists for the full audit were generated by the practices, which could also make them subject to selection bias. Without a computerized registry or roster, these lists might not truly represent the full number of eligible patients. These biases could affect estimates of mammography rates in the sequential sampling approach. As patients who have appointments are more likely to have mammograms ordered, sampling such patients is likely to produce a higher rate of mammography. All the physicians were practising in south-central Ontario, and most were affiliated with McMaster University so they are not likely to be representative of Canadian family physicians in general. This chart audit used the presence of a report in the chart as evidence of a mammogram being done. It did not capture mammograms that were ordered but not done. Although a doctor’s endorsement of mammography helps encourage women to have screening, some women prefer not to have mammograms.7,8

Conclusion

A mammography rate based on a sample generated through a sequential approach can create a snapshot of a whole practice for physicians. Having peers collecting similar data creates a credible standard of care against which participating physicians can measure their performance without feeling uncomfortable. Further study needs to be done on other audit topics to see how well rates generated through a sequential approach approximate actual rates that would be found in a full practice audit. Questions about how practice might change after an audit need to be explored with participating physicians.

Acknowledgment

This project was supported by a grant from Physicians’ Services Incorporated.

Notes

EDITOR’S KEY POINTS

  • Quality assurance activities in primary care are important for providing feedback to physicians and improving the quality of care for patients. The amount of time and energy it takes to perform data extraction, however, often deters physicians from participating in such activities. In this study, the authors tested a strategy of sampling consecutive eligible patients, in an attempt to make data extraction easier and less time-consuming, and compared results with those from full practice audits.

  • The authors found a positive and significant correlation between rates based on the data generated by the 2 different approaches. Most physicians did not find the sequential audit technique interfered with routine office function and almost half of them said they would be willing to undertake another audit.

  • Although some physicians in Ontario are eligible to receive performance incentives to encourage increased mammography screening rates, rates were found to be similar among practices whether physicians were receiving performance incentives or not.

POINTS DE REPÈRE DU RÉDACTEUR

  • Dans les soins primaires, les activités de contrôle de la qualité sont importantes pour fournir un feedback au médecin et améliorer la qualité des soins. Toutefois, la quantité de temps et d’énergie nécessaire pour effectuer l’extraction des données décourage souvent le médecin d’entreprendre de telles activités. Dans cette étude, les auteurs ont testé une stratégie d’échantillonnage séquentiel de patientes éligibles, dans l’espoir de rendre l’extraction des données plus facile et plus rapide, et ils ont ensuite comparé les résultats à ceux obtenus avec toutes les clientes d’une clinique.

  • Les auteurs ont trouvé une corrélation positive significative entre les taux calculés à partir des données des 2 approches différentes. La plupart des médecins ont trouvé que la technique d’échantillonnage séquentiel n’interférait pas avec l’activité normale de la clinique et près de la moitié d’entre eux ont dit qu’ils seraient disposés à entreprendre une autre vérification.

  • Même si certains médecins ontariens sont susceptibles de recevoir des incitatifs pour augmenter les taux de dépistage mammographique, les taux étaient les mêmes chez tous les médecins, qu’ils aient reçu ou non de tels incitatifs.

Footnotes

  • Contributors

    Dr Swanson, principal investigator and author, had the original idea and developed it with Dr Kaczorowski, co-investigator and statistician. Dr Swanson drafted the article, and Dr Kaczorowski reviewed, revised, and edited it. Both the authors agreed on the version submitted for publication.

  • Competing interests

    None declared

  • *Full text is available in English at www.cfp.ca.

  • This article has been peer reviewed.

  • Copyright© the College of Family Physicians of Canada

References

  1. ↵
    College of Family Physicians of CanadaStandards for accreditation of residency training programsMississauga, ONCollege of Family Physicians of Canada2006Available from: www.cfpc.ca/local/files/EducationAccessed 2008 Jun 5
  2. ↵
    LemelinJHoggWBaskervilleNEvidence to action: a tailored multifaceted approach to changing family physician practice patterns and improving preventive careCMAJ2001164675763
    OpenUrlAbstract/FREE Full Text
  3. ↵
    JamtvedtGYoungJMKristoffersenDTThomson O’BrienMAOxmanADAudit and feedback: effects on professional practice and health care outcomesCochrane Database Syst Rev20033CD000259
  4. ↵
    SwansonJGEdeyLKaczorowskiJQuality assurance through reflective change (QuARC): involving family medicine community-based preceptors in a continuous quality improvement programmeQual Prim Care200614315560
    OpenUrl
  5. ↵
    MorrisonBJCanadian Task Force on the Periodic Health ExaminationScreening for breast cancerCanadian guide to clinical preventive health careOttawa, ONHealth Canada199478895
  6. ↵
    College of Family Physicians of Canada, Canadian Medical Association, Royal College of Physicians and Surgeons of Canada2004 National Physician SurveyMississauga, ONCollege of Family Physicians of Canada2004Available from: www.nationalphysiciansurvey.ca/nps/2004_Survey/2004nps-e.aspAccessed 2008 Jun 5
  7. ↵
    AndrewsFJPain during mammography: implications for breast screening programmesAustralas Radiol20014521137
    OpenUrlCrossRefPubMed
  8. ↵
    MillerDMartinIHerbisonPInterventions for relieving the pain and discomfort of screening mammographyCochrane Database Syst Rev20024CD002942
View Abstract
PreviousNext
Back to top

In this issue

Canadian Family Physician: 54 (10)
Canadian Family Physician
Vol. 54, Issue 10
1 Oct 2008
  • Table of Contents
  • About the Cover
  • Index by author
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on The College of Family Physicians of Canada.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Can you use a sequential sample of patients as a substitute for a full practice audit?
(Your Name) has sent you a message from The College of Family Physicians of Canada
(Your Name) thought you would like to see the The College of Family Physicians of Canada web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Can you use a sequential sample of patients as a substitute for a full practice audit?
Graham Swanson, Janusz Kaczorowski
Canadian Family Physician Oct 2008, 54 (10) 1436-1437.e5;

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Respond to this article
Share
Can you use a sequential sample of patients as a substitute for a full practice audit?
Graham Swanson, Janusz Kaczorowski
Canadian Family Physician Oct 2008, 54 (10) 1436-1437.e5;
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • ABSTRACT
    • METHODS
    • RESULTS
    • DISCUSSION
    • Acknowledgment
    • Notes
    • Footnotes
    • References
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Patients eligible and referred for bariatric surgery in southeastern Ontario
  • PEER systematic review of randomized controlled trials
  • Electronic communication between family physicians and patients
Show more Research

Similar Articles

Subjects

  • Collection française
    • Résumés de recherche

Navigate

  • Home
  • Current Issue
  • Archive
  • Collections - English
  • Collections - Française

For Authors

  • Authors and Reviewers
  • Submit a Manuscript
  • Permissions
  • Terms of Use

General Information

  • About CFP
  • About the CFPC
  • Advertisers
  • Careers & Locums
  • Editorial Advisory Board
  • Subscribers

Journal Services

  • Email Alerts
  • Twitter
  • RSS Feeds

Copyright © 2021 by The College of Family Physicians of Canada

Powered by HighWire