Skip to main content

Main menu

  • Home
  • Articles
    • Current
    • Archive
    • Supplemental Issues
    • Collections - French
    • Collections - English
  • Info for
    • Authors & Reviewers
    • Submit a Manuscript
    • Advertisers
    • Careers & Locums
    • Subscribers
    • Permissions
  • About CFP
    • About CFP
    • About the CFPC
    • Editorial Advisory Board
    • Terms of Use
    • Contact Us
  • Feedback
    • Feedback
    • Rapid Responses
    • Most Read
    • Most Cited
    • Email Alerts
  • Blogs
    • Latest Blogs
    • Blog Guidelines
    • Directives pour les blogues
  • Mainpro+ Credits
    • About CFP Mainpro+
    • Member Login
    • Instructions
  • Other Publications
    • http://www.cfpc.ca/Canadianfamilyphysician/
    • https://www.cfpc.ca/Login/
    • Careers and Locums

User menu

  • My alerts

Search

  • Advanced search
The College of Family Physicians of Canada
  • Other Publications
    • http://www.cfpc.ca/Canadianfamilyphysician/
    • https://www.cfpc.ca/Login/
    • Careers and Locums
  • My alerts
The College of Family Physicians of Canada

Advanced Search

  • Home
  • Articles
    • Current
    • Archive
    • Supplemental Issues
    • Collections - French
    • Collections - English
  • Info for
    • Authors & Reviewers
    • Submit a Manuscript
    • Advertisers
    • Careers & Locums
    • Subscribers
    • Permissions
  • About CFP
    • About CFP
    • About the CFPC
    • Editorial Advisory Board
    • Terms of Use
    • Contact Us
  • Feedback
    • Feedback
    • Rapid Responses
    • Most Read
    • Most Cited
    • Email Alerts
  • Blogs
    • Latest Blogs
    • Blog Guidelines
    • Directives pour les blogues
  • Mainpro+ Credits
    • About CFP Mainpro+
    • Member Login
    • Instructions
  • RSS feeds
  • Follow cfp Template on Twitter
OtherCollege

Questionnaire instrument development in primary health care research

A plea for the use of Bayesian inference

Hao Zhang and Tibor Schuster
Canadian Family Physician September 2018, 64 (9) 699-700;
Hao Zhang
Doctoral candidate in the Department of Family Medicine at McGill University in Montreal, Que.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tibor Schuster
Assistant Professor in the Department of Family Medicine at McGill University.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • eLetters
  • Info & Metrics
  • PDF
Loading

Careful instrument development and questionnaire validation are essential to ensuring the validity and reliability of self-reported outcomes in primary care research. In this article, we describe common issues with small sample sizes that arise in questionnaire development and we discuss the demand for more efficient statistical analysis approaches. We review the potential of modern Bayesian methods to increase parameter estimation efficiency and propose their application in the context of primary health care research.

Primary health care is the fundament of our health care system, covering comprehensive medical needs for the entire population from general child health to care of the elderly.1 It acts as the first point of contact for the medical needs of the general public, offering integrated and equally accessible health care services to all subgroups in society. An efficient and effective primary health care system facilitates diagnosis of common diseases and coordinates appropriate referrals to secondary or specialist care, offering timely treatment and prevention at early stages at relatively low cost. The system also offers a platform to deliver personalized care with accumulating effects that potentially convey better long-term health outcomes for individuals and higher patient satisfaction.1 A well sustained primary health care system is founded upon partnerships with patients and is guided by decisions made by family doctors, other health care providers, and health policy makers. Decision making by practitioners, grounded in the best available research evidence, is therefore crucial for both good patient outcomes and the viability of the primary health care system. The best available evidence for informing practice makes use of the most up-to-date research findings in a cost-efficient and timely manner.

Questionnaires in primary care research

Questionnaires are a frequently used instrument in primary care research that aim to obtain information relevant to 1 or more prespecified research questions. For example, in 2014, the Canadian Institutes of Health Research initiated a pan-Canadian strategy for the promotion and support of patient-oriented research.2 Patient engagement is an integral element in the implementation of all components of the Strategy for Patient-Oriented Research. Patient questionnaires are, therefore, a central approach for obtaining relevant input from patients in this domain of primary care research.

When developing and designing questionnaires, establishment of adequate content and construct validity is crucial.3 Reliable assessment of these 2 aspects requires a sufficient sample size of completed questionnaires from 1 or more pilot inquiries. The suggested minimum number of samples in the literature ranges between 100 and 250.4 However, it can be difficult, time-consuming, and expensive to obtain these relatively large numbers of study participants.

The requirement of relatively large sample sizes for evaluating instruments indicates the need for new, more efficient assessment approaches. Modern Bayesian methods offer promising solutions to this dilemma, as they allow for the incorporation of prior information to increase the efficiency of the conventional methods used for instrument validation.5 The holistic Bayesian approach integrates prior knowledge from experts or preliminary information with the data being collected in order to estimate statistical parameters of interest, such as item-to-domain correlations or factor loadings. If reliable prior information is available, Bayesian methods effectively improve the precision of the obtained parameter estimates.

Identifying gaps and solutions

Despite important recent methodologic advancements in the theory of questionnaire development using Bayesian methods, present applications in the primary care context reveal the need for further development of credible and accessible tools for researchers.6

Barriers exist to translating methodologic advancements into practice. Two considerable obstacles are linguistic and conceptual incompatibilities between the researchers who develop the statistical methods for questionnaire validation and the investigators who create and use questionnaires. For example, while latent factors (domains) are a key element in the statistical theory of questionnaire validation, practitioners often pursue a rather item-centred approach, followed by a post hoc, sometimes data-driven, classification of questionnaire items into instrument subscales representing the various domains. To better understand the nature of conceptual misalignments, more research assessing knowledge, attitudes, and practices on both sides is required. Such information will be essential to identifying gaps and related solutions that will enable better research practice in the context of developing and validating questionnaires.

One possible solution to the current underuse of Bayesian methods is the creation and promotion of educational modules, alongside user-friendly software. This will allow for the application of these methods for the purpose of instrument development and validation. We suggest, in particular, that the development and validation phase should be conducted in a complementary way, so that each phase considers the purpose of the other phase. Such a coherent approach must respect linguistic misalignments, and particular care is required when developing the educational modules that seek to resolve the potentially unclear procedure of specifying prior knowledge. We therefore propose to increase the use of participatory approaches when creating educational components by including representatives from both sides: the instrument developers and the validation experts. For example, there is a gap in knowledge and understanding about how domain experts in primary care can better contribute to the proper implementation of a Bayesian framework—ie, how professional expertise can be translated into valuable prior information to be incorporated when validating questionnaires using Bayesian inference methods.

Establishing new ways of learning from each other is a key step in improving knowledge translation and bridging the gap between available effective statistical methods and their underuse in primary health care research. Better understanding of the principle intentions of the questionnaire designers and the domain experts, and related expectations of the scope and usefulness of the developed instruments, is needed.

If the creation and design of modern research tools for questionnaire validation includes close engagement between primary care researchers, statistical experts, and patients, use and knowledge of modern Bayesian methods will improve and eventually lead to more time- and cost-efficient research in the field of primary care.

Acknowledgments

Ms Zhang was funded by the Quebec SPOR SUPPORT (Strategy for Patient-Oriented Research Support for People and Patient-Oriented Research and Trials) Unit and grants from the Canadian Institutes of Health Research. Dr Schuster is supported by funds obtained through a Canada Research Chair in Biostatistical Methods for Primary Health Care Research.

Notes

Hypothesis is a quarterly series in Canadian Family Physician (CFP), coordinated by the Section of Researchers of the College of Family Physicians of Canada. The goal is to explore clinically relevant research concepts for all CFP readers. Submissions are invited from researchers and nonresearchers. Ideas or submissions can be submitted online at mc.manuscriptcentral.com/cfp or through the CFP website www.cfp.ca under “Authors and Reviewers.”

Footnotes

  • Competing interests

    None declared

  • Copyright© the College of Family Physicians of Canada

References

  1. 1.↵
    1. Starfield B,
    2. Shi L,
    3. Macinko J
    . Contribution of primary care to health systems and health. Milbank Q 2005;83(3):457-502.
    OpenUrlCrossRefPubMed
  2. 2.↵
    1. Canadian Institutes of Health Research [website].
    SPOR SUPPORT Units. Ottawa, ON: Canadian Institutes of Health Research; 2018. Available from: www.cihr-irsc.gc.ca/e/45859.html. Accessed 2018 Jun 2.
  3. 3.↵
    1. Haynes SN,
    2. Richard DCS,
    3. Kubany ES
    . Content validity in psychological assessment: a functional approach to concepts and methods. Psychol Assess 1995;7(3):238-47.
    OpenUrlCrossRef
  4. 4.↵
    1. MacCallum RC,
    2. Widaman KF,
    3. Zhang S,
    4. Hong S
    . Sample size in factor analysis. Psychol Methods 1999;4(1):84-99.
    OpenUrlCrossRef
  5. 5.↵
    1. Garrard L,
    2. Price LR,
    3. Bott MJ,
    4. Gajewski BJ
    . A novel method for expediting the development of patient-reported outcome measures and an evaluation across several populations. Appl Psychol Meas 2016;40(7):455-68. Epub 2016 Jun 14.
    OpenUrlCrossRef
  6. 6.↵
    1. Zhang H,
    2. Sourial N,
    3. Vedel I,
    4. Schuster T
    . A web application for frequentist and Bayesian content validation of questionnaires in patient-oriented research; Presented at: 45th North American Primary Care Research Group Annual Meeting; 2017 Nov 17–21; Montreal, QC.
PreviousNext
Back to top

In this issue

Canadian Family Physician: 64 (9)
Canadian Family Physician
Vol. 64, Issue 9
1 Sep 2018
  • Table of Contents
  • About the Cover
  • Index by author
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on The College of Family Physicians of Canada.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Questionnaire instrument development in primary health care research
(Your Name) has sent you a message from The College of Family Physicians of Canada
(Your Name) thought you would like to see the The College of Family Physicians of Canada web site.
Citation Tools
Questionnaire instrument development in primary health care research
Hao Zhang, Tibor Schuster
Canadian Family Physician Sep 2018, 64 (9) 699-700;

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Respond to this article
Share
Questionnaire instrument development in primary health care research
Hao Zhang, Tibor Schuster
Canadian Family Physician Sep 2018, 64 (9) 699-700;
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Questionnaires in primary care research
    • Identifying gaps and solutions
    • Acknowledgments
    • Notes
    • Footnotes
    • References
  • Info & Metrics
  • eLetters
  • PDF

Related Articles

  • No related articles found.
  • Scopus
  • PubMed
  • Google Scholar

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

College

  • Évaluer des patients extrêmement âgés confinés à domicile et en sévère perte d’autonomie
  • Assessing extreme elderly homebound patients with severe loss of autonomy
  • Ce qui m’a attirée vers la médecine de famille continue de rendre notre domaine spécial aujourd’hui
Show more College

Hypothesis

  • Qualitative approaches
  • Shared vision for primary care delivery and research in Canada and the United States
Show more Hypothesis

Similar Articles

Navigate

  • Home
  • Current Issue
  • Archive
  • Collections - English
  • Collections - Française

For Authors

  • Authors and Reviewers
  • Submit a Manuscript
  • Permissions
  • Terms of Use

General Information

  • About CFP
  • About the CFPC
  • Advertisers
  • Careers & Locums
  • Editorial Advisory Board
  • Subscribers

Journal Services

  • Email Alerts
  • Twitter
  • RSS Feeds

Copyright © 2019 by The College of Family Physicians of Canada

Powered by HighWire