Careful instrument development and questionnaire validation are essential to ensuring the validity and reliability of self-reported outcomes in primary care research. In this article, we describe common issues with small sample sizes that arise in questionnaire development and we discuss the demand for more efficient statistical analysis approaches. We review the potential of modern Bayesian methods to increase parameter estimation efficiency and propose their application in the context of primary health care research.
Primary health care is the fundament of our health care system, covering comprehensive medical needs for the entire population from general child health to care of the elderly.1 It acts as the first point of contact for the medical needs of the general public, offering integrated and equally accessible health care services to all subgroups in society. An efficient and effective primary health care system facilitates diagnosis of common diseases and coordinates appropriate referrals to secondary or specialist care, offering timely treatment and prevention at early stages at relatively low cost. The system also offers a platform to deliver personalized care with accumulating effects that potentially convey better long-term health outcomes for individuals and higher patient satisfaction.1 A well sustained primary health care system is founded upon partnerships with patients and is guided by decisions made by family doctors, other health care providers, and health policy makers. Decision making by practitioners, grounded in the best available research evidence, is therefore crucial for both good patient outcomes and the viability of the primary health care system. The best available evidence for informing practice makes use of the most up-to-date research findings in a cost-efficient and timely manner.
Questionnaires in primary care research
Questionnaires are a frequently used instrument in primary care research that aim to obtain information relevant to 1 or more prespecified research questions. For example, in 2014, the Canadian Institutes of Health Research initiated a pan-Canadian strategy for the promotion and support of patient-oriented research.2 Patient engagement is an integral element in the implementation of all components of the Strategy for Patient-Oriented Research. Patient questionnaires are, therefore, a central approach for obtaining relevant input from patients in this domain of primary care research.
When developing and designing questionnaires, establishment of adequate content and construct validity is crucial.3 Reliable assessment of these 2 aspects requires a sufficient sample size of completed questionnaires from 1 or more pilot inquiries. The suggested minimum number of samples in the literature ranges between 100 and 250.4 However, it can be difficult, time-consuming, and expensive to obtain these relatively large numbers of study participants.
The requirement of relatively large sample sizes for evaluating instruments indicates the need for new, more efficient assessment approaches. Modern Bayesian methods offer promising solutions to this dilemma, as they allow for the incorporation of prior information to increase the efficiency of the conventional methods used for instrument validation.5 The holistic Bayesian approach integrates prior knowledge from experts or preliminary information with the data being collected in order to estimate statistical parameters of interest, such as item-to-domain correlations or factor loadings. If reliable prior information is available, Bayesian methods effectively improve the precision of the obtained parameter estimates.
Identifying gaps and solutions
Despite important recent methodologic advancements in the theory of questionnaire development using Bayesian methods, present applications in the primary care context reveal the need for further development of credible and accessible tools for researchers.6
Barriers exist to translating methodologic advancements into practice. Two considerable obstacles are linguistic and conceptual incompatibilities between the researchers who develop the statistical methods for questionnaire validation and the investigators who create and use questionnaires. For example, while latent factors (domains) are a key element in the statistical theory of questionnaire validation, practitioners often pursue a rather item-centred approach, followed by a post hoc, sometimes data-driven, classification of questionnaire items into instrument subscales representing the various domains. To better understand the nature of conceptual misalignments, more research assessing knowledge, attitudes, and practices on both sides is required. Such information will be essential to identifying gaps and related solutions that will enable better research practice in the context of developing and validating questionnaires.
One possible solution to the current underuse of Bayesian methods is the creation and promotion of educational modules, alongside user-friendly software. This will allow for the application of these methods for the purpose of instrument development and validation. We suggest, in particular, that the development and validation phase should be conducted in a complementary way, so that each phase considers the purpose of the other phase. Such a coherent approach must respect linguistic misalignments, and particular care is required when developing the educational modules that seek to resolve the potentially unclear procedure of specifying prior knowledge. We therefore propose to increase the use of participatory approaches when creating educational components by including representatives from both sides: the instrument developers and the validation experts. For example, there is a gap in knowledge and understanding about how domain experts in primary care can better contribute to the proper implementation of a Bayesian framework—ie, how professional expertise can be translated into valuable prior information to be incorporated when validating questionnaires using Bayesian inference methods.
Establishing new ways of learning from each other is a key step in improving knowledge translation and bridging the gap between available effective statistical methods and their underuse in primary health care research. Better understanding of the principle intentions of the questionnaire designers and the domain experts, and related expectations of the scope and usefulness of the developed instruments, is needed.
If the creation and design of modern research tools for questionnaire validation includes close engagement between primary care researchers, statistical experts, and patients, use and knowledge of modern Bayesian methods will improve and eventually lead to more time- and cost-efficient research in the field of primary care.
Acknowledgments
Ms Zhang was funded by the Quebec SPOR SUPPORT (Strategy for Patient-Oriented Research Support for People and Patient-Oriented Research and Trials) Unit and grants from the Canadian Institutes of Health Research. Dr Schuster is supported by funds obtained through a Canada Research Chair in Biostatistical Methods for Primary Health Care Research.
Notes
Hypothesis is a quarterly series in Canadian Family Physician (CFP), coordinated by the Section of Researchers of the College of Family Physicians of Canada. The goal is to explore clinically relevant research concepts for all CFP readers. Submissions are invited from researchers and nonresearchers. Ideas or submissions can be submitted online at mc.manuscriptcentral.com/cfp or through the CFP website www.cfp.ca under “Authors and Reviewers.”
Footnotes
Competing interests
None declared
- Copyright© the College of Family Physicians of Canada