Questionnaire instrument development in primary health care research

Questionnaires in primary care research

Questionnaires are a frequently used instrument in primary care research that aim to obtain information relevant to 1 or more prespecified research questions. For example, in 2014, the Canadian Institutes of Health Research initiated a pan-Canadian strategy for the promotion and support of patient-oriented research.² Patient engagement is an integral element in the implementation of all components of the Strategy for Patient-Oriented Research. Patient questionnaires are, therefore, a central approach for obtaining relevant input from patients in this domain of primary care research.

When developing and designing questionnaires, establishment of adequate content and construct validity is crucial.³ Reliable assessment of these 2 aspects requires a sufficient sample size of completed questionnaires from 1 or more pilot inquiries. The suggested minimum number of samples in the literature ranges between 100 and 250.⁴ However, it can be difficult, time-consuming, and expensive to obtain these relatively large numbers of study participants.

The requirement of relatively large sample sizes for evaluating instruments indicates the need for new, more efficient assessment approaches. Modern Bayesian methods offer promising solutions to this dilemma, as they allow for the incorporation of prior information to increase the efficiency of the conventional methods used for instrument validation.⁵ The holistic Bayesian approach integrates prior knowledge from experts or preliminary information with the data being collected in order to estimate statistical parameters of interest, such as item-to-domain correlations or factor loadings. If reliable prior information is available, Bayesian methods effectively improve the precision of the obtained parameter estimates.

Identifying gaps and solutions

Despite important recent methodologic advancements in the theory of questionnaire development using Bayesian methods, present applications in the primary care context reveal the need for further development of credible and accessible tools for researchers.⁶

Barriers exist to translating methodologic advancements into practice. Two considerable obstacles are linguistic and conceptual incompatibilities between the researchers who develop the statistical methods for questionnaire validation and the investigators who create and use questionnaires. For example, while latent factors (domains) are a key element in the statistical theory of questionnaire validation, practitioners often pursue a rather item-centred approach, followed by a post hoc, sometimes data-driven, classification of questionnaire items into instrument subscales representing the various domains. To better understand the nature of conceptual misalignments, more research assessing knowledge, attitudes, and practices on both sides is required. Such information will be essential to identifying gaps and related solutions that will enable better research practice in the context of developing and validating questionnaires.

One possible solution to the current underuse of Bayesian methods is the creation and promotion of educational modules, alongside user-friendly software. This will allow for the application of these methods for the purpose of instrument development and validation. We suggest, in particular, that the development and validation phase should be conducted in a complementary way, so that each phase considers the purpose of the other phase. Such a coherent approach must respect linguistic misalignments, and particular care is required when developing the educational modules that seek to resolve the potentially unclear procedure of specifying prior knowledge. We therefore propose to increase the use of participatory approaches when creating educational components by including representatives from both sides: the instrument developers and the validation experts. For example, there is a gap in knowledge and understanding about how domain experts in primary care can better contribute to the proper implementation of a Bayesian framework—ie, how professional expertise can be translated into valuable prior information to be incorporated when validating questionnaires using Bayesian inference methods.

Establishing new ways of learning from each other is a key step in improving knowledge translation and bridging the gap between available effective statistical methods and their underuse in primary health care research. Better understanding of the principle intentions of the questionnaire designers and the domain experts, and related expectations of the scope and usefulness of the developed instruments, is needed.

If the creation and design of modern research tools for questionnaire validation includes close engagement between primary care researchers, statistical experts, and patients, use and knowledge of modern Bayesian methods will improve and eventually lead to more time- and cost-efficient research in the field of primary care.