Navigating Canada’s primary care crisis

Finding appropriate solutions through innovation

Interventions to address administrative burden can target individual categories, such as removing unnecessary steps or forms to simplify processes,^12-14 better and appropriate task delegation across health care team members (ie, people),^12,14,21 improving interoperability of digital tools and EMRs,^12-14,17 and revising system-level policies that contribute to burden.^12,14,17,22 Interventions that take a comprehensive approach by considering all these factors are more likely to be successful. Box 2 provides an example.^23-27

To realize the potential for digital technology to reduce administrative burden, Canadian provinces and territories need a set of strategies that include strong end-user input and co-design, a focus on usability, and infrastructure that enables a range of data collection techniques in an efficient and responsive manner. This approach should consider the entire primary care ecosystem when seeking to optimize, implement, and scale digital tools and should include observations to measure workflows in clinical settings, collaboration with a range of end users for timely feedback, and a usability laboratory where workflows can be measured and refined before implementation in clinics. These key considerations should be central to this process when combined with consistent and transparent communication channels with policy-makers, health care professionals (working within both primary and specialty care), patients, researchers, vendors, and other actors.

Innovative solutions in practice

An example of this approach is the Women’s College Hospital (WCH) Living Lab in Toronto, Ont, which is currently in development. In general, the Living Lab concept is well described in the literature as a form of open innovation where co-creation and research in naturalistic settings converge.²⁸ The WCH Living Lab draws from and extends pivotal work by Schuurman et al,²⁹ incorporating 3 core components for testing digital innovations (including 2 settings that capture unique information): an artificial setting called the WCH Virtual Care Lab, natural settings for observation (eg, observation in a primary care practice), and spread and scale embedded throughout (eg, facilitated through partnerships with stakeholders like Ontario Health).

Artificial setting. An artificial environment (ie, the WCH Virtual Care Lab) resembles a community doctor’s office equipped with clinical digital systems (eg, EMRs, digital tools, virtual care modalities). This artificial environment is designed to mimic real-world clinical settings while offering an adaptable, yet controlled, setting to address pressing research questions about the digitization of interventions. While not an original component of the model proposed by Schuurman et al,²⁹ the WCH Virtual Care Lab allows for timely and detailed analyses using methods such as time-motion studies, competitive analysis, and usability testing of various tools, including those that are not yet ready for real-world clinical use. Depending on what innovation is being tested (eg, a tool not yet used in practice, a tool implemented in select areas, a tool more broadly implemented), it will precede or occur in parallel with a natural setting.

Natural setting. Naturalistic observation is a key element of Living Lab infrastructure, capturing the “uncontrollable dynamics of everyday life”²⁹ within innovation contexts. This is particularly useful for understanding how digital innovation interacts with a given context and its implications (eg, exploring how a tool is embedded in and impacts a primary care physician’s workflow). Essential for selecting natural settings are pre-established relationships before data collection, and meeting operational conditions, such as site-specific agreements for funding and data sharing, to enable rapid EMR or administrative data collection. Optimally, partnerships with diverse primary care clinics across Ontario, representing varied geography, practice types, and patient demographic characteristics, allow for swift, representative data collection using multiple modalities (eg, interviews, surveys, EMRs, tool usage data) to address health system challenges.

Spread and scale. Spread and scale is embedded throughout all aspects of the Living Lab ecosystem through partnerships with policy-makers, end users, and people with lived experience. Ideally, partners are involved throughout all stages of the research process from study conceptualization, design, dissemination and implementation. This co-creation process helps to bridge the “know-do” gap that can limit the uptake, reach, impact, and sustainability of research outputs. Integral to establishing such partnerships is the necessary commitment to relationship development over time, including building and sustaining trust for effective collaboration (Figure 1).²⁹

• Download figure
• Open in new tab

Figure 1.

Women’s College Hospital Living Lab: The Living Lab includes 3 core components: 1) an artificial setting (eg, a lab environment), 2) a natural setting (eg, real-world observation in a primary care clinic), and 3) considerations for spread and scale (eg, working directly with policy-makers, end users, and people with lived experience). Across all components are use of multiple methods (eg, interviews and usage analytics) and the involvement of multiple stakeholders (eg, end users and researchers) throughout research processes (ideally from conception to dissemination) in a user-centric manner (ie, end users are actively involved in the process as co-producers). The iterative interactions within and across these components are integral to the Living Lab approach, as depicted by the arrows.

Case example: evaluating artificial intelligence (AI) scribes through the Living Lab at WCH

We recently applied the Living Lab approach in a provincial evaluation of AI scribes,³⁰ which are rapidly emerging digital health tools designed to reduce administrative burden in primary care.^31,32 The evaluation started in October 2023, at a time when AI scribe research was limited, making it challenging to establish a clear value proposition despite the technology’s rapid market growth.

Artificial setting. The evaluation began in the WCH Virtual Care Lab, a controlled laboratory environment where simulated clinical encounters between standardized patients and primary care professionals (ie, family physicians and nurse practitioners) were conducted. This setup allowed for head-to-head comparisons of 6 AI scribes across multiple domains, including usability, effectiveness, and documentation accuracy and quality. These 6 were the only AI scribe vendors with Canadian clients at the time, and were chosen for their compliance with Ontario’s Personal Health Information Protection Act (PHIPA) and adherence to best practices in data privacy and security (ie, ensuring personal health information [PHI] was collected, used, and disclosed only for authorized purposes with valid consent; implementing safeguards to prevent unauthorized access; and ensuring proper retention and disposal timelines are followed). Vendors were also required to store and process data within Canada or, if stored outside, provide proper notice to health information custodians to ensure regulatory compliance and notification obligations. While there was some variation in performance, the simulations demonstrated that most AI scribes could produce high-quality medical notes that required minimal editing by primary care professionals, and significantly reduced documentation time by nearly 70% (P<.001).³⁰ The artificial environment allowed for a comprehensive and standardized evaluation approach, ensuring existing pain points, potential gains, and necessary adjustments to AI scribes were identified by key users (eg, primary care professionals) before broad implementation in practice.³⁰ The data collected in the WCH Virtual Care Lab not only provided support for further testing (ie, investing more resources in testing in a natural setting), but also indicated where to focus time and resources when implementing tools in practice. For example, from the WCH Virtual Care Lab, we shortlisted 3 of the 6 scribes for implementation testing in the natural setting based on their performance in head-to-head comparisons.

Natural settings. Building on these promising results, the evaluation moved into real-world clinical settings. The top 3 performing AI scribes were selected for implementation in the practices of more than 150 primary care professionals across Ontario for a 3-month period. By leveraging existing partnerships and building new collaborations, operational infrastructure and necessary data-sharing agreements were quickly set up to enable swift implementation and evaluation. The findings from real-world observation complemented those observed in the artificial setting, with family physicians and nurse practitioners reporting a 3-hour reduction per week in after-hours documentation (P<.05). Family physicians and nurse practitioners also reported reductions in administrative burden and cognitive load alongside improvements in job satisfaction, professional fulfillment, and work-life balance, based on both qualitative interviews and quantitative scales measuring satisfaction and work-life balance. Importantly, these changes from implementing AI scribes could increase system capacity both directly, by freeing up more time for direct patient care, and indirectly, by reducing burnout and increasing retention, which helps maintain a stable workforce in primary care.

Spread and scale. Moving from an artificial to a natural setting allowed us to identify, refine, and validate the value proposition of AI scribes while using a multi-method approach to pinpoint key factors for sustainability, spread, and scale. These factors included the need for data privacy and security regulations, seamless EMR integration, enhanced capabilities for equitable access, personalized workflows, and funding support. Collaborating with key stakeholders (ie, policy-makers, health system leaders, implementation specialists, health care professionals, and patients) created a dynamic environment for knowledge sharing and identifying best practices for implementing AI scribes. This collaboration also led to the development of recommendations for the spread and scale of AI scribes across Ontario, and a provincial request for proposals for procurement was released. Findings from this early evaluation using the Living Lab approach are published on the WCH website and aim to guide future policy and purchasing, and have informed procurement decisions for health care organizations in Ontario and other health systems (including the procurement of 10,000 licences across every region by Canada Health Infoway).³⁰ By embedding this work in the Living Lab concept, we were able to continuously evaluate the effectiveness of AI scribes against the needs and workflows of primary care. This ensured a thorough and practical assessment of their impact on reducing administrative burden and improving primary care professional satisfaction.