Recognition of inherent biases in administrative data

Sources of bias in administrative data

Administrative data are usually collected routinely by hospitals, pharmacies, laboratories, and health authorities, which are products of institutional oppression and colonialism, and thus the data reflect the structural biases inherent in those systems. To understand these data fully, we need to understand their provenance. While one dictionary initially defines provenance as “source,” its second definition adds depth to our understanding and its use in this context: “the history of ownership of a valued object or work of art or literature.”² Using the word provenance suggests we need to know not just the source of data, but also the circumstances of their collection. In Canada, understanding the history of collection and ownership of data is particularly important considering past and ongoing colonial violence, which is itself embedded in the health care system.

Population-based analyses using health system data include in the numerator only those who have accessed the health care system. While financial barriers to care are less common in countries such as Canada that have universal health care coverage, these barriers still exist. There might be costs associated with transportation, child care, or lack of benefits such as sick leave at work, or even with the price of medication. Access is particularly challenging in many rural and remote communities owing to the distances people must travel for care. Accessibility challenges are not limited to physical accessibility and include difficulties in accessing information and other resources. For example, language translation services are not routinely available at most health care sites. All of these factors influence the circumstances of data collection and embed systemic biases in data that are not initially apparent.

Further, Canadians who have had negative experiences when seeking care—such as racism, gender discrimination, ableism, and other demeaning attitudes at points of care—might not seek care as often or as quickly. These negative experiences and resultant mistrust of the health care system are of particular relevance to Indigenous peoples.^3,4 Some biases are systemic owing to provider ignorance and attitude. When providers do not understand the cultural norms of patients, they might have communication challenges that lead to misunderstandings or failure to make correct diagnoses. We cannot assume that because a health service was accessed that the service was appropriate and will lead to the desired health outcome⁵; this disconnect is especially concerning with respect to mental illness, as both communication and cultural awareness are crucial components of the diagnostic process.

Implications for research using administrative data

Although some see these limitations as fatal flaws,⁶ research using administrative data still provides critical information. It is the responsibility of the research team to acknowledge and mitigate these limitations, just as they must do with other data sources. Seeking other data sources to confirm the findings and comparing the results with those from studies using other methods are appropriate mitigations to use where possible.^7,8 Key differences may result from limitations in 1 or more of the studies being compared, and it is important to explore how those limitations might have affected results. Using within-group analysis or identifying different comparison groups might also help bring intersectional inequities to light.⁹

Mitigating the impact of biases in administrative data can mean including people with the greatest risk of marginalization and discrimination as members of the research team, training researchers to recognize inequities and their own potential biases, and creating data systems and processes that are inclusive, diverse, equitable, and accessible. The aphorism “the results speak for themselves” does not recognize the critical importance of ensuring a valid and culturally appropriate interpretation of results that accounts for biases.