Abstract
Objective To design reliable survey instruments to evaluate needs and expectations for provision of women's health services in rural communities in British Columbia (BC). These tools will aim to plan programming for, and evaluate effectiveness of, a women's health enhanced skills residency program at the University of British Columbia.
Design A qualitative design that included administration of written surveys and on-site interviews in several rural communities.
Setting Three communities participated in initial questionnaire and interview administration. A fourth community participated in the second interview iteration. Participating communities did not have obstetrician-gynecologists but did have hospitals capable of supporting outpatient specialized women's health procedural care.
Participants Community physicians, leaders of community groups serving women, and allied health providers, in Vancouver Island, Southeast Interior BC, and Northern BC.
Methods Two preliminary questionnaires were developed to assess local specialized women's health services based on the curriculum of the enhanced skills training program; one was designed for physicians and the other for women's community group leaders and aboriginal health and community group leaders. Interview questions were designed to ensure the survey could be understood and to identify important areas of women's health not included on the initial questionnaires. Results were analyzed using quantitative and qualitative methods, and a second draft of the questionnaires was developed for a second iteration of interviews.
Main findings Clarity and comprehension of questionnaires were good; however, nonphysician participants answered that they were unsure on many questions pertaining to specific services. Topics identified as important and missing from questionnaires included violence and mental health. A second version of the questionnaires was shown to have addressed these concerns.
Conclusion Through iterations of pilot testing, we created 2 validated survey instruments for implementation as a component of program evaluation. Testing in remote locations highlighted unique rural concerns, such that University of British Columbia health care professional training will now better serve BC community needs.
Women living in rural Canada have a higher allcause mortality rate; higher rates of cervical cancer mortality, teen pregnancy, and domestic violence; and lower rates of Papanicolaou and mammography screening than their urban counterparts. Women in rural areas also have lower education levels and lower mean income, making travel to access specialist services more burdensome.1 In 2004, when 21% of Canada's population was living in rural areas, only 16% of family physicians and only 3% of obstetrician-gynecologists practised there.2 These numbers are similar to those in British Columbia (BC).3 The same study showed that family physicians in rural areas were more likely to have expanded scopes of practice than their urban counterparts were. The burden therefore falls on the shoulders of rural family physicians to manage women's health issues within their communities.
A new, evidence-based, objective-directed Enhanced Skills in Specialized Women's Health Residency (ESSWHR) program for rural family physicians has been introduced at the University of British Columbia (UBC). This 3-month program targets family physicians in rural practice or recent graduates. Previous programs have not been explicitly evidence-based or evaluated with structured objectives; neither have they assessed community needs before and after a physician begins his or her practice in a community or satisfaction with the quality of family practice skills delivered. The primary objective of this program is to address the health care needs of women in rural BC; an important determinant of success of the program is accurate needs assessments.
This study aimed to develop and validate questionnaires for implementation in each rural community where a physician will practise after completing the ESSWHR. The questionnaires developed within this study will be used before and after ESSWHR program completion to assess, within a target community, the expectations and needs for specialized women's health services before a physician has undertaken the training, and the effectiveness of meeting these expectations and needs after the trained physician has returned to the community. This external assessment will both inform and direct elements of the training to be undertaken, and complement the usual internal program evaluation. The results of both evaluations will be used to iteratively improve the curriculum, and thus to better meet the health care needs of BC's rural women and families.
METHODS
Study design
This qualitative study included administration of surveys along with face-to-face interviews in several rural communities. The survey and interview responses were used to amend the questionnaires for a smaller second iteration of testing.
Questionnaire development
The questionnaires were designed with the help of the ESSWHR program directors to develop appropriate content, using standard approaches4 and with the assistance of a statistician. Two preliminary questionnaires were developed based on the curriculum of the ESSWHR program— one was administered to physicians in the community and the other was administered to women's community group leaders and aboriginal health providers or leaders (nonphysicians). Questions for physicians focused on the availability of 10 specific health services. In addition to these questions, nonphysicians were asked about access to 8 additional health services in their communities using a Likert scale measuring agreement. Both questionnaires concluded with 3 summary questions assessing, on a Likert scale, agreement with perceived adequacy of specialized women's health services in the community.
Interview script development
A semistructured interview guide was developed to ensure the surveys could be understood and to identify important women's health services not already covered in the questionnaires. In addition, physician interview participants were asked if they believed that the 8 extra questions included on the nonphysician questionnaire should also be included on the physician questionnaire.
Participant selection and recruitment
Four communities in BC were selected for participation in the study. The 2 inclusion criteria were lack of a specialist in women's health (obstetrician-gynecologist) and presence of a clinic or hospital facility capable of supporting outpatient procedural care by a family physician practising specialized skills in women's health (ie, the presence of a procedure room, instrument sterilization capability, potential access to an ultrasound machine, and 24-hour emergency services). The study communities were selected from different regions of the province to better reflect the diversity of services and needs across BC; these regions included Vancouver Island, Southeast Interior BC, and Northern BC.
Our sample at each site included 3 groups: community physicians, leaders of community groups serving women, and leaders or health care providers for aboriginal women in the community. By selecting participants who worked and interacted with women of the community in different capacities and settings, we aimed to capture a representative sample without surveying individual female patients, as a larger population-based sample would not have been feasible for iterative program evaluation.
Through snowball sampling, which involves using acquaintances and contacts of initial participants to identify further potential participants, we contacted as many people as possible who met these criteria. Contact information for physicians was obtained through the College of Physicians and Surgeons of British Columbia directory. Women's groups, allied health professionals serving women, and aboriginal leaders were identified through medical clinics, public health offices, and the non-profit health care organization Options for Sexual Health, if a branch existed in the community. Additional physicians were contacted with the help of BC Women's Hospital and Health Centre Aboriginal Health Program staff. Our sample size ranged from 6 to 10 individuals per community. This was determined by the total number of people we were able to identify and contact who met our inclusion criteria and who agreed to participate. Therefore, the sample size was larger for communities with larger populations and more health and social services for women.
Data collection
Contact was initially made by faxing the study cover letter and consent forms. Interviews were scheduled by telephone. The researcher (M.G.) visited each community for 2 to 3 days between December 2010 and March 2011. Surveys were distributed at the time of the interview, and consent was obtained in person. The interviews were conducted in private locations at clinic or program offices, and were recorded with a digital recorder. After signing a confidentiality agreement, a professional transcriptionist transcribed the interviews verbatim.
Data analysis
A basic quantitative analysis of categorical survey responses was done by a statistician, including frequency tables and levels of agreement. Survey responses were qualitatively analyzed with the help of an experienced qualitative researcher. Interview transcriptions were analyzed by question, with recurring topics grouped according to themes identified. Because the data set was conducive to manual review, a software program for qualitative analysis was not used. Collectively, an iterative mode of interpretation was applied throughout the qualitative analysis process that reflected on the data collected. The aim of analysis was to identify gaps in information, using concepts and frames to examine the data with respect to the objective of the questions. Categories and themes were broadly generated initially and then later checked against the data to see if they fit. A summary of generalized themes covering a range of topics was used with feedback and suggestions to confirm issues and perceivable gaps in the data to inform questionnaire revisions.5
Questionnaire revisions
A second version of each questionnaire was developed according to our findings. Questions were altered if multiple participants asked for clarification as they were filling out the questionnaire. Questions were removed or reworded if 50% or more of the nonphysician participants responded that they were unsure. No topics were removed entirely from the questionnaires, as the physician version included all specialized services. A group of questions was added to the physician survey that had originally been present only on the nonphysician survey. These amended questionnaires were used in the second iteration of interviews in the fourth community in Southeast Interior BC.
Ethics
Ethics approval for this study was obtained from the UBC Children's and Women's Health Centre of British Columbia Research Ethics Board.
FINDINGS
Demographic characteristics
Population in the 4 communities ranged from 2648 to 7271.6Table 1 describes the characteristics of the communities. The nonphysician participants included public and community health nurses (n = 9), women's centre or safe-shelter staff (n = 6), an aboriginal community health advocate (n = 1), mental health counselors (n = 3), a nurse practitioner for aboriginal health (n = 1), and pregnancy or childhood development outreach coordinators (n = 3). Of the participants, 55% had 5 or more years of experience in the community.
Survey responses
Responses to the physician questions pertaining to service availability are presented in Table 2. Nearly half of questions were answered differently by the 2 physicians surveyed from each community, primarily because of the option to clarify whether local physicians or visiting specialists were providing services (Table 3). Responses to the nonphysician questions pertaining to access to services are presented in Table 4. In the first iteration of the nonphysician questionnaire, at least 50% of participants answered that they were unsure on 9 of 18 questions about the availability of specific services (Table 5); thus, language corrections and clarification were integrated into the second iteration. Items that were identified as needing clarification are listed in Box 1. Although there was no statistical significance, respondents' perceived adequacy of women's health services correlated with the community's availability score as calculated from the physician questionnaire responses (Table 6).
Physician responses about service availability: Responses were provided by 2 physicians in each community.
Number of questions answered differently by the 2 physicians surveyed for each community: N = 12 questions.
Services on the first questionnaire for which nonphysicians answered they were unsure about availability: N = 20.
Items requiring clarification
Physician questionnaire
-
What is your role in the medical community? (eg, a hospital or leadership appointment)
-
Has at least one person completed the SAS training course? (referring to the BC Women's Hospital and Health Centre provincial training course in sexual assault services)
Nonphysician questionnaire
-
Birth control counseling and management
-
Management of pelvic pain
-
Management of birth control around menopause
-
Treatment of urinary incontinence
-
Biopsy of skin lesions around the vagina
-
Person with specialized medical training to help women after sexual assault
-
Treatment of miscarriage with medications
-
Biopsy of breast lumps
Respondents' perceived adequacy of women's health services with each community's availability score as calculated from physician questionnaire responses: N = 26 respondents.
Interview themes
Themes identified for the meaning of the term good access
Participants were asked what the statement “Women have good access to the following services in our community” meant to them. Good service was described as available, timely, respectful, cost-effective, and culturally sensitive service: “So not only accessing the services, but [having them provided] in a way that's culturally sensitive. And also if it is provided in such a way where it's not going to be a financial burden for them.”
Good access, on the other hand, was described as reducing barriers to services:
When I think of teenagers, forget it. Right? So I think that's why the OPT [Options for Sexual Health] clinic [is important]—and even that for a teenager can be very intimidating. Like, what if their mother's friend is working there or … you know? They're in a small town so I think it's a huge issue.
I think it would mean that they would know where to access it and how to access it and who to access it through.
Themes identified for missing and important health topics
Many topics were identified as important and missing from the first iteration of the questionnaire. The most commonly mentioned missing topics included mental health (eg, depression, postpartum depression), sexual abuse, drug addiction, and domestic violence.
I hear from women, certainly we're always concerned around depression—around perhaps sexual abuse—in their relationships and going into doctors and without even really talking about what's going on in their life—the trauma in their life—they're given antidepressants. So things like that are concerns that I think women specifically experience.
These topics were frequently mentioned not only as important aspects of women's health, but also as substantial barriers to accessing other health services. Other topics identified by more than one participant included screening for sexually transmitted infections and Pap tests.
Themes identified for overall impression of the questionnaire
Most nonphysician participants expressed a sentiment of not being informed enough to properly fill out the survey, and that they were filling it out as much from their own experiences as those of their clients: “If I knew more about gynecology I would probably add, or think of, stuff.”
Participants also believed that they would answer questions differently for different groups of women, specifically youth, aboriginal women, or women of lower socioeconomic status: “I find there's a huge difference as to which women you're referring to, so [I would answer differently for] First Nation[s] or non-First Nation[s] women.”
When responding to the 3 final questions addressing overall needs and expectations for women's health services in their community, participants acknowledged a general lack of specialized care in their communities, but expressed an acceptance of that, by virtue of the communities being rural.
The only thing is when you read all the procedures and you realize how many you don't actually have … I mean, the ones we do have [are] wonderful but it would be nice to be able to access medical abortion in this town or whatever other … I don't know how realistic that would be in a small town. So I put disagree, but I'm certainly happy that we do at least have some of the services.
Themes identified by physician participants
There were no concerns with comprehension of questions, aside from question 2, for which participants asked for clarification: “What is your role in the medical community?” Participants also asked for clarification of the term SAS [sexual assault services] training. When asked whether the 8 topics included on the nonphysician survey (questions 3 to 10) should be included on the physician survey, all physicians answered yes. They believed that although there were services that all family physicians should provide, not everyone was comfortable enough with their knowledge and skills to provide them: “I think I'd be inclined to put those questions in because I know that some people don't do those things and I actually get referrals from other GPs to deal with some of these things, at times.”
Second iteration
The 6 participants interviewed about the revised versions of the questionnaires included 3 physicians and 3 nonphysicians from a fourth study community (Table 1). No concerns were identified in terms of comprehension or clarity. Physician feedback to the newly added questions was positive; they expressed that although these services were core family medicine services, many physicians were uncomfortable managing them. Participants denied feeling insulted by being asked these questions. Nonphysician responses were similar to previous responses in their description of the term good access to services, and they too did not identify questions needing clarification.
DISCUSSION
Interview responses revealed that clarity and comprehension of the questionnaires were good. Despite this, nonphysician participants answered that they were unsure on many questions pertaining to specific services, implying a general lack of awareness of specialized medical services in the community. Similarly, the discrepancy among physician responses highlighted a general lack of communication about specialized services being offered in these communities. We removed technically detailed questions that most nonphysician participants believed they could not answer, as these did not contribute to evaluating availability and access of these services. Availability of specialized services is better evaluated by surveying all physicians in a community to obtain the most accurate assessment of existing services.
By asking whether women have good access to services, we are assessing everything the term access implies, which presents a more holistic view of a community's health services. This made the question difficult to answer, as did the diversity of the women and the specific barriers they faced. In some communities partial services are offered. For example, public health system youth sexual health clinics offer services to young women but exclude those older than 19 years of age. We made some questions more specific in the second version of the questionnaire to address this.
Topics identified as important and missing from questionnaires included violence and mental health. Paradoxically, both of these issues act as barriers for women to access health services and so they play an integral role in women's medical care. We included questions assessing access to services for mental health, sexual and domestic violence, and substance abuse on the second version of the questionnaire to address these concerns.
Physicians believed it was important to inquire about physician comfort with providing women's health services, a factor that had not been originally included on the physician questionnaire. By asking physicians to rate their comfort level with 8 basic women's health services we were better able to assess the need for further physician education in the community.
Limitations
An important limitation of our study is not surveying the service users directly. The decision to design the survey to target community professionals and leaders was made to be consistent with the most feasible future implementation of this instrument. However, the leaders answering the survey discussed their contributions as representing not only their professional observations, but also their personal perspectives as women living and accessing care in these communities. Testing more question stems might have shown whether questions other than asking women which services were available would offer more specific answers; participants might have been less inclined to answer that they were unsure if more specific questions were asked. Ongoing study of our questionnaires, planned for each physician who completes the ESSWHR program and begins a new practice, will generate a larger sample size to enable statistical validation, including Cronbach a, construct validity, and scale reliability, as has been done in validation of other medical services questionnaires.7-11
Conclusion
Through iterations of pilot testing, we have created 2 survey instruments for implementation as a component of program evaluation for the new UBC ESSWHR program. Testing in remote locations has highlighted unique rural concerns, such that UBC health professional training will now better serve BC community needs. By conducting interviews, along with preliminary administration of the questionnaires, we were able to optimize the content validity and reliability of this instrument to evaluate expectations and needs for local women's health services. We were also able to identify many important women's health considerations that were not addressed on the initial questionnaire. By incorporating this information and iteratively retesting the questionnaires, we demonstrated better inclusion of topics of concern to women living in rural BC, which in addition informs training for future physicians in rural communities.
Given the variety of experiences in a rural community, it will be important for the questionnaires to be administered to as many people as possible to identify the needs and expectations within the community for women's health services. We hope administration of this instrument will not only inform appropriate rural physician training, but also highlight the importance of specialized women's health medical services among those survey participants within rural communities. Thus, use of the instrument we developed might help in at least 2 ways to provide better access to women's health services for the women of rural BC.
Acknowledgment
The authors are grateful for the financial support provided by The Sue Harris Family Practice Research Grant, which made this study possible.
Notes
EDITOR'S KEY POINTS
-
This study developed and validated 2 survey instruments for implementation in each rural community in which a physician will practise after completing the Enhanced Skills in Specialized Women's Health Residency program. The authors optimized the content validity and reliability of this instrument to evaluate expectations and needs for local women's health services.
-
Nonphysician participants answered that they were unsure on many questions pertaining to specific services, implying a general lack of awareness of specialized medical services in the communities. Similarly, discrepancy among physician responses highlighted a general lack of communication about specialized services being offered in these communities.
-
Most nonphysician participants expressed a sentiment of filling the questionnaire out as much based on their own experiences as those of their clients. Participants also believed that they would answer questions differently for different groups of women, specifically youth, aboriginal women, or women of lower socioeconomic status.
Notes
POINTS DE REPÈRE DU RÉDACTEUR
-
Cette étude a servi à développer et à valider 2 instruments d'enquête devant être utilisés dans toute communauté où un médecin ira pratiquer après avoir complété le programme de résidence Enhanced Skills in Specialized Women's Health. Les auteurs ont amélioré la validité et la fiabilité du contenu de ces instruments en leur permettant d'évaluer aussi les attentes et besoins en matière de services de santé locaux pour les femmes.
-
Les participants non médecins ont déclaré qu'ils ne comprenaient pas bien certaines questions concernant des services spécifiques, ce qui laisse croire que ces services ne sont pas bien connus dans les communautés. De même, des différences de réponses entre médecins ont révélé un manque général de communication au sujet des services spécialisés offerts dans ces communautés.
-
La plupart des participants non médecins ont dit qu'ils avaient répondu au questionnaire en se fondant autant sur leur propre expérience que sur celle de leurs clientes. Les participants croyaient aussi que leurs réponses aux questions seraient différentes selon les différents groupes de femmes, notamment les jeunes, les femmes autochtones ou celles qui ont un statut socioéconomique bas.
Footnotes
-
Competing interests: None declared
-
This article has been peer reviewed. Can Fam Physician 2013;59:e101-9
- Copyright© the College of Family Physicians of Canada
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.