Study design

This study implements a pragmatic RCT design with random allocation of youth aged 14–18 years to either TAU at one of four outpatient hospital sites or ICCT treatment at one of three community-based sites, all in Toronto, Canada. The hospitals include the Centre for Addiction and Mental Health (CAMH), the Hospital for Sick Children (SickKids), Sunnybrook Health Sciences Centre and Michael Garron Hospital (MGH). The ICCT services will be provided by East Metro Youth Services (EMYS), Skylark Children, Youth and Families, LOFT Community Services and Sashbear Foundation, in collaboration with the South East Toronto Family Health Team, and the Anne Johnston Health Station.

Project management and participation plan

The project is governed by multistakeholder groups, including representatives of the four hospital sites, community agencies, youth and family members. The Core Team is composed of the principal investigators and research team members, as well as site representatives and youth and family member coinvestigators. This team is supported by four working groups focusing on (1) community services, (2) study methodology, (3) hospital processes and (4) implementation science. Participating in each working group are also two youth advisors to provide the youth voice to the project. Youth and family advisory groups have also been established to facilitate patient-oriented research. The full project team includes collaborators from various research, clinical and broader community specialties ensuring that the necessary expertise is available for all aspects of the project. This represents an academic-stakeholder cocreation model that can maximise relevance and impact of the study and intervention design.25

Participant selection and withdrawal

Participant recruitment: Participant recruitment will be conducted at the four Toronto hospital sites. Recruitment will take place over a 12-month period. A total of 500 youth will be randomised into the study; 250 will be randomised to ICCT and 250 to TAU. For each youth, the participation of one family member (ie, a primary caregiver) in the study process will be encouraged, but is optional. Across hospitals, the following procedures will be used as appropriate and feasible: hospital intake staff will identify new referrals for appropriateness for referral to the project; standardised telephone screening calls will be conducted by clinical research assistants (RAs); a visit will then be scheduled at a neutral community site with the youth and family member (if any) for consent, enrolment, intake assessments and randomisation. Those who do not meet the criteria or who do not agree to participate in the study will be immediately connected with a regular hospital appointment outside of the research study. For participants who are randomised to the ICCT arm, Consent for Disclosure of Personal Health Information will be completed to enable the clinical RA to forward referral information to the ICCT, as well as a standardised letter indicating referral to the ICCT to the referring physician, to ensure adequate information sharing. Participating youth will receive honoraria in the form of gift cards valued at $50 at time 1, time 2 and time 3. Caregivers will receive $25 in gift cards at each time point, as well as entry into a random draw for a $250 gift card. The study is currently in the recruitment phase; the first participant was enrolled on 19 September 2016.

Inclusion and exclusion criteria. To be eligible, youth must be between 14 and 18 years of age at the time of presentation (the common age range of services at the four hospital sites), referred to one of the four participating hospitals with MHA challenges and be among the population regularly accepted for outpatient adolescent psychiatric services at that hospital. An individual meeting any of the following criteria will be excluded from participation in the study: referral for specialty forensic or firesetting treatment; autism without MHA problems; primary diagnosis of an eating disorder; active psychosis or imminent risk of self-harm requiring immediate intervention; inability to read and write in English due to the self-report assessment component; and inability to consent to the study. As a pragmatic trial, exclusion criteria are intentionally minimised to ensure an inclusive sample representative of the youth who would normally be targeted by this service delivery model. Family member participants must be a primary caregiver of the youth and aware of the youth's everyday functioning. Exclusion criteria are the inability to read and write in English or to consent to the study.

Consent and withdrawal from study: Informed, signed consent will be obtained from all study participants at the intake visit prior to study enrolment and randomisation (see online supplementary appendix A and B). If youth/family members wish more time to consider participation, a second intake visit will be scheduled. Youth will provide consent rather than assent regardless of age, as they are considered to play an equal role in the decision to participate in research. The family member will consent for his/her own respective participation. Copies of the signed consent forms will be given to the youth and family member; originals will be filed securely in study files. Participation will be voluntary and participants will be free to withdraw from the study at any time without affecting their treatment. Whether assigned to the TAU or ICCT arm, youth will be able to continue receiving full treatment within that model if they withdraw from the study. If youth randomised to the ICCT arm withdraw from the study and wish to receive treatment as usual in the hospital setting, the research team will assist them in returning to the original hospital to schedule treatment. Other youth who withdraw from the research project and request community services will be provided with information about available services.

Study interventions

Integrated collaborative care teams: ICCT treatments and services are summarised in table 1. ICCTs are housed in the community in three neighbourhoods across Toronto, Canada. Each ICCT will include MHA care providers (eg, youth worker, social worker, psychiatrist, nurse practitioner), trained peer support workers, access to a primary care providers and a care navigator responsible for working with the various specialists to coordinate care. For each intervention, standardised intervention protocols will be used, employing evidence-informed sources wherever possible. All participants will begin with rapid access to a SFBT session14 ,15 as a gateway to the ICCT services; this will serve as a brief intervention, as well as to identify the needs and preferences of the youth and his or her family member and to develop a treatment plan. Subsequent interventions will then be selected based on need, using clinical assessments and selected by the ICCT service provider in a stepped-care manner.

Youth will be offered higher or lower intensity services, following a needs-based staging model.26 The Columbia Suicide Scale Clinical Practice Screener27 and PRIME Screen28 prodrome screener will be used clinically with all youth across ICCT sites to identify risk suggesting the need for high-intensity/high-risk psychiatric services in the presence of psychotic prodromal or suicidal symptoms. The HEADS-ED29 will be used clinically by ICCT service providers to identify the functional level of the youth, suggesting moderate intensity when both risk and functioning are low, but low intensity when risk is low and functioning is high. These scales will be followed to inform treatment planning based on a treatment algorithm provided to the community sites, in conjunction with the service provider's judgement of treatment response and youth/family preference or expressed need.

The low-intensity intervention will consist of SFBT, a goal-directed single-session intervention that focuses on the youth's strengths and identifies concrete solutions; this will be provided by the ICCT service provider. Youth will receive an established number of single-session SFBT sessions based on need and will have ongoing access to the SFBT model through the ICCT walk-in services. The moderate-intensity intervention will be a modular DBT skills group, provided by group facilitators at each site and consisting of modules focusing on developing the emotional regulation, distress tolerance and interpersonal effectiveness skills of the youth. The high-intensity psychiatric response will consist of management by a child and adolescent psychiatrist within the ICCT environment, and/or a Nurse Practitioner, focusing on psychiatric assessment, medication management and other interventions deemed clinically appropriate; treatment duration will be based on need, as assessed by the clinician. Family members will be offered participation in Family Connections, a DBT-based skills and support group designed for families,17 regardless of the youth's risk and functional level. Additional community services will be offered to youth/family members where appropriate. Since the study aims to evaluate the effectiveness of the pathway rather than the specific effectiveness of each individual intervention, intervention fidelity will be monitored at the level of delivery of intervention modules, components and subcomponents; service providers will document the delivery of intervention modules, components and subcomponents for each youth and/or family member using detailed self-report intervention logs and checklists.

Treatment as usual: The TAU condition consists of the standard outpatient treatment provided at each participating hospital site. This typically entails assessment and treatment planning by a psychiatrist at the participating hospital, and may include medication, psychotherapy and/or internal and external referrals to treatment and other services, guided by local service standards. The hospital services used in the TAU condition will be tracked for each participant using chart review (based on medical record numbers).

Application of interventions: Each hospital and ICCT site will be responsible for the application of the appropriate clinical interventions. Training in core ICCT interventions (SFBT, DBT Skills Group, Family Connections) has been conducted together in a single group, using the same treatment manual. It is expected that interventions will be selected for each youth based on the clinical assessments and the related treatment algorithm provided to the clinical sites, complemented by the clinical judgement of the site staff and drawing from the available services, taking into consideration youth and family member preferences.

Outcome measures

Outcome measures are summarised in table 2. For psychometric properties, see online supplementary appendix C.

Sample description: To describe the basic characteristics of the sample recruited into the study, a custom demographic information form will be administered to the youth and family member. In addition, a clinical description of the sample will be provided by the Diagnostic Interview for Affective and Anxiety Spectrum Disorders—Child Version (DIAS-C)30, which is an interview that will be administered by a trained clinical RA. Risk for psychosis will be assessed by the self-report PRIME.28 Youth will also complete the PTSD CheckList—Civilian Version (PCL-C)31 and a checklist indicating any comorbid physical health conditions.

Clinical/functional outcome: The primary objective, to test the benefits of the ICCT model in improving functioning among youth aged 14–18 years with MHA challenges, will be measured using the Columbia Impairment Scale (CIS)32. This self-report scale includes 13 items, versions for both youth and family members, is reliable and valid, has been used in several clinical trials33–35 and was chosen by youth in focus groups as the main measure of interest to them. Youth CIS scores will represent the primary outcome variable, with caregiver CIS scores serving as a secondary outcome variable.

Our second objective, to assess clinical improvement, will be measured using the Strengths and Difficulties Questionnaire (SDQ)36 in its versions for youth self-report and parental report. Further symptom profiles, including problematic substance use, will be assessed using the GAIN Short Screener,37 which will be supplemented by the substance use table of the Adolescent Alcohol and Drug Involvement Scale.38 An additional clinical outcome will include caregiver burden, measured using the self-report Burden Assessment Scale (BAS)39 administered to family members only.

Satisfaction, goal achievement, engagement, empowerment, burden: Our third study objective, satisfaction with the service models, will be assessed using the Ontario Perception of Care Tool for Mental Health and Addictions, client and family versions (OPOC)40. To provide insight into goal achievement, each youth and parent will be asked to determine his or her own goals at intake; we will then use an 11-point progress scale based on the Goal Progress Chart41 at the 6-month and 12-month assessments to determine the average extent to which these goals were met. Further information on client empowerment and engagement will be collected using the self-report Family Empowerment Scale42 for family members, the Youth Efficacy/Empowerment Scale43 for youth and the Continuity of Care in Children's Mental Health questionnaire44 in its family member and youth versions.

Economic evaluation: Our final objective is to determine the cost-effectiveness of the ICCT model compared to TAU. We will perform two analyses, a cost-effectiveness analysis (CEA) and a cost-utility analysis (CUA), to determine the incremental costs of ICCT compared to TAU in modifying health outcomes as measured by the CIS and quality-adjusted life years (QALYs) measured with a validated utility instrument, the Assessment of Quality of Life-6D (AQOL-6D)45. The AQOL-6D will be administered at baseline and at 6 and 12 months to determine the change in utility for the duration of the intervention, and will be used to calculate QALYs. Custom data collection tools have been developed to measure direct costs to the youth/family member, as well as indirect costs (eg, lost income due to appointments; see table 3 for a detailed description of each cost category). These tools will be completed by treatment teams in the respective treatment arms and by the youth and their family member (if applicable). This data will be collected at all three assessment times. Additional data regarding health services use and direct costs to the healthcare system will be obtained through the Institute for Clinical Evaluative Sciences (ICES), which holds administrative healthcare use and cost data, such as physician billings, hospital emergency services and inpatient stays, for the province of Ontario. Since participants will not be prevented from engaging in concomitant service use during the trial period, all service use will be documented using the data collection tool. Follow-up data will be collected from participants regardless of the amount of time during which they receive services.

Randomisation

Sequence generation: Participants will be block-randomised to the TAU or ICCT arms using random block sizes for two treatments within each of the two strata defined by sex, separately at each of the four hospital sites (62–63 participants in each of the 8 strata, for sex and site). Randomisation will be executed using computer algorithms generated by REDCap research software.46

Allocation concealment: Participants will be randomised to the two study arms at entry into the trial. The allocation sequence will be concealed from the clinical RAs responsible for consenting, registering and assessing participants until all intake assessments are complete and the clinical RA accesses the website to obtain the randomisation result.

Implementation: The allocation sequence will be generated centrally by REDCap system and accessed by the clinical RA at the time of randomisation. Clinical RAs will receive a training session on the randomisation process prior to study launch. To randomise, the clinical RA will enter the sex of the participant into the software to obtain stratified randomisation within that hospital site.

Blinding: Clinical RAs will be blind to treatment allocation at intake, as intake assessments will be conducted prior to randomisation. Data analysts will be blind as to the treatment arms.

Data collection procedures

Clinical data: Clinical data will be collected by clinical RAs at the intake visit. The intake meeting will be ∼2½ hours in length based on pilot tests. The clinical RA will have access to an on-call clinician at all times during appointments to provide support for emergent issues. The assessment process will be repeated at 6 months and 12 months. Data from assessments will be directly collected into the electronic data capture system REDCap46 using a tablet application and stored on a secure server hosted at CAMH.

Economic evaluation: Data will be collected by clinical RAs at baseline, 6-month and 12-month follow-up points. Site measures will be collected at each treatment site. Additional information on clinical contacts will be collected by clinical RAs from patient files (number, duration, type of service provider). At the end of the follow-up period, and once data is available, the health card number will be used to link the patient to their healthcare usage data from the Ministry of Health and Long-Term Care (MOHLTC). This data will be accessed through the Institute for Clinical Evaluative Sciences. The data will remain de-identified throughout the analysis (healthcare visits, emergency department visits, hospitalisations, specialist referrals, etc; see Statistical Plan below).

Statistical plan

Analysis will be by intention to treat, and every attempt will be made to collect outcome data from all participants, including those who do not complete treatment. General linear models with repeated observations per youth (6 and 12 months) will be used to compare treatment and TAU groups with respect to outcome variables. Generalised estimation equations will be employed for parameter estimation. The respective baseline values will be added to the model as a covariate. For the primary outcome, a two-sided level of 0.05 will be used to examine for statistical significance. To control type I error probabilities, the analyses of secondary outcomes and the subgroup analyses will employ a two-sided level of 0.005. Subgroup analyses for the primary and secondary outcomes will be performed based on the following variables: age (14–16.5 vs 16.5–18 years old), severity of functional impairment (CIS) at baseline, disorder duration/age of onset and comorbidity of mental health symptoms (DAIS-C and self-report measures), all dichotomised depending on the distribution. Additional analyses will include examining, within treatment arms, the between-site heterogeneity and identifying baseline predictors of treatment response. Participants will also be asked to report their impression of the change observed, as an indication of clinical significance.47

Sample size determination and statistical power: A sample size of 500 participants will provide 80% power for the primary outcome, that is, the CIS.32 This is based on the alternative hypothesis that the arms differ by 0.15 SD at 6 months and 0.3 SD at 12 months, α=0.05 (two-sided), and an intraclass correlation coefficient ≤0.4 between the 6-month and 12-month observation periods. An effect size of 0.3 SD is considered the smallest that is clinically relevant and represents at most a 3-point difference on a scale of 0–52.

Economic evaluation: The economic evaluations will take both a healthcare system and societal perspective; these will include a CEA and a CUA. The time horizon will be the youth's lifetime. For the CEA, we will account for direct costs to the healthcare system and to the youth/family members, as well as indirect costs. Costs will be adjusted for inflation to 2018 Canadian dollars using the Statistics Canada Consumer Price Index for health and personal care.48 All health outcomes and costs will be discounted at 5%/year.49 We will account for all costs associated with delivering the proposed intervention. This will include costs with personnel, supplies and services, equipment and programme resources, among others. We will make use of a decision model in our CEA to help incorporate the benefits and costs beyond the time horizon of the existing data, and to help evaluate hypothetical scenarios. Health-related costs for the standard care costing cohort will be determined through linkage to population-based administrative databases at ICES using encrypted unique patient identifiers.50 For each youth, we will determine the total usage of health resources and respective costs during the observation window in addition to the data collected directly from participants.

The incremental cost-effectiveness ratio (ICER), the primary outcome of the CEA, will be calculated as the difference in discounted mean costs between the ICCT and TAU groups divided by the difference in functional impairment on the CIS. Similarly, for the CUA the ICER will be a measure of the difference in costs divided by the difference in QALYs between the treatment arms. A 95% CI around these estimates will be estimated non-parametrically using 1000 bootstrap replications.

One-way deterministic sensitivity analyses will be performed to evaluate the robustness of our results. The ranges for the sensitivity analysis will be obtained from 95% CIs. We will also perform a probabilistic sensitivity analysis using a second-order Monte Carlo simulation with 1000 iterations. Using a net-benefit framework,51 a cost-effectiveness acceptability curve (CEAC) will be produced using varying willingness-to-pay thresholds. A CEAC describes the probability that an intervention is cost-effective, compared with the alternative, for a range of maximum monetary values that a decision-maker may be willing to pay for a particular unit change in the outcome analysed. We will derive the CEAC from the joint distribution of incremental costs and incremental effects; we will use non-parametric bootstrapping of the observed data to estimate these joint distributions.

For the CUA, we will measure the change in utility as obtained from the AQOL-6D45 instrument, which will allow us to calculate quality adjusted life years (QALYs). The AQoL-6D is a validated, self-reported instrument that contains 20 items that cover six domains of quality of life, including ‘independent living’, ‘relationships’, ‘mental health’, ‘coping’, ‘pain’ and ‘senses’. The advantage of the CUA approach is that QALYs capture general well-being/disease burden directly from study participants. In addition, because the QALY is a widely used measure in economic evaluations across all clinical disciplines, the results generated from this study will be easily compared to the results of other, different healthcare interventions.

As a subsequent analysis to the economic evaluation, we also propose to examine the real world budget impact of implementing ICCTs for hospitals that treat adolescents across the province. One of the main objectives of this analysis will be to determine how much the Ontario Ministry of Health and Long-Term Care would need to spend to implement this model of care either in the entire province or in a given jurisdiction. Another outcome of interest will be an estimate of the potential savings achieved by the system by implementing this intervention.

Confidentiality

The study will adhere to the Personal Health Information Protection Act52 and all other regulatory and organisational standards for privacy, confidentiality and security of database information. All study investigators will sign confidentiality agreements with ICES, following their guidelines, as PHI will be used to access ICES data.

All identifying information will be stored in locked cabinets separate from the study data. All study data will be coded numerically and password protected in the case of electronic data and stored on a secure server at CAMH. Only study personnel will have access to this data (including keys to cabinets for hard copies, knowledge of passwords for data).

Records retention: Study data will be stored in compliance with appropriate regulations.

Ethical considerations

This study will be conducted according to the guidelines established by Good Clinical Practice (Food and Drug Administration, http://www.fda.gov/oc/gcp/regulations.html). Any protocol modifications will be communicated to all relevant parties immediately as they are made.

Adverse events: Adverse events will be clearly documented at the central study site in an Adverse Event Log immediately on notification and duly reported to the Research Ethics Board. An independent data safety monitoring panel will track and consider any adverse events to ensure participant safety.

Regulatory binder

A regulatory Standard Operating Procedures binder will be kept in a secure location in the project coordinator's office. A separate binder containing personal health information will also be kept in a locked drawer within the project coordinator's office.