Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS)

Improving the Implementation and Sustainment of Evidenced Based Practices in Mental Health: Developing and Piloting the Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies

National Institute of Mental Health (NIMH), North Carolina Child Treatment Program, National Child Traumatic Stress Network, UCLA-Duke National Center for Child Traumatic Stress

The purpose of this study is to partner with the North Carolina Child Treatment Program (NC CTP) and the SAMHSA-funded National Child Traumatic Stress Network (NCTSN) to develop and pilot the Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS). The COAST-IS intervention will involve coaching organizational leaders and therapists to use Intervention Mapping to select and tailor strategies. Intervention Mapping is a multistep process that is inherently ecological and incorporates theory, evidence, and stakeholder perspectives to ensure that intervention components effectively address key determinants of change. After collaboratively developing COAST-IS in Year 1, the investigators will conduct a randomized pilot trial of the intervention within an NC CTP learning collaborative, randomly assigning eight organizations to the learning collaborative-only condition or the learning collaborative plus COAST-IS condition. Participants will include organizational leaders (e.g., CEOs/Directors, Clinical Directors, Supervisors) and therapists (e.g., Licensed Clinical Social Workers, Licensed Psychologists, Licensed Professional Counselors). The investigators will evaluate COAST-IS in the following aims: 1) to assess the acceptability, appropriateness, feasibility, and utility of COAST-IS; 2) to evaluate organizational stakeholders' fidelity to the core elements of Intervention Mapping; and 3) to demonstrate the feasibility of testing COAST-IS in a larger effectiveness trial. This work is significant because it will yield a systematic method that integrates theory, evidence, and stakeholder perspectives to improve the effectiveness and precision of implementation strategies. Ultimately, COAST-IS may have the potential to improve implementation and sustainment of a wide-range of EBPs in mental health and other health sectors.

In addition to participating in a learning collaborative for implementing trauma-focused cognitive behavioral therapy (TF-CBT) with periodic coaching calls, organizations will receive additional training and tailored implementation support.

Organizations will participate in a learning collaborative for implementing trauma-focused cognitive behavioral therapy (TF-CBT) with periodic coaching calls.

COAST-IS has been designed to equip organizations with the knowledge, motivation, and skill needed to thoughtfully match implementation strategies to identified determinants by applying intervention mapping. The COAST-IS intervention will include four different modes of delivery: 1) dissemination of educational materials; 2) five web-based interactive education sessions on the need for tailoring implementation strategies and the application of intervention mapping; 3) site visits to discuss change objectives necessary to successfully implement trauma-focused cognitive behavioral therapy; and 4) organizational coaching related to using an intervention mapping approach.

The learning collaborative model was adapted from the Breakthrough Series Collaborative model. Leadership of the collaboratives includes experts in evidence-based practices, implementation, and quality improvement. Main components include: 1) three face-to-face learning sessions (2-days each) that provide clinical training; 2) post-learning session action periods structured to facilitate therapists' application of learned skills; 3) a secure website to facilitate faculty-to-participant and peer-to-peer learning and document use of quality improvement methods; 4) fidelity monitoring and coaching; 5) a senior leader track supporting organizational change; 6) monthly outcomes monitoring; and 7) sustainability planning.

This will be assessed with the 4-item measure: Acceptability of Intervention. This measure is rated on a 5 point scale (completely disagree, disagree, neither agree nor disagree, agree, completely agree), with completely disagree having a score of 1 and completely agree having a score of 5. Higher scores indicate greater acceptability.

This will be assessed with the 4-item measure: Intervention Appropriateness.This measure is rated on a 5 point scale (completely disagree, disagree, neither agree nor disagree, agree, completely agree), with completely disagree having a score of 1 and completely agree having a score of 5. Higher scores indicate greater appropriateness.

This will be assessed with the 4-item measure: Feasibility of Intervention.This measure is rated on a 5 point scale (completely disagree, disagree, neither agree nor disagree, agree, completely agree), with completely disagree having a score of 1 and completely agree having a score of 5. Higher scores indicate greater feasibility.

Fidelity to COAST-IS will be measured with a fidelity assessment tool adapted from the Stages of Implementation Completion measure, which assesses organizations' progression through eight stages of implementation (engagement through competency [conceptualized as the start of sustainment]).

Therapist fidelity and adherence to TF-CBT will be assessed with the TF-CBT Fidelity Metric. This instrument consists of 12 scales (e.g., gradual exposure, cognitive processing) that allow a trainer to rate (on a 4-point scale) each TF-CBT component applied by a therapist within a session.

Therapist fidelity and adherence to TF-CBT will be assessed with the TF-CBT Fidelity Metric. This instrument consists of 12 scales (e.g., gradual exposure, cognitive processing) that allow a trainer to rate (on a 4-point scale) each TF-CBT component applied by a therapist within a session.

The 12 items comprising the ORIC measure will be included in an organizational barrier assessment that will be administered to all enrolled participants and will guide implementation support provided through COAST-IS. This measure assesses whether members of an organization, collectively, are (1) committed to implementing a new practice or process and (2) believe that they have the capacity to do so. For these two subscales and the overall ORIC scale, organizations receive scores between 1 and 5 (higher scores are considered more positive).

The 15 items comprising the EBPAS measure will be included in an organizational barrier assessment that will be administered to all enrolled participants and will guide implementation support provided through COAST-IS. This measure articulates practitioner willingness to adopt new interventions as a function of four factors: (1) appeal of the new intervention, (2) organizational requirements, (3) practitioner openness, and (4) divergence (perceived incompatibility with practice or limited usefulness of research-based interventions). For these four subscales and the overall EBPAS scale, organizations receive scores between 0 and 4 (higher scores are considered more positive).

This questionnaire assesses factors related to the inner setting of organizations that are believed to be important in influencing the implementation of interventions. Thirty items from this questionnaire will be included in an organizational barrier assessment that will be administered to all enrolled participants and will guide implementation support provided through COAST-IS. These items measure three factors: (1) culture, (2) learning climate, and (3) available resources. Organizations receive scores between 1 and 5 for each of these factors (higher scores are considered more positive).

The six items comprising this measure will be included in an organizational barrier assessment that will be administered to all enrolled participants and will guide implementation support provided through COAST-IS. This measure assesses whether organizations are primed for implementation via three factors: the extent to which employees believe implementation is (1) expected, (2) supported, and (3) rewarded. For these three subscales and the overall ICS scale, organizations receive scores between 1 and 5 (higher scores are considered more positive).

The seven items comprising the PSS measure will be included in an organizational barrier assessment that will be administered to all enrolled participants and will guide implementation support provided through COAST-IS. This measure assesses how much the environment of a work team makes members feel they can safely engage in learning behavior (e.g., seeking feedback, discussing errors, asking questions, experimenting) without great risk of losing face. Organizations receive a score between 1 and 7 for this scale (higher scores are considered more positive).

The 12 items comprising the ILS measure will be included in an organizational barrier assessment that will be administered to all enrolled participants and will guide implementation support provided through COAST-IS. This measure assesses whether leadership is proactive, knowledgeable, supportive, and perseverant based on specific actions leaders take in promoting implementation. For these four subscales and the overall ILS scale, organizations receive scores between 0 and 4 (higher scores are considered more positive).

The six items comprising the ICBS measure will be included in an organizational barrier assessment that will be administered to all enrolled participants and will guide implementation support provided through COAST-IS. This measure assesses two critical employee behaviors that go beyond what is required to support the implementation of evidence-based practices: (1) helping their peers with implementation-related activities and (2) keeping informed about issues related to evidence-based practice and implementation efforts. For these two subscales and the overall ICBS scale, organizations receive scores between 0 and 4 (higher scores are considered more positive).

This questionnaire assesses employees' experiences and judgments toward previous innovation implementation, which may affect their implementation behavior regarding future innovations. Sixteen items from this questionnaire will be included in an organizational barrier assessment that will be administered to all enrolled participants and will guide implementation support provided through COAST-IS. These items measure four subscales: (1) perceived intensity of previous innovations, (2) perceived failure of previous innovations, (3) innovation-targeted helplessness, and (4) innovation fatigue. Organizations receive scores between 1 and 5 for each of these subscales (higher scores are considered more negative).

Employed at an organization participating in a North Carolina Child Treatment Program TF-CBT learning collaborative Working as an organizational leader (e.g., senior leaders and clinical supervisors) or therapist Direct involvement in the organization's implementation of TF-CBT

Powell BJ, Haley AD, Patel SV, Amaya-Jackson L, Glienke B, Blythe M, Lengnick-Hall R, McCrary S, Beidas RS, Lewis CC, Aarons GA, Wells KB, Saldana L, McKay MM, Weinberger M. Improving the implementation and sustainment of evidence-based practices in community mental health organizations: a study protocol for a matched-pair cluster randomized pilot study of the Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS). Implement Sci Commun. 2020;1:9. doi: 10.1186/s43058-020-00009-5. Epub 2020 Feb 25.