Leveraging the science of implementation: the case for specialized mental health community supervision

Abstract

People with mental illnesses are overrepresented in criminal legal systems internationally, making addressing mental health among this population a global public health concern. Across the world, community supervision agencies (i.e., probation and parole) have implemented a variety of innovative and evidence-informed approaches to improve outcomes for people with mental illness. However, the demonstrated success of these approaches in one region or country does not guarantee effectiveness in another due to significant variations in the implementation context, including differences in governance and administration, society and culture, and resource constraints. Applying implementation science methods throughout the phases of intervention (design and development, implementation, and evaluation) provides tools that can help translate innovations within and across different agencies, countries, and contexts. To highlight how implementation science methods can be used to adapt and implement health interventions within criminal legal system settings, this perspective uses the example of specialized mental health community supervision in the USA. Drawing on general implementation science principles and the Exploration, Preparation, Implementation, Sustainment (EPIS) framework, we articulate key questions and steps agencies can take to translate health interventions from theory into practice.

Background

In many countries across the globe, people with mental illnesses are overrepresented in their nation’s jails, prisons, and on community supervision caseloads (i.e., probation and parole) [1,2,3,4,5,6,7,8,9]. Given these higher rates of criminal legal system involvement, agencies look to evidence-informed interventions to improve outcomes for people with mental illnesses. Although selecting evidence-informed interventions is considered best practice, the evidence base is not the sole criteria for selection, particularly when implementing an intervention developed and tested in contexts outside the intended implementation site. Rather, there are contextual factors (e.g., culture, sociopolitical environment, governance, and administrative structure of the agency) that impact every stage of the intervention implementation process, from identifying the problem, conceptualizing its etiology or root cause, selecting interventions, and implementing and adapting them. These contextual differences impact whether a given intervention may be the right fit for an agency and the problem it was selected to address. Additionally, variation in implementation context also means that the conditions under which the intervention initially demonstrated effectiveness differ from the context in which it is being implemented. Therefore, the same demonstrated effectiveness cannot necessarily be expected.

Although these contextual differences should be considered, they should not stifle agency innovation and intervention adoption. Rather, researchers and practitioners can apply implementation science methods and frameworks to account for and address contextual differences and adapt evidence-informed interventions as necessary. This perspective uses specialized mental health community supervision to illustrate how implementation science methods can be used to adopt health interventions within criminal legal system settings.

What is implementation science?

Implementation science research is “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and hence to improve the quality and effectiveness of health services” [10]. Evidence-based practices encompass a variety of interventions, programs, processes, policies, and guidelines that are supported by evidence of a problem’s etiology and burden upon a specific population or agency, the effectiveness of an evidence-based practice in addressing the problem, and/or the strategies and contextual factors that affect the practice’s implementation [11]. Implementation science enhances the uptake of evidence-based practices to improve outcomes by (1) understanding the factors that impact implementation (i.e., implementation determinants and contextual factors), (2) developing and testing strategies to address implementation challenges (i.e., implementation strategies), and (3) assessing implementation outcomes [12,13,14,15].

Despite the utility of implementation science methods, a limited number of correctional health studies have integrated them. In a systematic review examining two decades of correctional health research studies in the United States (US), only 26 explicitly applied implementation science methods [16]. Of those, a majority focused on prisons or jails, and only a quarter used implementation science methods in health-related interventions implemented within community supervision settings.

Although there are relatively few correctional health studies using implementation science methods, their findings make significant contributions to understanding the complexity of the implementation context. Some of these studies described multi-level factors (e.g., organizational- and policy-level factors) that can impact intervention implementation and the salience of interorganizational relationships in implementing health interventions in corrections settings [16]. For example, Zielinski et al. [17] illustrated how two implementation science frameworks either did or could have helped identify factors (e.g., staff characteristics, perceptions of leadership support, agency resources, local partnerships) affecting implementation of health interventions in correction settings. Other work, including several studies supported by US federal funding agencies like the National Institute of Drug Abuse (e.g., Criminal Justice Drug Abuse Treatment Studies and the Justice Community Opioid Innovation Network) [18, 19], has demonstrated the benefits of using implementation science methods to promote the uptake of health interventions in corrections settings, such as treatment for substance use, HIV, and hepatitis C.

To further promote the use of implementation science to adopt interventions within correction settings, we offer one example of how researchers and practitioners could apply implementation science methods to systematically approach the adoption of specialized mental health supervision caseloads within a community supervision agency. Those interested in learning more about the practical application of implementation science could review Bauer et al. and Bauer and Kirchner [20, 21].

Applying an implementation science framework to adopt specialized mental health community supervision

Across the globe, cultural context weighs significantly on our perceptions of mental illness, including what it is, how it is developed, and the relative (dis)advantage a society may place on a person experiencing it. Additionally, how a nation defines criminal behavior varies by context. For example, nations evincing patterns of deinstitutionalization, including many in North and South America, often use the criminal legal system to respond to mental illness [22]. In contrast, other nations, including some in Asia and Europe, have increased their capacity to rely on psychiatric or community resources [22]. Consequently, how we define the problem to be addressed and what we believe are the root causes impact the selection of interventions or programs to address the issue.

Not only does context determine what we deem a problem and the interventions we choose to address it, but context is the linchpin of successful implementation. Although an intervention may be selected for its evidence base, there are no guarantees that it will produce the same results when implemented in a new setting. This is particularly true for complex interventions that, for example, may rely on external partners or resources beyond the agency’s control. Additionally, an intervention may have shown efficacy in a setting vastly different from where the planned implementation will occur. For instance, practitioners in correctional health intervention research often implement interventions developed and tested with non-justice-involved groups and in non-carceral community-based settings. Even among justice-involved populations, significant variations exist in the issues individuals face. For example, the risk of overdose mortality is greater for people with felony convictions who are in the community unsupervised or on parole supervision than those on probation supervision [23]. Variations in the intended clients’ characteristics and differences in the agency context mean that an intervention cannot be passively added to the agency’s operations. Instead, the agency must use intentional and systematic processes to select and implement that intervention in the organizational environment. This process of translating an intervention from one setting to another can be facilitated by using implementation science methods.

In this section, we describe and apply the Exploration, Preparation, Implementation Sustainment (EPIS) framework [15, 24] to a hypothetical agency’s adoption of specialty mental health community supervision [25]. There are countless theories, frameworks, taxonomies, and models that can guide, describe, or evaluate intervention implementation and address the contextual factors that impact the intervention planning and implementation process [13]. We selected the EPIS framework because it is often applied to complex interventions in the public sector and due to its capacity to attend to both the process of implementation (i.e., process model) and the contextual factors that impact implementation (i.e., determinants framework). The process model component refers to the stages described in the EPIS acronym — Exploration, Preparation, Implementation, and Sustainment — which guide implementation from initial identification of a problem to selection of an intervention (i.e., the adoption decision), to longer-term maintenance of the selected intervention. The determinants component of the framework refers to the multi-level factors that can impact the implementation process and affect the agency’s uptake of the intervention and its efficacy. These determinants are associated with the inner context (i.e., factors associated with the host organization of the intervention), outer context (i.e., factors related to the service environment in which the intervention is implemented), bridging factors (i.e., factors that span the inner and outer contexts), and innovation factors (i.e., factors associated with the intervention itself) [15, 24, 26]. For additional details about the EPIS process model components and implementation determinants, see Aarons et al. [15], Moullin et al. [24], and https://episframework.com/.

Given that the focus of this perspective is on the adoption of interventions, we will focus on the first two phases of EPIS: exploration and preparation. Table 1 lists potential activities or action steps associated with the exploration and preparation phases of the EPIS framework [15, 24] and publicly available information on the resource’s website (https://episframework.com/). These steps do not represent a complete list of tasks and are used here to illustrate the implementation of specialized mental health supervision.

Table 1 Focusing questions for the exploration and preparation phases

The exploration phase and specialized mental health supervision

During the exploration phase of EPIS, a service system or agency explores a problem impacting the health and needs of a client system [15, 24]. The primary task during the exploration phase is to assess the problem, identify potential interventions, and assess the fit of the potential intervention. The agency or service system then decides whether to adopt the intervention and, if so, whether the intervention needs to be adapted to fit the implementation context. For the purposes of illustrating how EPIS can be applied to specialty mental health community supervision, we explore four tasks or actions the agency may complete during the exploration phase: (1) articulate the organizational or system issue that needs to be addressed, (2) conduct a needs assessment that describes the problem and its scope, (3) identify potential interventions or evidence-based practices, and (4) assess the fit of the intervention within the organizational or system context.

Articulate the organizational or system issue that needs to be addressed

When agencies decide to adopt specialized mental health community supervision, it is typically because there is an organizational or system issue that needs to be addressed through evidence-based interventions. At first glance, the issue addressed by specialized mental health supervision may be the overrepresentation of people with mental illnesses on community supervision caseloads across the globe [1,2,3,4,5,6,7,8,9]. However, is the fact that there are a lot of people with mental illnesses on community supervision the issue that compels agencies to find an intervention? Or is it that people under community supervision with mental illnesses have higher rates of revocations [27, 28], substance use [29], and suicide [30] compared to those on community supervision without mental illnesses? Or is the issue the lack of adequate training for officers in how to address mental illnesses among people on their caseload? [31, 32] The problem the agency wants to address could also be that officers who supervise people with mental illnesses report higher rates of emotional exhaustion, work stress, and depressive symptoms [33, 34], which hinders their job performance. Indeed, when psychologically distressed, officers can be less supportive, more likely to perpetuate stigmas associated with mental illness, and less competent in assisting clients with mental health needs [35, 36]. These different ways to articulate the problem are helpful examples of how context can impact what we call a problem worthy of intervention and, consequently, determine what intervention will be selected or developed.

Conduct a needs assessment that describes the problem and its scope

In practice, a needs assessment means that a researcher or practitioner in an agency is finding information that describes the problem within the context of the agency itself. Knowing the size of the population of people with mental illness under community supervision at the agency and any relevant trends in supervision outcomes is an important first step in determining the scope of the intervention. Additionally, needs assessments should document the agency’s assets and resources to address the issue. Concerning specialized mental health supervision, demonstrating the prevalence of mental illnesses among those on supervision—as well as any negative correlates—is challenging, given that many agencies may not screen for or track mental health conditions in their supervision census [32]. Consequently, to identify the scope of mental illness among those under community supervision, agencies may need to consider the implementation of a mental health screening tool.

Identify potential interventions or evidence-based practices

Researchers and agency practitioners can look to the research literature for studies examining the effectiveness of community supervision interventions for people with mental illnesses. They may also find interventions through technical assistance centers, government agencies, and registries that report on evidence-based practices (e.g., the National Institute of Justice’s Crime Solution, https://crimesolutions.ojp.gov/). For illustrative purposes, we will narrow the focus to two types of interventions.

One type of intervention for people with mental illnesses under community supervision is embedding services within the community supervision agency [32]. This type of intervention may include a therapist or other type of clinician who can complete a comprehensive assessment and/or provide ongoing mental health care in a co-located setting. In determining whether this type of intervention is a good fit for the agency, it is important to evaluate what resources are available. Some systems have the resources to embed mental health counselors directly in the agency (e.g., resocialization and probation penitentiary counselors in France) [37] or establish specialty courts to provide intensive case management for people on community supervision, such as the Assessment and Referral Court List in Victoria, Australia [38]. In other cases, the agency may need to rely on community resources. For example, the Turkish Probation Service incorporates volunteers to serve as psychologists and social workers for people with mental illness [39].

Another type of intervention focuses on building the capacity of the community supervision officers to supervise people with mental illnesses and facilitate their treatment engagement in community-based services. In centralized systems, this may be achieved via training and refresher courses offered by national academies like the Center for Penitentiary Studies in Spain [40]. In decentralized systems like the USA, this type of intervention is aligned with the specialized mental health supervision model, which has emerged as a promising practice for supervising people with mental illnesses. The model typically consists of five elements, including designated and exclusive mental health caseloads, a reduced caseload size, ongoing mental health training, enhanced interactions with external resource providers (e.g., mental health providers), and a problem-solving orientation among officers [25, 28, 41,42,43,44].

Assess the fit of the intervention within the organizational or system context

Once potential interventions are identified, the agency would consider the potential fit of the interventions. In this example, decision-makers would think about agency context, including but not limited to the system and community mental health resources and personnel, staff perceptions about co-locating mental health services in community supervision, staff capacity for advancing their mental health knowledge and skill set, staff perceptions about whether community supervision officers should have a role in addressing mental illness and treatment facilitation, staff workload and vacancies, space available for potential service co-location, confidentiality agreements and regulations around the protection of private mental health information, and adequacy of in-house expertise about mental illness to manage the selected intervention. In this example, the ultimate question that should be answered during the exploration phase is which of the two interventions identified — embedded mental health treatment vs. specialized mental health supervision — is the best fit for the agency and whether adaptations to the selected model may be needed to fit the agency’s specific context.

The preparation phase and specialized mental health supervision

Once the agency decides to adopt the intervention, they begin to plan for implementation (i.e., the preparation phase). During this phase, the agency assesses potential factors that could impact implementation (e.g., a pre-implementation assessment) [14], continues to consider whether adaptations are necessary, and begins to foster a climate and culture for implementation. Importantly, the agency also considers implementing strategies to enhance the uptake of the intervention (e.g., implementation strategy development and testing) [45, 46]. In the sections that follow, we assume the agency chooses to adopt specialized mental health supervision, and we describe three preparation phase tasks: (1) assessing potential factors that can impact implementation, (2) planning implementation strategies to address challenges and enhance uptake of the intervention, and (3) developing a data monitoring and evaluation plan (see Table 1).

Assess potential factors that can impact implementation

A pre-implementation assessment is conducted before the intervention is implemented to systematically assess potential factors that can impact implementation and to understand the implementation context before the work of the intervention begins [14, 47]. In this example, as the agency prepares to implement specialized mental health supervision caseloads, the implementation team (i.e., those tasked with implementing the model) would begin by defining a pre-implementation period in which they (1) identify potential barriers and challenges to implementing specialized mental health caseloads (e.g., officer workload, mental health stigma), (2) identify resources that can be leveraged to enhance implementation of specialized mental health caseloads (e.g., potential champions within the agency, strategic partnerships with mental health agencies), and (3) define metrics for data monitoring and evaluation to enable early identification of any implementation-related problems (e.g., problems with mental health caseload assignment, identification processes for determining caseload eligibility). Data collection methods used for the pre-implementation assessment can be as formal as a series of interviews, focus groups, and surveys with officers and members of the implementation team or more informal or unstructured time set aside for talking with staff about their needs, ideas, and potential challenges that may be relevant to address before implementing the intervention. Results from the pre-implementation assessment directly inform the following two example action steps.

Plan implementation strategies to address challenges and enhance uptake of the intervention

Implementation strategies are not part of the intervention itself but are a set of actions or activities that are implemented ahead of the actual intervention to address potential challenges identified [45, 46]. For example, during the pre-implementation assessment for specialized mental health supervision caseloads, agency staff could have expressed concern about their lack of relationships with mental health service providers (i.e., an implementation challenge) and how that can impact rapid referral and connection to services for people on their caseloads (i.e., an implementation outcome). Using this information, the agency may decide to focus on enhancing aspects of the implementation context, namely by improving relationships between officers and providers to facilitate the referral process. For example, the community supervision agency may decide to host a series of engagement meetings or increase networking with local mental health providers to discuss the newly implemented specialized mental health supervision caseloads. At the events, the officers and providers would be expected to exchange information about their roles and programs and to provide each other with contact information. This type of implementation strategy is aligned with strategies listed within the Expert Recommendations for Implementing Change (e.g., network weaving), which is a compilation of 73 implementation strategies to enhance the uptake of interventions [45, 46]. These networking activities are not part of the intervention itself but rather aid in its implementation by enhancing officers’ ability to implement a key component of the intervention, which is to connect a person on their caseload to services.

Develop a data monitoring and evaluation plan

During the pre-implementation assessment, it is also useful to ask staff about relevant outcomes and programmatic indicators that should be monitored. For example, during the pre-implementation assessment, the implementation team could gather staff perspectives about how to evaluate and monitor the implementation and effectiveness of specialized mental health supervision. Relying on the input and expertise of those most familiar with specialized mental health supervision, their agency’s data and monitoring system, and community supervision generally will improve the validity of the overall evaluation. Additionally, being able to identify relevant programmatic indicators of intervention adoption, such as treatment referrals and specialized caseload sizes, will enable the agency to quickly identify early roadblocks to implementation, which can suggest additional adaptations.

The implementation and sustainment phases and specialized mental health supervision

Although the primary focus of this perspective is on the exploration and preparation phases, it is important to briefly describe activities associated with the implementation and sustainment of the intervention. During the implementation phase, training and coaching begin, and leadership provides guidance and support for intervention initiation. Any implementation strategies planned during the preparation phase are initiated, and the agency begins monitoring the implementation process and using data to determine what additional implementation support may be needed. In terms of the specialized mental health supervision example, the agency then begins selecting and onboarding their mental health officers and implementing a protocol for identifying eligible people for the caseload. Additionally, the agency begins to use the identified implementation strategy: networking and engagement meetings with service providers. The monitoring and evaluation plan is also launched, and the agency begins assessing the degree to which specialized mental health supervision officers address the mental health needs of people on their caseloads via referrals and other actions.

In the sustainment phase, the agency focuses on quality assurance and ensuring that structures and processes are in place to support the intervention, including clearly articulated protocol for key aspects of the model. For example, agencies will need to clearly define the protocol for mental health supervision officer selection and consider whether the selection process might require a supervisor’s recommendation, a quality assessment of the officer’s case management notes, or even perspectives from supervisees on the officer’s caseload. Further, the agency should ensure that clear eligibility determination processes are in place that describe whether and how officers will confirm that the supervisee has a mental illness and what role risk and need level may play in deciding whether to assign someone to specialized mental health supervision. Additionally, data monitoring and evaluation continue as program staff focus on fidelity to the specialized mental health supervision model and its longer-term sustainability.

Conclusions

This perspective uses specialty mental health supervision as an example of how implementation science methods can be used to adopt interventions and evidence-based practices in corrections settings. This perspective is not meant to comprehensively describe how implementation science methods can enhance research rigor across criminal legal system entities. Instead, this serves as an introduction that promotes the broad application of these methods when agencies are implementing interventions within the complex environments of corrections settings. Specifically, agencies should consider the context of the issue being addressed through intervention, including its etiology and even staff perception of the agency’s role and approach in addressing it. Additionally, agencies must consider the fit between the intervention and the agency context. Finding an evidence-based practice or an intervention with good evidential rigor is essential; however, it is of limited practical consequence if that intervention has shown little efficacy in settings outside the one in which it was tested. This does not mean agencies should not select the intervention; rather, agencies should assess the fit and consider whether systematic adaptation is necessary.