NIRN Discussion: Measuring, Assessing and Improving Implementation

Date Published: 
10/18/2012

by Allison Metz


What types of assessments are you and your organization using to assess the strengths and gaps of the implementation infrastructure? There is growing recognition in the health and human services field of the importance of effective implementation and the need for implementation research that can guide selection, development, evaluation, and ongoing improvement of interventions.  Allison Metz from NIRN discusses the Initial Implementation Assessment tool. Your comments and ideas are invited.


The Initial Implementation Assessment developed by the National Implementation Research Network (2012) assesses the strengths and gaps of “Implementation Drivers” which are the core components or building blocks of the implementation infrastructure. These components are needed to support practice, organizational, and systems change.  The Implementation Drivers include: Competency Drivers, Organization Drivers, and Leadership Drivers (see diagram).

The assessment is designed build the capacity of Implementation Teams to develop and install action plans for ongoing improvement of the implementation infrastructure for any intervention.  The Initial Implementation Assessment was administered at baseline and follow-up periods during a five year initiative to develop, implement and improve a post-care service delivery system for children exiting foster care to permanency – the Catawba County Child Wellbeing Project.  The service system consisted of selected interventions ranging from evidence-based programs to evidence-informed practice models that required intensive program development by the public child welfare agency. Implementation Teams served as accountable structures for promoting high-fidelity implementation of all interventions. In some cases the Implementation Teams were accountable for building the infrastructure from scratch, while in other cases the Teams were accountable for filling in the infrastructure gaps of national purveyors of evidence-based models or working with purveyors to shift the locus of responsibility of some drivers from the purveyor to the public agency over time. The assessment was used for all interventions.

Building the capacity of Implementation Teams to shift responsibility of certain drivers from the purveyor to the public agency without compromising program fidelity is important for sustaining evidence-based models.

+ More About the Tool
The Initial Implementation Assessment was administered at baseline and follow-up periods during a five-year initiative to develop, implement and improve a post-care service delivery system for children exiting foster care to permanency (the Catawba County Child Wellbeing Project).  The service system consisted of selected interventions ranging from evidence-based programs to evidence-informed practice models that required intensive program development by the public child welfare agency. Implementation Teams served as accountable structures for promoting high-fidelity implementation of all interventions. In some cases the Implementation Teams were accountable for building the infrastructure from scratch, while in other cases the Teams were accountable for filling in the infrastructure gaps of national purveyors of evidence-based models or working with purveyors to shift the locus of responsibility of some drivers from the purveyor to the public agency over time. The assessment was used for all interventions.
+ Major Findings

Strengthening the implementation infrastructure (i.e., implementation drivers) improves fidelity scores

  • Conducting a stage-based formal assessment of the implementation drivers yields important findings for purposeful action planning by Implementation Teams  whether a purveyor is formally involved or not
  • Building the capacity of Implementation Teams to address infrastructure gaps not installed by purveyors is important for achieving fidelity

 

Community Questions

  • What types of assessments are you and your organization using to assess the strengths and gaps of the implementation infrastructure?
  • How can results from assessments like this Initial Implementation Assessment tool help Implementation Teams?
  • What types of data collection are important to consider before moving to the next stage of implementation?