Extension excels at working across program areas and disciplinary boundaries to address the complex and overlapping issues of substance use recovery and mental wellbeing. However, evaluation of the collective impact of these concerted programmatic initiatives remains a persistent challenge for Extension. In Kentucky, the 2019 statewide community assessment identified substance use and mental health (hereafter, SUMH) as priority issues requiring immediate Extension programmatic efforts. Four years later, the 2023 needs assessment confirmed the persistence of these challenges in the Commonwealth. UK Extension addresses the pressing issues of SUMH through direct education in the form of innovative community-based and evidence-informed programming delivered across the state. The statewide reach notwithstanding, UK Extension faced barriers evaluating the collective impact of these programs because the SUMH programs lacked common indicators of success and measured different outcomes, which hindered the aggregation of evaluation data across programs, resulting in limited statewide assessment of broader impact. 

Recognizing the need to streamline evaluation processes to fully capture the impact of these efforts and enhance our ability to tell compelling stories of broader impact, I established a workgroup to drive the necessary systems change needed to design and implement a collective impact assessment of UK Extension’s SUMH programs. The workgroup consists of fourteen (14) specialists and faculty representing all Extension program areas of 4-H, FCS, ANR, and CED. Between 2022 and 2023, I led the workgroup to develop a catalogue of UK Extension SUMH programs; create indicators of success to assess the outcomes and impact; develop common evaluation tools; and implement a plan for collecting, aggregating, and reporting indicator data. 

Since my faculty appointment in May 2024, I have led the SUMH workgroup to develop and deliver four in-person training sessions to promote the use of the program indicators, data collection tools, and reporting strategies. A total of ninety-one (91) Extension personnel have participated in the trainings, of which eighty-one (81) completed the post-participation evaluation survey. 

• 94% of survey respondents reported that the training increased their awareness of UK Extension SUMH programs and outreach;
• 93% reported that they feel confident that they can use the indicators and tools to evaluate their SUMH programs; 
• 99% reported that they understand how to report the data from the evaluation tools in KERS; 
• 94% reported that the SUMH tools are easy to use; and 
• 94% reported that the indicators and evaluation tools will make evaluation of their SUMH programs easier. 

One participant noted: “I really appreciate this initiative to encourage us to be more mindful when reporting and evaluating these specific [SUMH] programs in our areas. I have never reported or done anything in particular like this before but I am excited to use these tools once I begin working with a program in the county.”

My work with the SUMH (and other ongoing collective impact evaluation groups) has pioneered a paradigm shift from isolated evaluations of individual UK Extension programs to a focus on collective impact assessment. The SUMH workgroup has served as a vehicle to familiarize UK Extension personnel with the new structure of aggregated success indicators.