Taking the Youth Program Evaluation Plunge: Starting Where You Are and Figuring Out What You Need to Know
Members of the youth sector understand the youth they work with and design engaging and meaningful activities that make an impact on youths’ lives.
At the same time, funders ask organizations to report on the impact that their programs have by using quantitative and qualitative methods and metrics, approaches that often seem removed from the primary goal of engaging and supporting youth. The rationale behind evaluating and reporting is to demonstrate how a program is working. In a reality where funding dollars are limited, understanding and communicating how a program is working improves its chances of receiving and maintaining funding support. Youth programs understand the importance of evaluation, but often need resources and support to make it happen.
Evaluation Challenges and Needs at YouthREX’s Northeastern Hub
YouthREX launched our Customized Evaluation Supports program, in the Northeastern Hub, just under a year ago in order to support youth programs through their evaluation journey. In the process, some of our organizational partners have expressed that they are intimidated by the evaluation process. One program lead expressed that she didn’t know what she should be doing in terms of the evaluation, but that she knew it needed be “evidence-informed”. Another client said, “I know I’m doing good work; I just don’t know how to prove it.”
Our Customized Evaluation Supports organizational partners know that youth in our smaller communities are benefitting from their programming, but effectively measuring activities and communicating outcomes is another story. Adopting an evaluation strategy and implementing it in a way that is complementary to their day-to-day organizational workflow can be a challenge. Through our CES service, our goal is not to turn the staff we’re working with into data scientists, but to help them think in new ways and use the tools they already have to collect quality data that they can use to effectively share the impact of their program! We see our role as supporting our organizational partners through these challenges and fears by making evaluation more accessible and beneficial for them.
We believe that the evaluation process should be organic and complement the daily activities of the organization.
YouthREX’s Customized Evaluation Supports Approach
Working with youth programs to build their capacity to better understand their data, identify successes and challenges, and improve the work they are doing is essentially what program evaluation is all about. The key is not only to integrate reflection and analysis into each step of the program and administration, but also to document this work and think critically about evaluation at each step. We do this by helping our organizational partners use the tools they have on hand. They are experts in the programs they deliver and know what works.
We begin our partnerships by trying to gain a thorough understanding of the program and the services they offer. We do this through the development of a program logic model and/or theory of change. This is an important part of the process that helps guide the evaluation plan and provides a comprehensive understanding of how and why the program is supposed to work. Once this is accomplished, together, we explore what they hope to discover through the evaluation process.
Below are some of the questions we ask CES partner organizations to help get them thinking about their evaluation plans. They may be helpful for you to consider if your program is thinking about embarking on an evaluation journey!
1. What do you want to evaluate?
This question can be daunting. Among the many things we talk about are: Are we counting participants? Attendance? Feedback? What would be useful to know? What could we use to refine the program delivery to ensure we are doing the most effective job? It’s important to realize that there is no one metric that fully explains a program. Indeed, even with all the numbers gathered, it’s still important to talk to the youth to obtain insights about their experiences. Ideally, we will end up with evaluation tools that explain how the program is accomplishing some of its goals.
2. What evaluative work are you already doing?
Most organizations collect some type of information. The critical question is, where does this information go? If numerical information is already being collected, it’s simply a matter of putting it into a spreadsheet or database and build tools that analyze the data automatically. Using the tools and data already available in the organization is a key first step to integrating evaluation into your work. It’s important to note that the evaluation process should not get in the way of the work that the organization is doing. For example, burdening youth programs with extra data entry work or too many surveys are not great ideas, because in the end it may lead to fatigue and won’t be effective in collecting the right data over time.
3. What resources do you need to support your evaluation journey?
Not everyone approaches evaluation from the same starting point. While some organizations may find it easy to check off names on a clipboard and enter the data once a month, others may collect data electronically and measure it in a database. It’s important not to reinvent the wheel and to focus on the program’s existing strengths. The goal is to build a system that enables data entry and analysis, and works with the capabilities of the organization. There are many free to low cost options for software that we can use, and often, we will create tools to match the client’s software and maximize compatibility.
Program evaluation can seem intimidating at first, but once a youth program takes the initial plunge, you can learn amazing insights about your work and how to improve it!
We encourage you to learn more about evaluation, and YouthREX’s approach to evaluation in the links below.