Beyond Measure? The State of Evaluation & Action in Ontario’s Youth Sector

YouthREX Research Summaries ask Just Six Questions of research publications on key youth issues. These summaries get at what the youth sector needs to know in two pages or less!

1. What was this research about?
Ontario’s youth sector provides essential ‘youth relevant’ services and opportunities that are culturally and locally relevant to youth who may not otherwise have access. While youth-serving and youth-focused programs, particularly small grassroots groups, can use evaluation and research to improve their practice and better understand the impact of their work, they often do not have the capacity to engage in ongoing evaluative work, or broadly communicate successful program level practices, models, and outcomes. Without the tools evaluation provides, it is difficult to assess the impact of investments in the youth sector. In order to gain a greater understanding of the current evaluation capacity of the youth sector, YouthREX undertook a province-wide study to examine the experiences of youth-serving agencies with conducting evaluations of their programs and services.

2. Where did the research take place?
The study spanned the province of Ontario, representative of the geographic and regional diversity of youth sector stakeholders across Ontario.

3. Who is this research about?
This study focuses on youth-serving organizations in Ontario’s youth sector. These organizations provide a wide array of services and programs that are delivered by organizations that range in size from large multi-service organizations to small grassroots youth-led organizations and initiatives. The study includes perspectives from Executive Directors of organizations as well as Program Managers, coordinators, frontline workers, evaluators, and capacity builders.

“Youth-sector organizations need and deserve a more fulsome model of evaluation, one that meets them where they are and builds on their strengths while recognizing their importance.”

4. How was this research done?
This multi-focal study was grounded in a Community Dialogue Approach; 197 sector stakeholders completed an online survey and 60 sector stakeholders participated in in-depth interviews about their experiences of evaluation in Ontario’s youth sector.

5. What are the key findings?
Five key messages emerged from findings:

1. Youth sector organizations understand the benefits of evaluation and have enthusiasm for it.
2. There is an urgent need for evaluation processes and practices that can make evaluation less burdensome for programs.
3. Youth programs and the broader youth sector may be missing the opportunity to use evaluation for strategic learning.
4. Youth involvement in evaluation is important for youth wellbeing but doing this meaningfully is still a work in progress.
5. The distinctive characteristics of grassroots youth sector organizations – when compared to mainstream organizations – require a distinct understanding and approach to how these organizations engage with evaluation.

6. Why does it matter for youth work?
Beyond Measure? provides a contextualized examination of the sector’s evaluation strengths, challenges, and areas of opportunity. The report aims to facilitate strategic, critical conversations across diverse stakeholder groups and raise the profile of issues that influence the evaluation capacity and positioning of the youth sector.

This research finds that youth-sector organizations need and deserve a more fulsome model of evaluation, one that meets them where they are and builds on their strengths while recognizing their importance. In order to support this agenda, the researchers put forward the following recommendations:

For Funders

• • Embrace a Learning Evaluation Model
  • Match evaluation expectations to realistic resource constraints
  • Streamline the reporting burden on service-delivering organizations

For Sector-Supporting Organizations (such as YouthREX)

• • Meet youth-serving organizations where they are
  • Develop a strategy for supporting evaluation in Northern, remote, and rural communities
  • Develop innovative, ‘ready to use’ tools and encourage standardization where appropriate
  • Document and curate the tacit knowledge already embedded in the youth sector

For Youth-Serving Organizations

• • Explore the potential of evaluation activities as youth work practice
  • Incorporate evaluation activities into organizational learning and strategic planning
  • Recognize the tacit knowledge already present in your organization

Categorised in: Research Summary