Evaluating Youth Drop-In Programs: The Utility of Process Evaluation Methods

YouthREX Research Summaries ask Just Six Questions of research publications on key youth issues. These summaries get at what the youth sector needs to know in just two pages or less!

1. What is the research about?
Neighbourhood youth centres offer essential services for marginalized youth, typically through a combination of structured and drop-in programs. These centres must be able to effectively evaluate their programs in order to identify challenges and opportunities for improvement. While structured programs allow for the collection of outcome data, drop-in programs require a more flexible evaluation approach.

This study addresses some of the challenges of evaluating drop-in programs and outlines the value of using process evaluation methods. It is “a much needed ‘real-world’ example of how process evaluations can effectively support youth drop-in programming” (p. 153).

2. Where did the research take place?
Data was collected at a youth centre in a multicultural, densely populated, low-income neighbourhood in Windsor, Ontario.

3. Who is this research about?
The research is about youth aged 12-19 who attend drop-in activities at a neighbourhood youth centre.

“Recognizing that there are many challenges to bridging research and practice, well-designed process evaluations allow service providers to explore barriers to implementation and establish necessary changes to programs in real-time through feedback provided to those who are in a decision-making role” (p. 161).

4. How was this research done?
This study draws on data from a process evaluation: a method that examines “what happened to who, and how in a program” (p. 154). This type of evaluation is useful for identifying opportunities for program improvement. The authors draw on three types of process data: overall attendance, activity attendance, and qualitative feedback from program participants.

Overall and activity attendance data were provided by the youth centre. Qualitative feedback was collected using a feedback form developed by the evaluation team. Participants were asked about their reasons for attending the youth centre, aspects they would like to change, their perceptions of the building and staff, and the likelihood they would recommend the centre to their peers. Participation in the study was voluntary, and data was kept confidential and anonymous.

5. What are the key findings?
Analysis of attendance data showed that the youth centre serves more young men than women, and that it is especially busy during March Break. The qualitative feedback revealed that the centre’s drop-in basketball program was not working as envisioned: rather than acting as mentors, older youth prevented the younger players from fully engaging in the game. This finding led to the implementation of a new training and orientation program for older participants, illustrating the significance of evaluation data in enabling programs to respond to the needs, concerns, and experiences of youth.

6. Why does it matter for youth work?
This study provides an example of an effective process evaluation method for youth drop-in programs. The authors suggest that including qualitative feedback in evaluations gives organizations “additional valuable information about service delivery that cannot be captured as effectively in any other way” (p. 161). Collecting this type of feedback from youth can also help organizations interpret administrative data, such as attendance records.

The authors argue that process evaluation methods should be considered valuable because they allow organizations to make timely changes in programming that reflect the needs of program participants. Young people’s feedback can also inform decisions about resource allocation and future programming.

Categorised in: Research Summary