Process Evaluation Examining the Implementation of a Sport-Based Positive Youth Development Program

YouthREX Research Summaries ask Just Six Questions of research publications on key youth issues. These summaries get at what the youth sector needs to know in two pages or less!

1. What is the research about?
Research suggests that sport-based positive youth development programs (i.e., those that focus on young people’s strengths rather than their perceived deficits) can support the development of life skills, such as social responsibility, social competence, transfer of learning, effort, teamwork, and self-control. However, there is little evidence on how program implementation – the ways in which a program is carried out, which can be related to staff practices and program design features – contributes to positive outcomes.

This study assessed the degree to which the staff of a sport-based positive youth development program reported implementing prescribed program design features (i.e., promoting a caring climate, adhering to the program cur­­­ricula, and using effective instructional techniques). The study also examined how s­­­port characteristics (sport type and setting) impacted program implementation.

2. Where did the research take place?
The research took place at a Midwestern university in the United States. Most of the youth participants in the program studied were from economically-disadvantaged communities.

3. Who is this research about?
This study gathered perspectives from 26 adult staff in a sport-based positive youth development program. The program’s youth participants were aged 9 to 15, and either lived below the federal poverty line or were eligible for a reduced-price or free lunch at school. The majority of youth identified as Black and/or African American.

“To promote the development of life skills among youth, staff and program administrators must intentionally design and facilitate sport-based [positive youth development] programs” (p. 72).

4. How was this research done?
This pilot study took place over the course of a 19-day summer program, which included 15 days of curricula and a four-day culminating event. Each day of curricula included four 60-minute sessions: three sport sessions and one classroom-based education session.

The researchers developed a self-reflexive evaluation tool to assess staff’s perceptions of program implementation related to the following program design features:

• Program climate: the creation of a caring climate
• Program curriculum: use of the program curriculum, which aimed to promote life skill development through sport activities
• Program instruction: adherence to the program’s “framing, facilitating, and debriefing framework” (p. 78)

The tool was divided into three sections (one for each program design feature) and included 37 items (e.g., “emotionally safe climate created”, “emphasis on teamwork”, “introduction and demonstration of sport skill”). Staff assessed each item using a 5-point scale, from 0 (‘none’) to 4 (‘total’). The tool was designed to take five minutes to complete.

Program staff were provided with four copies of the implementation log at the beginning of each day of curricula and asked to complete the log after each 60-minute session. Each staff member completed 60 session logs, and a total of 1,260 logs were collected over the course of the program.

The program also provided documentation on the characteristics of each sport session, including type (contact and non-contact) and setting (indoor and outdoor).

Data was analyzed using statistical software. Program implementation across program design features (climate, curriculum, and instruction) was calculated by summing up each staff member’s score, and then calculating the average of all staff members’ scores. Differences in program design features were then tested across sport characteristics (sport type and setting).

5. What are the key findings?
Of the three program design features, program climate was reported to have the highest degree of perceived implementation (90.15%), followed by quality of instruction (84.91%) and curriculum usage (83.73%).

Although there were no statistically significant differences in reported implementation across sport types, non-contact sports were perceived to have a greater degree of implementation than contact sports. Past research has shown that contact sports often involve violence and aggression, which may have made it difficult for staff to promote life skill development and create a caring climate.

Two of three program design features (program climate and curriculum) had a higher degree of perceived implementation for indoor sessions than outdoor sessions. The authors suggest this may be because an indoor setting allowed staff to facilitate sessions with fewer distractions and weather-related considerations.

6. Why does it matter for youth work?
Organizations should consider the key program elements outlined in this study (i.e., program climate, curriculum, instruction, and sport characteristics) when designing, managing, and facilitating sport-based programs.

It is critical for program staff to be equipped with the proper skills, training, and knowledge to effectively facilitate and assess sport-based youth programming. For example, this study suggests that contact and outdoor sports may pose unique challenges for staff in the context of sport-based positive youth development programming.

This study also suggests that the session log tool is a reliable instrument that can be used to assess the implementation of sport-based programming. It can also support capacity-building, as it allows staff to reflect on their work and identify areas for improvement.

Categorised in: Research Summary