Sustainability of Programs

Does Implementation of SWPBIS Enhance Sustainability of Specific Programs, such as Playworks?

Paul M. Meng, Kent McIntosh, Jennette Claassen, and Robert Hoselton University of Oregon

Download PDF

School-wide positive behavioral interventions and supports (SWPBIS) is an example of a framework for implementing school-based practices with fidelity and durability. Critical features of SWPBIS include: (a) teaming, (b) data-based decision making, and (c) using systems for supporting staff in implementation. These features of SWPBIS allow specific practices within the framework to be implemented more fully and sustainably. Teaming and systems of support are necessary components of many practices implemented within schools; they ease the implementation of virtually any practice within this context. Further, data-based decision making could enhance the effectiveness and efficiency of practices wherein concrete outcomes are valued (e.g., increased student achievement, reduced problem behavior). These features of SWPBIS, once in place, could be applicable to an array of potential programs and practices.

 

Playworks is an example of a program schools may elect to implement within or outside of a SWPBIS framework. Playworks Coach is a recess engagement program delivered by an external recess coach. The Playworks Coach model provides elementary schools with a full-time recess coach throughout the day and after school. The coach orchestrates play and physical activity through five core components:

1. Supporting

2. Working with smaller groups through Class Game Time

3. Training peer mentors through the Junior Coach Leadership Program

4. Supporting before and after school time

5. Reinforcing positive teamwork through developmental sports leagues

Emphasis is placed on student-led activities, clearly and positively stated expectations, and creating an inclusive environment for all students to participate in a physically active recess period.

The purpose of this evaluation brief is to assess whether schools implementing SWPBIS with adequate fidelity of implementation were more likely to sustain specific school practices, such as Playworks, than schools not implementing SWPBIS with fidelity. Fidelity can be thought of as the degree to which an intervention or practice is implemented as designed (Gresham, 1989). For the purposes of this brief, fidelity of SWPBIS is the extent to which its critical features are implemented and whether the critical features are implemented at a level meeting established criteria-criteria which have been shown to produce the desired results associated with that practice.

The following question was addressed: To what extent does implementing SWPBIS with fidelity increase the likelihood Playworks will be sustained?

Method

Data Sources

Data for this brief came from archival records across the 2007-08 to 2013-14 school years from 72 schools with information from both of two sources: (a) a record of schools receiving the Playworks Coach intervention, and (b) the PBIS Assessment database (http://www.pbisapps.org). PBIS Assessment is a freely available, web-based data application for entering and analyzing SWPBIS fidelity of implementation data. Schools enter data into PBIS Assessment related to any of the available, validated measures. School teams use these data to track implementation progress over time.

Sustained Implementation of Playworks

Sustained implementation of Playworks Coach was assessed by whether schools elected to continue the Playworks Coach program for one or more years after their first year of implementation.

SWPBIS Fidelity of Implementation

The following four validated measures were used to assess SWPBIS implementation: Benchmarks of Quality (BoQ), School-wide Evaluation Tool (SET), Self-Assessment Survey (SAS), or Team Implementation Checklist (TIC). Each measure has its own threshold criterion for implementation fidelity: either 70% or 80%. Based on at least one of these scores (see McIntosh et al., 2013), the schools were split into two groups: schools implementing SWPBIS with fidelity (i.e., at or above the criterion) and schools not implementing SWPBIS with fidelity.

Results

Of the 72 schools in the dataset, 57% were implementing SWPBIS with fidelity (n = 41), and 43% were not implementing SWPBIS with fidelity (n = 31). Schools implementing SWPBIS with fidelity were more than three times more likely (Odds Ratio = 3.3) to sustain the Playworks Coach intervention than schools below criterion. Over three-quarters (76%) of schools implementing SWPBIS with fidelity sustained Playworks Coach, compared to less than half (48%) of schools not implementing with fidelity.

Figure 1. Likelihood of sustained Playworks Coach implementation for schools by SWPBIS fidelity of implementation (n = 72).

 

Discussion

This evaluation brief sought to examine whether implementing SWPBIS with fidelity makes it more likely a school will sustain a given school-based practice. The results indicate new practices are more likely to be sustained over time when implemented within an existing implementation framework like SWPBIS. Schools included in the evaluation brief were more than three times more likely to sustain the Playworks Coach program if they were implementing SWPBIS with fidelity. The structures within SWPBIS (e.g., teaming, data-based decision-making, and using systems to support staff in implementation efforts) appear to allow school teams to integrate and sustain new, related practices – like Playworks Coach – as part of the framework. The prominence of these features of SWPBIS seems to build schools' capacity to implement a range of practices.

Although the findings of this evaluation are preliminary due to the small sample size, they are consistent with what may be hypothesized given what is known from the existing literature on implementation and sustainability. If an organization invests in putting the framework in place for supporting effective practices, and prioritizes it over time, it appears the framework may create a context within which various school-based practices are more likely to sustain as well.

References

Gresham, F. M. (1989). Assessment of treatment integrity in school consultation and Psychology Review, 18, 37-50.

Kincaid, D., Childs, K., & George, H. (March, 2010). Schoolwide benchmarks of quality (Revised). Unpublished instrument. University of South Florida.

McIntosh, K., Mercer, S. H., Hume, A. E., Frank, J. L., Turri, M. G., & Mathews, S. (2013). Factors related to sustained implementation of school-wide positive behavior support. Exceptional Children, 79, 293-311.

Sugai, G., Horner, R., & Lewis-Palmer, T. (2001). Team Implementation Checklist (TIC). Eugene, OR: Educational and Community Supports. Available at http://www.pbis.org.

Sugai, G., Horner, R., & Todd, A. (2000). PBIS Self-Assessment Survey 2.0. Eugene, OR: Educational and Community Supports. Available at http://www.pbis.org.

Sugai, G., Lewis-Palmer, T., Todd, A., & Horner, R. (2005). School-wide Evaluation Tool (SET). Eugene, OR: Educational and Community Supports. Available at http://www.pbis.org.

 

This project is supported by the U.S. Department of Education, Office of Special Education Programs (OSEP). Opinions expressed herein are those of the authors and do not necessarily represent the position of the U.S. Department of Education