Improve Equity

Do Wisconsin Schools Implementing an Integrated Academic and Behavior Support Framework Improve Equity in Academic and School Discipline Outcomes?


Jessica Swain-Bradway, Midwest PBIS Network
Kim Gulbrandson, Wisconsin RtI Center
Anthony Galston, Wisconsin RtI Center
Kent McIntosh, University of Oregon


This evaluation brief describes how Wisconsin is implementing an equitable multi-level system of supports (MLSS) framework, also known as a multi-tiered system of supports (MTSS), and how schools implementing this framework with both a behavior and reading focus have shown positive outcomes for all students. The data analyzed in this brief were derived from a small, Wisconsin RtI Center staff directed project. The focus was on the impact of an equitable multi-level system of supports (MLSS). The data look at schools that engaged in professional development in several MLSS training series that have an explicit focus on equitable outcomes for all students.

The brief responds to three research questions:

  1. Do schools implementing MLSS show improved outcomes and equity in suspension rates?
  2. Do schools implementing MLSS show improved outcomes and equity in the percent of students meeting academic growth benchmarks?
  3. What are the effects of implementing MLSS on the fiscal value of instructional and administrative time saved?

Multi-Leveled System of Supports

A multi-tiered system of supports (MTSS) bridges the perceived gap between academic Response to Intervention (RtI) and Schoolwide Positive Behavioral Interventions and Supports (SWPBIS) and is summarized by McIntosh and Goodman as “...one coherent, strategically combined system meant to address multiple domains or content areas in education” (McIntosh & Goodman, 2016, p. 5). The Wisconsin Department of Public Instruction (DPI) has clearly articulated a statewide MTSS system, referred to as an equitable multi-level system of supports (MLSS) framework for delivery and achievement of equitable academic and social supports to all students in Wisconsin. The explicit investment in equity comes as a result of large historical gaps in school outcomes for students of color and students with IEPs. For example, a 2015 Civil Rights Project Report showed Wisconsin as having the second-largest gaps in suspension rates between black and white elementary aged students, and the largest gap in a comparison of 48 states for secondary students (Losen, Hodson, Keith, Michael, Morrison, & Belway, 2015).

Wisconsin DPI began the initial statewide investment in an equitable multi-level system of supports (MLSS) approach in 2009, starting with a focus on SWPBIS. Statewide trainings were provided through initial partnership with the Illinois PBIS Network. Between the 2009-10 and 2016-17 school years, 1,803 (81%) of the 2,214 public schools in Wisconsin participated in professional learning offered by the Wisconsin RtI Center.

Wisconsin RtI Center Equitable MLSS Framework

Unique to Wisconsin’s vision of an equitable multi-level system of supports (MLSS) framework is that equity is both at the center of outcomes, and also embedded into all of the key system features. This overt attention is intended to challenge and change historical inequitable access, opportunity and outcomes experienced by learners currently underserved in Wisconsin schools. The Wisconsin RtI Center, guided by DPI’s aligned vision and in partnership with Cooperative Educational Service Agencies (CESAs), has developed considerable resources to assist schools to implement, and sustain MLSS. These include: (a) a statewide, comprehensive model to inform culturally responsive educational practices; (b) MLSS trainings developed and implemented through a lens of equity, including trainings specifically targeting the advancement of equity and fluency in culturally responsive practices within the system; (c) data and tools to examine both the systems (fidelity) and impact on equitable outcomes for all students; and (d) a cadre of technical assistance coordinators to provide feedback and guidance on MLSS.

A Comprehensive Model to Inform Culturally Responsive Practices

Being culturally responsive is an ongoing process of reflection on both (a) the outcomes for students within an organization, and also (b) a reflection on the system and the component parts that shape student and educator behaviors. Wisconsin’s comprehensive model to inform culturally responsive practices emphasizes the process of building teacher fluency in practices, data to guide decision-making, and systems that support adult behaviors to ensure that every student is successful. The model identifies and describes eight necessary actions for cultural responsiveness: (a) develop self-awareness of privilege; (b) examine systems’ impact on families and students; (c) believe all students will learn; (d) understand we all have unique identities and worldviews; (e) know your communities; (f) lead, model, and advocate for equity practices; (g) accept institutional responsibility; and (h) use practices and curricula that respect students’ cultures. These eight necessary actions are made explicit in the Building Culturally Responsive Systems (BCRS) training, and are also embedded into the other MLSS training series.

Multi-Level System of Supports (MLSS) Trainings

The MLSS trainings collectively ensure educators develop comprehensive MLSS systems through Tier 1/Universal SWPBIS, Reviewing Universal Reading, and the Building Culturally Responsive Systems (BCRS) Training Series.

Tier 1/Universal SWPBIS. The four-day, Tier 1/Universal SWPBIS training prepares school teams for implementation of SWPBIS at the universal level. Training focuses on the critical features required to install systems and develop mindsets needed to create a positive school culture. The sequence includes establishing leadership structures and identifying the adult practices related to student behavior. It also includes systems level data for decision-making, and actions plans supporting stakeholders in sustainable implementation. The Wisconsin RtI Framework training provides school leadership teams with the most current information, messaging, and resources to develop and refine their local RtI framework.

Reviewing Universal Reading. The Reviewing Universal Reading training builds on the Tier 1/Universal SWPBIS systems, data, and practices and guides teams through implementing the systems and practices specific to delivery of effective reading instruction at the universal level. Teams learn evidence-based practices that support every student at the universal level, and action plan for school improvement around the universal reading instruction. These trainings both provide foundations for, and complement, the Building Culturally Responsive Systems (BCRS) Training Series, which highlights the explicit priority for equitable student outcomes.

Building Culturally Responsive Systems (BCRS) Training Series. Wisconsin’s MLSS key system features and model to inform culturally responsive practices frame the Building Culturally Responsive Systems (BCRS) training series. This series engages participants in activities, including self-reflection on how to systematically advance understanding of policies and practices that have direct implications for inequitable student outcomes. The curricular activities in the series allows participants to examine the subtleties within culture and diversity and equity. The long term goal for educators and administrators is to be more culturally competent and work cohesively as a school staff to establish culturally responsive school environments for achieving more equitable academic and social-behavioral school outcomes. The three primary areas of focus for the BCRS training are:

  1. Self-awareness. The practitioner’s awareness of self is critical to the equity work because both individual and collective practitioner identity have a strong influence on decisions about policies, practices and procedures in schools or districts (Helms & Cook, 1999), such as disciplinary actions (Smolkowski, Girvan, McIntosh, Nese & Horner, 2016; Skiba et. al, 2011). BCRS training participants engage in activities to increase self-awareness related to how their own ethnicity, culture, and life experiences may affect other adults and students in the school system. Participants are also guided to analyze their organizations’ systems, structures, policies, practices, vision, mission, and values in relation to achieving equitable academic and social-behavioral outcomes for all students.
  2. Culturally responsive classroom practices. School leadership teams learn to create more inclusive learning environments, including developing or refining academic curricula and classroom practices that include and honor the life experiences and cultures of students (Hollie, 2012). Examples of culturally responsive practices emphasized through BCRS training and ongoing supports include imaging, incorporating literacy books that match demographics, learning environment surveys, call and response techniques, protocols for increasing student engagement, and culturally responsive read-aloud activities.
  3. Use of data. Participants use various types of data to make meaning for systems change. Participants analyze their own disaggregated school data to identify the areas of greatest need for impact. They also utilize additional data sources (including family voice) to develop action plans to address specific equity issues within their school. These issues may be over or under representation, or outcomes, of a subpopulation related to office referrals, special education placements, or intervention access or success. Schools then identify the specific practices required to address gaps in practice, systems, and data. By the conclusion of the six-day training, schools have a fully articulated action plan to impact equitable and culturally responsive practices within their organizations.

Data Resources

The framework, training and resources provided by the Wisconsin RtI Center personnel support school teams to understand their institutional responsibility for equitable, or inequitable outcomes. Data resources that provide insights into equity of outcomes, include the risk ratio tool, and WISEDash, the state data dashboard, are supported through trainings, networking and technical assistance, and are made available on the Wisconsin RtI Center website.

Fidelity tools, such as the School-Wide Implementation Review (SIR), created by the Wisconsin RtI Center to assess academic RtI, and the SWPBIS Tiered Fidelity Inventory (TFI), are used to support fidelity of the framework. The goal is for schools and districts to disaggregate data, examine the strengths and needs of their systems, use those data to identify root causes for inequitable outcomes and inform changes in implementation.

Method

Research Questions:

  1. Do schools implementing MLSS show improved outcomes and equity in suspension rates?
  2. Do schools implementing MLSS show improved outcomes and equity in the percent of students meeting academic growth benchmarks?
  3. What are the effects of implementing MLSS on the fiscal value of instructionaland administrative time saved?

Sample

The Wisconsin RtI Center team assessed outcomes for 65 schools. To be included in the analyses, schools needed to have completed the Wisconsin RtI Center training series in the following content areas: MLSS Framework and/or Universal Reading, and SWPBIS tier 1. They also needed to have implemented an MLSS with adequate fidelity of implementation in both behavior and reading for at least 2 of 3 years (2013-14, 2014-15, 2015-16), as measured by their Benchmarks of Quality (BoQ), TFI and SIR-Reading systems assessments.

1. Do schools implementing MLSS show improved outcomes and equity in suspension rates?

Wisconsin schools implementing both behavior and academic-focused equitable MLSS with fidelity had 2,204 fewer students receiving out of school suspensions (OSS) between 2009-10 and 2015-16. A one-way repeated measures ANOVA showed a statistically significant reduction over time, Wilks’ λ = 0.61, F(1,64) = 40.47, p < 0.001, η2 = 0.39.

Figure 1 shows the suspension rate change for students as a whole and by subgroup of interest. The largest decrease in the suspension rate was for Black students, with a 9.67% decrease (1,710 fewer Black students suspended). This reduction was statistically significant, Wilks’ λ = 0.50, F(1,64) = 65.22, p < 0.001, η2 = 0.51. Similar and statistically significant effects were seen for students with individualized education plans (IEPs; 851 fewer students suspended), Wilks’ λ = 0.85, F(1,64) = 11.77, p < 0.001, η2 = 0.16.

Figure 1. Decreases in out of school suspension rates from 2009-10 to 2015-16 for All, Black, Hispanic, and Students with IEPs.

2. Do Schools Implementing MLSS Show Improved Outcomes and Equity in the Percent of Students Meeting Academic Growth Benchmarks?

Schools implementing both behavior and reading-focused equitable MLSS with fidelity had increased academic achievement over time, with 1,058 additional students (7.28%) meeting or exceeding Measures of Academic Progress (MAP) reading benchmarks between 2011-12 and 2014-15. A one-way repeated measures ANOVA showed a statistically significant increase over time, Wilks’ λ = 0.76, F(1,52) = 16.29, p < 0.001, η2 = 0.24.

Figure 2 shows that implementing schools had improved achievement for all groups of interest. Black students had the largest gains, with an 11.78% increase in students meeting or exceeding benchmarks (742 students), Wilks’ λ = 0.72, F(1,46) = 17.78, p < 0.001, η2 = 0.28. There were also statistically significant increases for Hispanic students (394 additional students), Wilks’ λ = 0.81, F(1,48) = 11.31, p < 0.01, η2 = 0.19. In addition, there were statistically significant effects for students with IEPs (116 additional students), Wilks’ λ = 0.87, F(1,51) = 7.57, p < 0.01, η2 = 0.13.

 

Figure 2. Increases in percentage of students meeting or exceeding projected MAP growth from 2011-12 to 2015-16 for All, Black, Hispanic, and Students with IEPs.

3. What are the Effects of MLSS on the Fiscal Value of Instructional and Administrative Time Saved?

The sample schools saw gains in administrative and instructional time from reduced rates of out of school suspensions (OSS). Administrative and instructional time saved, and value of time saved was estimated using conservative recommendations by Scott and Barrett (2004), and publicly available 2016-17 staff salary (2017a) and average instructional costs per pupil (2017b) reports from the Wisconsin Department of Public Instruction (DPI). The administrative days saved, for all students was equivalent to 601 days, for a total financial value of $39,247.34 of administrative time.

Reductions in suspensions were also calculated as saved instructional time for students. Overall, the reductions in suspensions were equivalent to an increase of 10,525 school days for all students represented in the fidelity cohort. For Black students, this represented a savings of 8,366 instructional days, and Hispanic students, of 1,103 days. These recouped instructional days represent a “re-investment” of $536,320.56 and $70,710.20 respectively. Although these savings do not affect a school’s overall budget, those monies represent the value of the instructional time saved due to a decrease in out of school suspensions, that otherwise, would have been spent on an empty seat in the classroom. While this brief emphasizes the immediate, short term cost savings estimates, there is also long-term impact on quality of life for students and overall financial impact to our systems. The long-term impact estimates related to factors such as dropouts, workforce productivity and health, can be made based on the research of others, such as Rumberger (2016) and Rosenbaum (2018).

Summary

Although these data represent a limited sample of 65 schools in Wisconsin, and the findings were not generated through experimental design, the documented student discipline and MAP reading impact suggest that schools implementing a behavior and reading-focused equitable MLSS with fidelity over time realized significant suspension reductions and academic (reading) gains. Improvements were also seen for Black and Hispanic students, and students with Individualized Education Plans (IEPs), which suggest that schools receiving and implementing equitable MLSS supports with fidelity are to some extent removing obstacles to learning for students who have been historically marginalized.

This impact is in part related to sustained investment in the Wisconsin RtI Center’s culturally responsive, data-driven, equitable MLSS framework, made possible by the alignment of resources through Wisconsin RtI Center, a strong commitment by the Wisconsin Department of Public Instruction (DPI), and adherence to promising and best practices.

References

Algozzine, B., Barrett, S., Eber, L., George, H., Horner, R. H., Lewis, T. J., Putnam, B., Swain-Bradway, J., McIntosh, K. & Sugai, G. (2014). School-wide PBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. www.pbis.org.

Helms, J.E., & Cook, D.A. (1999). Using race and culture in counseling and psychotherapy: Theory and process. Needham Heights, MA: Allyn & Bacon.

Hollie, S. (2012). Culturally and linguistically responsive teaching and learning: Classroom practices for student success. Huntington Beach, CA: Shell Education.

Kincaid, D., Childs, K., & George, H. (March, 2010). Schoolwide Benchmarks of Quality (Revised). Unpublished instrument. USF, Tampa, Florida.

Losen, D., Hodson, C., Keith, M.A., Morrison, K., & Belway, S. (2015). Are we closing the student discipline gap? Los Angeles, CA: The Center for Civil Rights Remedies.

McIntosh, K., & Goodman, S. (2016). Integrated multi-tiered systems of support: Blending RTI and PBIS. New York: Guilford Press.

Rosenbaum, J. (2018). Educational and criminal justice outcomes 12 years after school suspension. Youth & Society, 1-33.

Rumberger, R.W., Losen, D.J. (2016). The high cost of harsh discipline and its disparate impact. Los Angeles, CA: The Center for Civil Rights Remedies, The Civil Right Project.

Scott, T. M., & Barrett, S. B. (2004). Using staff and student time engaged in disciplinary procedures to evaluate the impact of schoolwide pbs. Journal of Positive Behavior Interventions, 6, 21-27.

Skiba, R. J., Horner, R. H., Chung, C.-G., Rausch, M. K., May, S. L., & Tobin, T. (2011). Race Is Not Neutral: A National In- vestigation of African American and Latino Disproportionality in School Discipline. School Psychology Review, 40, 85-107.

Smolkowski, K., Girvan, E. J., McIntosh, K., Nese, R. N. T. & Horner, R. H. (2016). Vulnerable decision pointsfor disproportionate office discipline referrals: Comparisons of discipline for African American and White elementary school students. Behavioral Disorders, 41, 178-195.

Wisconsin Department of Public Instruction. (2017a). Ad- ministrative salary reports [Data file]. Retrieved from https:// dpi.wi.gov/cst/data-collections/staff/published-data

Wisconsin Department of Public Instruction. (2017b). Com- parative cost per member [Data file]. Retrieved from https:// dpi.wi.gov/sfs/statistical/cost-revenue/section-d#Item%201a

Wisconsin’s framework for equitable multi-level systems of supports (n.d.). Retrieved from https://dpi.wi.gov/sites/de- fault/files/imce/rti/pdf/rti-emlss-framework.pdf

Wisconsin’s model to inform culturally responsive practices (n.d.). Retrieved from https://www.wisconsinrticenter.org/ assets/files/resources/Model%20to%20Inform%20Culturally%20Responsive%20Practices.pdf

Wisconsin RtI Center (2015). School-wide Implementation Review. Retrieved from https://wisconsinrticenter.org/educa- tors/rti-in-action/self-assessment-tools.html

 

Suggested Citation for This Publication

Swain-Bradway, J., Gulbrandson, K., Galston, A., & McIntosh, K. (2019). Do Wisconsin Schools Implementing an Integrated Academic and Behavior Support Framework Improve Equity in Academic and School Discipline Outcomes? OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. www.pbis.org.