Positive
Behavioral Interventions & Supports Newsletter
Contents
Using Information in State or District Level Implementation of School-wide PBS
Rob Horner, Rachel Freeman, C. Michael Nelson,
and George Sugai
Introduction
Implementing School-wide PBS at the State or District Level
To date, PBS has been implemented primarily at the school level. The goal has been to emphasize that behavior support in schools needs to focus not just on active design of individualized interventions, but on school-wide systems that emphasize prevention. As documentation of the value of investing in school-wide behavioral systems has improved, states and districts throughout the country have initiated planning to determine how School-wide PBS can be implemented, not just within a few “demonstration schools,” but across large numbers of schools within a state/district.
The National Technical Assistance Center for Positive Behavioral Interventions
and Supports (PBIS) has proposed a blueprint for large-scale implementation
of school-wide PBS. This blueprint recommends the following four components
for successful implementation: (a) a Leadership Team to actively coordinate
implementation efforts; (b) an organizational umbrella composed of adequate
funding, broad visibility, and consistent political support; (c) a foundation
for sustained and broad-scale implementation established through a cadre
of individuals who can provide coaching support for local implementation,
a small group of individuals who can train teams on the practices and
processes of school-wide PBS, and a system for on-going evaluation; and
(d) a small group of demonstration schools that documents the viability
of the approach within the local fiscal, political and social climate
of the state/district (Sugai, 2002). Figure 1 illustrates the integration
of the core elements of this large-scale implementation blueprint.
Figure 1: Blueprint for broad implementation of school-wide PBS.
Follow this link to view the Blueprint which describes the organization of district or state level implementation of School-wide PBS. This blueprint contains: (a) the elements and logic for going to scale at the district or state level, (b) a self-assessment instrument, and (c) an action planning process. The purpose of the blueprint is to assist districts and states in gathering and using information to guide and assess large-scale implementation efforts.
Using Information to Guide Large-scale Implementation of School-wide PBS
Gathering and using data for decision-making is emphasized at every level
of school-wide PBS implementation (i.e., individual student, classroom,
school, district, state). The core ideas for gathering and using data
upon which to base educational decisions remain as useful today as decades
ago when they were first defined (Deno & Mirkin, 1978; Gilbert, 1978;
Wolery, Bailey, & Sugai, 1988). To be maximally useful in the implementation
of effective behavior support, data should be (a) an accurate reflection
of behavior, (b) collected with consistency and precision, (c) straightforward
and simple to collect, (d) easily summarized and reported on a regular
basis (weekly, monthly, quarterly, etc), and (e) reviewed regularly and
systematically in response to specific evaluation questions to ensure
meaningful action planning can be supported (Horner et al., in print).
When a district or state begins organizing to implement school-wide PBS,
data should be gathered and used to address the following questions:
1. Is the Leadership Team providing the resources, vision, and systems
needed for large-scale implementation?
The ultimate success of efforts to produce broad-scale systems change
often lies with the on-going details of implementation and the initial
planning and organization of the effort. Before a Leadership Team initiates
the training and technical assistance associated with school-wide PBS,
care should be taken to ensure that the Leadership Team is composed of
appropriate members, has allocated sufficient professional time to allow
effective coordination, and has built an action plan to deliver the full
set of elements defined in Figure 1. The Leadership Team can use the PBS
Leadership Team Self-Assessment and Planning Tool to assess if key
elements are in place and identify the action plans needed to attain those
elements not yet in place. The PBS Self-assessment tool provides a formal
structure for documenting whether key elements are in place and a direct
format for organizing an action plan to obtain missing elements.
A frequent error of Leadership Teams is to implement training and technical
assistance efforts without first establishing a plan for broad-scale implementation
and evaluation. Taking time to self-assess and to begin the coordination
of organizational elements associated with success may delay the speed
with which initial efforts are initiated; however these preliminary efforts
will result in greater fidelity and durability of implementation.
2. Are schools implementing school-wide PBS?
The Leadership Team should coordinate training for local teams of faculty/staff/administrators
within target schools. A core concern for the Leadership Team will be
not just whether training was provided and perceived as helpful, but also
whether the training actually changed the way behavior support systems
are organized in the schools. The
PBS Planning and Implementation Self-Assessment Checklist is a self-assessment
document used every month (or at least every quarter) to assess whether
the activities associated with implementation of school-wide PBS systems
have been completed. If activities have not been completed as scheduled,
the team should define action steps and time lines for moving towards
full implementation. The Self Assessment Checklist provides the local
team with valuable and concrete tools (on-going prompts and action planning
steps) that can be used to document an overall index of perceived implementation
progress and can be reported to the Leadership Team. The time invested
in collecting the Self Assessment Checklist information is no more than
30 minutes per assessment and results in a summary similar to that provided
in Figure 2 from an actual (though renamed) elementary school. The team
reported these Self Assessment Checklist data to the Leadership Team in
September, October, and December of 2002 during a time period when training
in school-wide PBS practices was being provided. The results document
progress across time and provide the Leadership Team with a simple strategy
for assessing staff perception of implementation outcomes. The data are
arrayed by each system component on the Self Assessment Checklist (commitment
established; school team developed; school-wide self-assessment completed
and action plan developed; prevention practices implemented; student behavioral
information gathered, summarized and used for decision-making; function-based
support available for individual intervention plans). The percent of total
points is gathered (1 point is allocated for an item being “in progress”
and 2 points are allocated for an item “completed”) and summarized for
a percent of total items implemented.
Figure 2: Self Assessment Checklist data for one elementary school
across a four-month period.
Leadership Teams also may wish to have a more formal evaluation of the
extent to which teams are implementing school-wide PBS practices and systems.
When a more rigorous assessment is needed, the Leadership Team may choose
to have external evaluators use the School-wide
Evaluation Tool (SET) (Horner et al., in press; Sugai, Lewis-Palmer,
Todd, & Horner, 1999). The SET involves a 2-3 hour review of systems
outcomes conducted in the school by an outside evaluator. The results
provide an acceptable psychometric measuring the extent to which PBS practices
and systems are in place within a school.
3. Does implementation of school-wide PBS affect student behavior?
Changes in school discipline systems will be of little importance unless
the results translate into substantive change in student behavior and
improvement in the ability of the school to build academic and social
competence. Among the most common measures of improvement in student behavior
is the rate of office discipline referrals in the school. Office discipline
referral (ODR) levels can reflect teacher behavior or administrative systems
as well as student behavior; therefore, ODR data must be carefully interpreted.
The validity of ODR data as a basic index of behavior change in schools
has gained credibility as more systematic approaches to ODR measurement
have been adopted (Irvin, Tobin, Sprague, Sugai, & Vincent, 2003).
Schools focusing on behavioral climate should gather and report information
to administrators, behavior support teams and faculty on (a) the average
number of ODRs per day per month, (b) the frequency of ODRs per type of
problem behavior, (c) the frequency of ODRs per student and (d) the frequency
of ODRs per location in the school. This information should be available
continuously for the administrator and behavior support team, at least
monthly for those managing change in behavior support systems, and quarterly
for the whole faculty.
Detailed information about ODRs, coupled with on-going counts of disciplinary
consequences (e.g., suspensions and expulsions) allows a faculty and staff
to monitor whether their behavior support efforts are effective. Recent
web-based data summary and reporting systems (e.g., SWIS.org)
make these data inexpensive and accessible to all schools. Preliminary
evaluation results suggest that regular access to ODR, suspension, and
expulsion data is among the most efficient ways for faculty teams to build
school-wide commitment for, and organize effective implementation of,
school-wide PBS (Ingram, Horner & Todd, 2002). An example of one chart
used for data-based decisions in schools is provided in Figure 3. The
average number of ODRs per school day per month may be compared across
months to identify both levels and trends that can guide the decision-making
within the school.
Figure 3: Average number of office referrals per school day per
month.
4. Are schools implementing school-wide PBS perceived as safer?
Student perception of school safety is associated with violence in schools
(Gottfredson, Gottfredson & Skroban, 1996). When students perceive
their school to be a dangerous place, they are more likely they are to
engage in problem behaviors (e.g. weapons violations) that they view as
necessary for their personal safety. The Oregon School Safety Survey is
an instrument developed to obtain an efficient and reliable index of perceived
school safety (Sprague, Colvin & Irvin, 1995). Leadership Teams investing
in broad implementation of school-wide PBS will benefit from documentation
of the absolute and relative level of perceived school safety. The Oregon
School Safety Survey provides a summary of “risk factors” and “protective
factors” that can be useful in determining training and support needs
related to school safety and violence prevention.
5. Does implementation of school-wide PBS allow schools to develop
better support for students with the most extreme needs?
A final question of relevance for Leadership Teams is the extent to which
schools are improving support for students with the most extreme behavior
support needs. Preliminary results suggest that when schools implement
primary level prevention efforts they (a) identify fewer students as needing
intensive behavior plans, and (b) are more successful in their support
of these students (Lewis, Newcomer, & Powers, 2003; Lewis-Palmer et
al., 2002). The Individual Student Systems Evaluation Tool (I-SSET) is
an instrument for assessing the extent to which the core systems needed
for effective individual student behavioral interventions are in place.
The I-SSET produces a total score and sub-scale scores that the Leadership
Team can use to determine the extent to which a school has key systems
in place.
Summary
School-wide PBS is being implemented in an increasing number of schools
throughout the nation. As states, regional agencies, and school districts
begin designing more extensive implementation plans, care should be taken
to identify the key information sources used to guide and evaluate large-scale
implementation efforts. This article suggests one set of questions that
Leadership Teams may use for evaluation and data sources that have proven
effective in implementation efforts (see Figure 4).
Figure 4. Evaluating school-wide PBS efforts.
References
Chapman, D. & Hofweber, C. (2000). Effective behavior support in British
Columbia. Journal of Positive Behavior Interventions, 2, 235-237.
Colvin, G., & Fernandez, E. (2000). Sustaining effective behavior
support systems in an elementary school. Journal
of Positive Behavior Interventions, 2, 251-253.
Deno, S.L., & Mirkin, P.K. (1978). Data-based program modification: A manual. Reston, VA: Council for Exceptional Children.
Gilbert, T.F. (1978). Human competence: Engineering worthy performance.
New York: McGraw Hill.
Gottfredson, D.C., Gottfredson, G.D., and Skroban, S. (1996). A multimodel
school based prevention demonstration. Journal of Adolescent Research,
11, 97-115.
Horner, R.H., & Sugai, G. (2000). School-wide behavior support: An
emerging initiative (special issue). Journal of Positive Behavioral
Interventions, 2, 231-232.
Horner, R.H., Todd, A.W., Lewis-Palmer, T., Irvin, L.K., Sugai, G., &
Boland, J.B. (In print). The school-wide evaluation tool (SET): A research
instrument for assessing school-wide positive behavior support. Journal
of Positive Behavior Interventions.
Ingram, K., Horner R.H. & Todd, A.W. (2002). Program evaluation
of the school-wide information system (SWIS). Unpublished paper.
Irvin, L.K., Tobin, T.J., Sprague, J.R., Sugai, G., & Vincent, C.G.
(2003) Validity of office discipline referral measures as indices of school-wide
behavioral status and effects of school-wide behavioral interventions.
Ms submitted for publication.
Kellam, S., Mayer, G., Rebock, L., Hawkins, G., & Wesley, E. (1998).
Effects if improving achievement on aggressive behavior and of improving
aggressive behavior on achievement through two preventative interventions:
An investigation of causal paths. In B. Dohrenwend (Ed.). Adversity,
stress, and psychopathology. (pp. 486-505). New York: Oxford UP.
Lewis, T.J., Newcomer, L., & Powers, L. (April, 2003). Critical
features of student-centered PBS plans. Presentation at the Annual
Convention of the Council for Exceptional Children. Seattle, WA.
Lewis, T. J., & Sugai, G. (1999). Effective Behavior Support: A systems
approach to proactive school-wide management. Focus on Exceptional Children,
31(6), 1-24.
Lewis-Palmer, T., Horner, R.H., Sugai, G., Eber, L., & Phillips, D.
(2002). Illinois Positive Behavior Interventions and Support Project:
2001-2002 Progress Report. University of Oregon: OSEP Center on Positive
Behavior Support.
Lohrman-O'Rourke, S., Knoster, T., Sabatine, K., Smith, D., Horvath, B.,
& Llewellyn, G. (2000). School-wide application of PBS in the Bangor
Area School District. Journal of Positive Behavior Interventions,
2(4), 238-240.
Nakasato, J. (2000). Data-based decision making in Hawaii's behavior support
effort. Journal of Positive Behavior Interventions, 2, 247-251.
Nersesian, M., Todd, A., Lehmann, J., & Watson, J. (2000). School-wide
behavior support through district-level system change. Journal of
Positive Behavior Interventions, 2, 244-247.
Sadler, C. (2000). Effective behavior support implementation at the district
level: Tigard-Tualatin School District. Journal of Positive Behavior
Interventions, 2(4), 241-243.
Schneider, T., Walker, H.M., & Sprague, J.R. (2000). Safe school
design: A handbook for educational leaders. Eugene, OR: ERIC Clearinghouse
on Educational Management, College of Education, University of Oregon.
School Wide Information System (2000). What is Swis? Retrieved
from the World Wide Web June 23, 2000: http://www.swis.org
Sprague, J., Colvin, G., & Irvin, L. (1995). Oregon School Safety
Survey (OSSS). Institute on Violence and Destructive Behavior, University
of Oregon, Eugene.
Sugai, G. (October 2002). School-wide positive behavior support.
Presentation at the National Forum on School-wide PBS. Naperville, Illinois.
Sugai, G., & Horner, R.H. (2002). The evolution of discipline practices:
School-wide positive behavior supports. Child & Family Behavior
Therapy, 24(1/2), 23-50.
Sugai, G, Lewis-Palmer, T., Todd, A.W., & Horner, R.H. (1999). School-Wide
Evaluation Tool. Eugene: University of Oregon, Positive Behavioral
Intervention and Supports Technical Assistance Center.
Taylor-Greene, S., Brown, D., Nelson, L., Longton, J., Gassman, T., Cohen,
J., Swartz, J., Horner, R.H., Sugai, G., & Hall, S. (1997). School-wide
behavioral support: Starting the year off right. Journal of Behavioral
Education, 7, 99-112.
Taylor-Greene, S.J., & Kartub, D.T. (2000). Durable implementation
of school-wide behavior support: The high five program. Journal of
Positive Behavior Interventions, 2, 233-235.
Wolery, M., Bailey, D. B., & Sugai, G. M. (1988). Effective teaching:
Principles and procedures of applied behavior analysis with exceptional
students. Boston: Allyn & Bacon.
Summary of School-wide PBS Tools
PBS Leadership Team Self-Assessment and Planning Tool
Blueprint
Self-Assessment for Implementing School-wide PBS
PBS
Planning and Implementation Self-Assessment Checklist
Oregon School Safety Survey (OSSS)
School-wide Evaluation Tool (SET)
The School-wide Information System (see www.swis.org)
Click here for additional tools and resources related to School-wide PBS
All content of this newsletter and the website are the property of the Technical Assistance Center on Positive Behavior Support. Duplication without express written consent by authors is strictly prohibited. For any questions please contact: mailto:riffy@ku.edu