Patterns/Predictors of CICO
- Evaluation Briefs
- Patterns/Predictors of CICO
- Economic Costs
- School Climate & SWPBIS
- High School Implementation
- Sustainability of Programs
- Use of FBA
- Suspensions and Future
- Drug and Alcohol Use Rate
- Drill Down Tool
- When to Use FBA
- Stronger Tier II and III
- Patterns of Minor ODRs
- Ethnicity Report
- Minor Misbehavior
- Discipline Referral Rates
- Cost of Implementation
- Measuring SWPBS
- Is BoQ Stable
- Revised BoQ
- Restraint-Seclusion Policies
- SWPBS and Socioeconomics
- ODR Across Grade Levels
- ODR Reductions and Ethnicity
- ODR and Population
- ODR and Enrollment Size
- Implementation Across US
- SWIS and Ethnicity
- Evaluation Tools
- State Implementation Survey
- Evaluation Examples
What are Patterns and Predictors of CICO Participation in U.S. Schools?
Kathleen Conley, Angus Kittelman, Michelle Massar, and Kent McIntosh
University of Oregon
Check In Check Out (CICO; Crone, Hawken, & Horner, 2010) is one of the most empirically supported Tier 2 behavior interventions for students who require additional supports to experience success (Campbell & Anderson, 2011; Maggin, Zurheide, Pickett, & Baillie, 2015). CICO incorporates regular performance feedback and adult interaction throughout the day in alignment with school-wide expectations. Performance feedback includes both reinforcing and corrective functions and is provided several times throughout the day so that the student has an opportunity to adjust behavior for the next check-in period. Interactions between the student and adult are designed to be efficient, structured, positive, and consistent. Increasing positive student-adult interactions has been linked to improved positive adult-child relationships (Anderson, Christenson, Sinclair, & Lehr, 2004) and may lead to improved student engagement, attendance, and work completion (Hamre & Pianta, 2001; Woolley & Grogan-Kaylor, 2006).
Despite the benefits (e.g., inexpensive, effective), implementing and monitoring CICO in a systematic manner can be challenging. Building efficient and sustainable Tier 2 systems that can adequately support up to 15% of the student population requires a significant investment at the front end of the adoption process (e.g., staff, time allocation). Given that CICO is the most commonly implemented Tier 2 intervention, enrollment in CICO may be one indicator of a school or other agency’s capacity to implement the intervention for up to 15% of students. Due to the lack of peer-reviewed research documenting the extent to which schools are providing CICO to proportions of students that match theoretical distributions (i.e., 5 to 15%), there is a need to develop an understanding of current patterns of CICO enrollment and its potential impact on intervention fidelity and effectiveness. A descriptive study investigating patterns of CICO use in schools was conducted by analyzing a multi-state database from CICO-SWIS, a subscription-based application that schools purchase to enter and track CICO data. The specific questions examined in this study were:
- What are patterns of use of CICO in schools?
- What variables predict the percent of students receiving CICO in schools?
The sample included 261 public schools across 25 states in the U.S. Schools were included in the sample if they were (a) using the Check-In Check-Out component of the School-Wide Information System (CICO-SWIS) application, (b) had enrolled at least one student in CICO, (c) reported enrollment data, (d) conducted the School-wide Benchmarks of Quality (BoQ; Kincaid, Childs, & George, 2005), and (e) agreed to share their data with the National Technical Assistance Center on PBIS for research purposes during the 2013-14 school year. Of the 95% of schools in the sample with complete data from the National Center for Education Statistics (NCES), the average enrollment was 506 students and the average percent of students receiving free or reduced-price meals was 48%. Schools in the sample included 173 elementary, 53 middle, 13 high, and 22 “other” schools, consisting of a different range of school grade levels served such as pre-kindergarten (pre-K) to Grade 12, pre-K to Grade 8, and pre-K to kindergarten. School urbanicity identified schools in rural (n = 65), town (n = 39), suburban (n = 91), and city (n = 53) locales.
School-level data used for this investigation were obtained from CICO-SWIS. For research question 2, 21 schools were excluded due to a lack of NCES information on free and reduced lunch rates and urbanicity.
The percent of students enrolled in CICO was measured by the number of students enrolled in CICO-SWIS divided by the total school enrollment. Due to the severe positive skew in the sample distribution, the variable was dichotomized based on a median split. The median number of students enrolled in CICO was 12 students, or 3% of total student enrollment. Schools with 3% or greater enrollment rates were coded as implementing to minimally acceptable proportions.
Total school enrollment was measured as the number of students enrolled in each school as reported to NCES for the 2013-14 school year. Due to total school enrollment not meeting assumptions of normality, we computed a standardized value (z-score). The percent of students receiving free and reduced lunch data were obtained from the NCES database for the 2013-14 school year. We included school grade type in the analysis as a dummy-coded binary variable, comparing the largest grade type, elementary (n = 173), to all other school types (n = 88). School urbanicity was also included in the analysis as a dummy-coded binary variable, comparing suburban to all other locales. We calculated rate of office discipline referrals (ODRs) per 100 students per day by dividing the total number of ODRs by the total number of students and school days, then multiplying by 100.
We included BoQ scores because it was the most common evaluation tools used in the sample. Tier 1 PBIS fidelity is correlated with increased fidelity of Tier 2 and Tier 3 systems (Kim, McIntosh, & Hoselton, 2014). For schools with multiple BoQs during the school year, we used the last score of the year.
What are Patterns of Use of CICO in Schools?
Descriptive statistics regarding CICO use, including enrollment and specifics regarding card use, were calculated. The median number of CICO expectations (i.e., the number of behaviors rated) was 3.00 (range = 3–5), and the median number of check-in (i.e., rating) opportunities was 8.00 (range = 2–22). The median default student goal line was 80% (range = 60–90%). The median number of students enrolled in CICO was 12 students (range = 1–121), and the median percent of the student body receiving CICO was 3% (range = 0.00–0.29).
What Variables Predict the Percent of Students Receiving CICO in Schools?
To examine the predictors associated with increased student participation in Tier 2 supports, we conducted a logistic regression analysis. The dependent variable was the percent of students enrolled in CICO. Because the dependent variable was coded dichotomously (i.e., at least 3% of students enrolled in CICO), a blocked logistic regression analysis was conducted using (a) total school enrollment, (b) percent of students receiving free or reduced lunch, (c) school grade type, (d) school locale, (e) rate of ODRs, and (f) BoQ scores. Block 1 included potential confounds or non-malleable school characteristics (i.e., enrollment, free and reduced lunch, type of school level), and Block 2 added variables of primary interest (school-level ODR rates and BoQ scores).
Of the variables entered in Block 1, total school enrollment was the only significant (negative) predictor of CICO enrollment (odds ratio [OR] = 0.45). This indicates that for schools with a total enrollment of 250 students, the probability of more than 3% of students enrolled in CICO is 49%. For schools with total enrollment of 500 students, the probability decreases to 33%, and for schools with 750 students the probability is only 20%. Figure 1 demonstrates that as total school enrollment increased, the mean percent of students enrolled in CICO decreased.
Figure 1. Percent of Students Enrolled in CICO Across Varying Ranges of Total School Enrollment
Rate of ODRs and PBIS Tier 1 implementation were added as additional predictors in Block 2. Rate of ODRs significantly and positively predicted CICO enrollment (OR = 1.41). This indicates that for schools with rates of 0.25 ODRs per
100 students per year, the probability of more than 3% of students enrolled in CICO is 28%. For schools with rates of 0.50 ODRs, the probability increases to 30%, and for schools with rates of 0.75 ODRs the probability is 32% (see Figure 2). PBIS Tier 1 implementation was not significantly associated with CICO enrollment criteria (p = .935).
Figure 2. Percent of Students Enrolled in CICO Across Varying Rates of ODRs
There were several limitations of this study worth noting. The sample size and selection criteria were limited to schools that entered point data into the CICO-SWIS application. In addition, the analysis of school-wide discipline (i.e., ODR) patterns is one potential indicator of need for targeted behavior interventions and was further limited by including only a broad measure of referrals per student per day across the school. Another limitation of the study is that CICO, although the most commonly used Tier 2 intervention, is still one intervention. Schools may implement additional targeted interventions with separate data systems to serve student needs. A small menu of interventions is recommended within the PBIS literature. Finally, the cut-off (3%) for schools implementing CICO systematically was selected based on the median student CICO enrollment (i.e., participation) within the sample, not a theoretically sound cut-off to measure acceptable implementation of CICO across students.
With these limitations in mind, results from the research sample may indicate that schools are not implementing CICO systematically across the school and that not enough students are able to access the intervention to maximize benefits. A median of 3% of students across the school year is significantly below the PBIS logic and CICO literature, which indicate that 10% to 15% of students may bene t from Tier 2 supports (Hawken, O’Neill, & MacLeod, 2011).
Practitioners may want to revisit current CICO implementation procedures related to (a) proactive student screening to identify students who may bene t from CICO and (b) maximizing available resources to serve these additional students. Three specific recommendations include:
Systematic Screening for Tier 2 Supports. School Tier 2 teams can use either existing data such as of ce discipline referrals (e.g., using a decision rule of two or more major ODRs), validated tools for universal screening of social behavior (e.g., the Student Risk Screening Scale [http:// www.ci3t.org/screening]), teacher nominations, or a combination of the above data. The team can examine data either monthly or at predetermined time points (e.g., October, January, May) to identify students who may need more support than the school’s Tier 1 systems currently provide.
Regularly Monitoring Fidelity of CICO Implementation. To ensure CICO is being implemented with fidelity, Tier 2 teams may want to consider using a formal evaluation measure. The Check-In Check-Out Fidelity of Implementation Measure (Crone et al., 2010) is an instrument designed specifically for the monitoring of CICO intervention fidelity. Although the items may not be sufficient to ensure systemic implementation, regularly (e.g., annually) measuring fidelity ensures that the core features continue to be implemented as intended and promotes quality improvement” of CICO implementation. A measure of CICO fidelity would not replace overall fidelity of PBIS Tier 2 supports, such as the Tiered Fidelity Inventory (TFI; Algozzine et al., 2014), which examines features of the larger framework.
Systematizing the Procedures and Processes Related to CICO. In terms of school-wide implementation, it is important to design materials and procedures that are standardized, contextually appropriate, and highly efficient so that staff can be easily trained to deliver the core components with fidelity. Collecting, analyzing, and sharing data across relevant stakeholders (e.g., staff, students, community partners) can be incorporated into regular CICO team routines. This should include mechanisms for measuring intervention fidelity, staff implementation fidelity, student and staff satisfaction, and student impact (school-wide and for individual students).
Algozzine, R. F., Barrett, S., Eber, L., George, H., Horner, R. H., Lewis, T. J., Sugai, G. (2014). PBIS Tiered Fidelity Inventory (TFI). Eugene, OR: OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. Available at http://www.pbis.org.
Anderson, A. R., Christenson, S. L., Sinclair, M. F., & Lehr, C. A. (2004). Check & Connect: e importance of relationships for promoting engagement with school. Journal of School Psychology, 42, 95-113. doi:http://dx.doi.org/10.1016/j. jsp.2004.01.002
Campbell, A. L., & Anderson, C. M. (2011). Check-in/check-out: A systematic evaluation and component analysis. Journal of Applied Behavior Analysis, 44, 315-326.
Crone, D. A., Hawken, L. S., & Horner, R. H. (2010). Responding to problem behavior in schools: The Behavior Education Program (2nd ed.). New York: Guilford.
Hamre, B. K., & Pianta, R. C. (2001). Early teacher-child relationships and the trajectory of children’s school outcomes through eighth grade. Child Development, 72, 625-638.
Hawken, L. S., O’Neill, R. E., & MacLeod, K. S. (2011). An Investigation of the Impact of Function of Problem Behavior on Effectiveness of the Behavior Education Program (BEP). Education & Treatment of Children, 34(4), 551-574.
Kim, J., McIntosh, K., & Hoselton, R. (2014). Do schools with adequate tier I SWPBIS implementation have stronger implementation at tiers II and III? Retrieved from OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports.
Kincaid, D., Childs, K., & George, H. (2005). School-wide Benchmarks of Quality (BoQ). Unpublished instrument, University of South Florida.
Maggin, D. M., Zurheide, J., Pickett, K. C., & Baillie, S. J. (2015). A systematic evidence review of the check-in/check- out program for reducing student challenging behaviors. Journal of Positive Behavior Interventions, 17, 197-208. doi:10.1177/1098300715573630
Woolley, M. E., & Grogan-Kaylor, A. (2006). Protective family factors in the context of neighborhood: Promoting positive school outcomes. Family Relations, 55, 93-104.
This project is supported by the U.S. Department of Education, O ce of Special Education Programs (OSEP). Opinions ex- pressed herein are those of the authors and do not necessarily represent the position of the U.S. Department of Education