Drill Down Tool
- Evaluation Briefs
- Fidelity in High Schools
- Use of TFI
- Racial Disproportionality
- Patterns/Predictors of CICO
- Economic Costs
- School Climate & SWPBIS
- High School Implementation
- Sustainability of Programs
- Use of FBA
- Suspensions and Future
- Drug and Alcohol Use Rate
- Drill Down Tool
- When to Use FBA
- Stronger Tier II and III
- Patterns of Minor ODRs
- Ethnicity Report
- Minor Misbehavior
- Discipline Referral Rates
- Cost of Implementation
- Measuring SWPBS
- Is BoQ Stable
- Revised BoQ
- Restraint-Seclusion Policies
- SWPBS and Socioeconomics
- ODR Across Grade Levels
- ODR Reductions and Ethnicity
- ODR and Population
- ODR and Enrollment Size
- Implementation Across US
- SWIS and Ethnicity
- Evaluation Tools
- State Implementation Survey
- Evaluation Examples
Drill Down Tool Evaluation Brief
by Bert Eliason and Kelsey Morris
University of Oregon
Data-based decision making is enhanced by identifying a precise problem statement or question to explore. When looking at a problem, the nearer we get to the precise problem (e.g., where and when it is occurring, who is involved, and why the problem is continuing in context and location) the more quickly we can build an action plan to effectively and efficiently address the problem. The School-Wide Information System (SWIS; May et al., 2006) has core and additional reports that help the user access that type of information quickly. SWIS has included a tool that allows the user to efficiently mine through and readily explore the data to create an even more precise problem statement. The SWIS Drill Down tool facilitates exploration of a school’s office discipline referral (ODR) data by using various filters to examine the school’s ODR records. The Drill Down tool was first included in the fall of 2013 when SWIS version 5.0 was released. This paper provides an overview of the Drill Down tool, shares information about its usage, the number and variety of filters being used, and suggestions for more effective and efficient use.
Overview of the Drill Down Tool
The SWIS Dashboard (displayed upon login to SWIS) disaggregates all of a school’s discipline referrals and displays seven reports depicting average referrals per day per month as well as referrals by time, location, day of week, problem behavior, grade, and student. These core reports depict peaks and valleys relative to cumulative referral patterns. However, it is not possible to verify connections among the reports and identify the information necessary for problem solving with precision using these reports alone.
The Drill Down tool allows schools/facilities to be more analytical with referral data and identify information necessary for more precise problem statements. The initial step in using the Drill Down tool is to identify an item for further inquiry (e.g., What is going on in the cafeteria? Who is involved? When is it happening? Why does it continue to happen?). The specific item (e.g., referrals from the cafeteria) is then selected from a comprehensive list of report filters. Report filters are organized into 12 different categories identified during the referral entry process (referral type, student/staff demographics, date/time, location,problem behavior, perceived motivation, others involved, action taken, seclusion/restraint, notes, and custom fields). The Drill Down tool filters all the school’s referral data and displays only data relative to the report filters included for the unique dataset. Once a report is generated, the graph type can be changed allowing the dataset to be viewed through different lenses. Changing the graph type aids in identifying connections among data and allows for a deeper level of analysis. Using the example of referrals from the cafeteria, generating a Drill Down report with that specific filter displays only referral data where the cafeteria was identified as the referral location. The generated report allows a user to view all data from the cafeteria and then change the graph type to view the data through various lenses. For example, the graph type could be set to “Problem Behavior.” As a result, referral data from the cafeteria would be formatted by problem behaviors allowing for the identification of the top problem behaviors in the cafeteria. Or, the graph type could be changed to “Time of Day” to identify the times when referrals are coming from the cafeteria.
Each time the graph type is changed and more information is learned, new filter(s) can be selected providing a new, more refined unique dataset. Generating a report with additional filters enables drilling into the data to engage in a deeper level of analysis, which enhances problem solving with precision. There is a video (www.pbisapps.org/Resources/Pages/Data-Drill-Down-How-To.aspx) explaining this process in SWIS Resources on the PBIS Applications website (www.pbisapps.org/).
Drill Down Usage
As shown in Figure 1, during January through May of 2014, approximately 45% of the schools that opened SWIS also opened the Drill Down tool. On average, 7109 schools opened the SWIS application each month, and 3194 (44.9%) of those schools also opened the Drill Down tool at least once that month. Over this 5-month time period, we can see an approximate 5% increase in the number of schools that are opening the Drill Down tool.
Figure 1: SWIS Schools Opening Data Drill Down Tool
As shown in Figure 2, schools that opened the Drill Down tool created a Drill Down report an average of 9 times each month. On average, schools used the Drill Down tool multiple times per week. During each visit to the Drill Down tool, the users considered an average of 1.2 unique datasets (data defined by a unique set of filters), and from each of those datasets, formulated an average of 1.9 reports (different views of the same dataset). Over this time period, there was approximately a 12% increase in the average number of times schools generated Drill Down reports monthly.
Figure 2: Monthly Use of the SWIS Data Drill Down Tool
Figure 3 shows the number of filters used per Drill Down search in May of 2014. Drill Down filters establish the parameters of a search. Once opened, a filter can be used or removed. In May, 16.2% of the searches were conducted using only one filter. The majority of the searches used either two (51.6%) or three (21.7%) filters, and most of the rest used either four (7.0%) or five (2.8%).
Figure 3: Number of Filters Used per Drill Down Search During May of 2014
A filter can be used by itself or in combination with other filters. Table 1 shows the top 15 filters used at any time in a data search during this study. The top five filters (Date Range, Student, Grade, Referral Type, and Problem Behavior) remained consistent in numerical order through all five months.
Table 1: Average Times Top 25 Filters Were Used per Month (January 2014- May 2014)
The most frequently used filter after “Date Range” is “Student.” This and the fact that during the average visit to the Drill Down tool users on the average are using only 1.2 unique datasets, would suggest users are coming to the Drill Down tool to look at information about a single student. In all iterations of SWIS versions 5.0 - 5.4, SWIS has included on the menu bar (Figure 4) a tool called the Student Dashboard. The Student Dashboard includes nearly all the historical information about a single student’s ODR activity, their demographic data, and any activity that would be included about the student if the student was involved in the Check in Check out (CICO-SWIS) or the Individual Student Information System (ISIS-SWIS) and the school was using those programs. If school users are using the Drill Down tool to look at a single student’s information, they would be much better off using the Student Dashboard to access more complete information about the student in question.
Histograms for the filters “Grade,” “Problem Behavior,” and “Location “ are also available on the SWIS Dashboard, and a histogram for the filter “Educator” is available as an Additional Report to SWIS and building administrators only.
The makers of SWIS had originally intended information about an educator’s referral activity be limited to access by building and SWIS administrators in hopes of avoiding the inadvertent sharing of information about an individual teacher’s referral activity among peers. Although most users would not have been able to access information about other staff members and the number and kind of referrals they had written, the Drill Down tool allowed a bit of a workaround for the enterprising data miner. As of version 5.2, SWIS has addressed this workaround and limits access to the “Educator” filter to those who may actually need that information (i.e., building administrators, SWIS administrators).
Figure 4: Location of Student Dashboard on SWIS Menu Bar
Table 2 identifies the top five filter sets used during May of 2014 when only one filter was used, as well as the top five filter sets when combinations of two, three, four, or five filters were used.
Table 2: Top Five Filter Sets Used Alone or in Combination
By May, the filter “Date Range” was a default filter set to the 2013-2014 school year. “Date Range” accounts for 88.6% of the times a single filter was used. A user generating a report using only “Date Range” would have opened the Drill Down tool and generated a report for all the referrals for the 2013- 2014 school year. That same information would have been available in graphic form on the SWIS Dashboard when the user first opened SWIS. For the other filters to have been used alone, the user would have had to replace the default, 2013-2014 “Date Range” with a different filter. The resulting reports would have represented all of the ODRs described by that filter the school entered into SWIS over the entire span of the school’s SWIS usage not just the information for that school year.
SWIS does allow a user to insert a specific “Date Range” with a beginning and end date (e.g., May 1, 2014 – May 31, 2014). This is a common method for obtaining the referrals that happened during one month or one week. It is also possible that the user could remove the default, school-year “Date Range” and replace it with a different year or a specific range of dates. From the information available, we are unable to determine whether an indicated “Date Range” filter refers to the default school year or a user-defined set of dates. In Table 2, we find a number of searches that include the filter “Date Range” used twice. Most likely in these cases, the user allowed the default school-year filter and also specified a defined period within that school year.
Going to the Drill Down tool with a specific question in mind is helpful (e.g., What precise information do we want to know about a current pattern of behavior to address it most effectively?). To build a precise problem statement, it is best to know what problems are occurring; who is involved; and when, where, and why the problem continues in context. From the SWIS Dashboard, we might notice that there is a lot of activity by the fifth grade. Using the Drill Down tool, we could add “Fifth Grade” to the default current school year and generate a report. Viewing the report by “Time of Day,” we might notice that a lot of activity is occurring between 12:00 and 12:30. If we add “Time Range” as a filter to the data set, include those times, generate another report, and view by location, we might notice that most of the activity is occurring in the cafeteria. If we then add “Cafeteria” to the data set, generate a report and look at “Problem Behavior,” we might see that “Minor Inappropriate Language” was the most frequent misbehavior. If we add that behavior to the filter set, generate a report, and view by “Perceived Motivation,” we might see that the most frequent motivation was to obtain peer attention. Through this process, we have enough information to determine that there are a large number of fifth graders using inappropriate language in the cafeteria from 12:00-12:30 to gain peer attention.
Looking at that question for the current school year, the question minimally used five filters, but the statement describes for us precisely who, what, when, where, and why a behavior is occurring and offers the PBIS team the specific knowledge needed to address and, hopefully, remediate the situation. If we had added multiple locations, grades, or problem behaviors, or an additional specific time range (e.g., the month of May), we would likely have had even more filters.
The SWIS Drill Down tool is a powerful tool for analyzing discipline data that is growing in popularity and use. The SWIS Core and Additional Reports can give a summary view of what is going on in a school, when and where it is happening, who is involved, and why it continues in context, but the SWIS Drill Down tool allows users to quickly explore a unique set of data and report on that data through a variety of lenses enabling the user to develop a precise problem statement.
Some initial users were using the Drill Down tool to access information that was already available in the Core Reports, the Additional Reports, and the Student Dashboard. To maximize effectiveness and efficiency, we suggest users access those reports to form a summary view, but open the Drill Down tool with a specific question in mind and develop a precise problem statement for the team to address.
May, S.L., Ard, W. I., Todd, A.W., Horner, R.H., Glasgow, A., Sugai, G., & Sprague, J.R. (2006). School-Wide Information System. Educational and Community Supports, University of Oregon, Eugene, OR.