•  
Intervention Summary

Safe & Civil Schools Positive Behavioral Interventions and Supports

Safe & Civil Schools Positive Behavioral Interventions and Supports

Safe & Civil Schools Positive Behavioral Interventions and Supports Model

The Safe & Civil Schools Positive Behavioral Interventions and Supports (PBIS) Model is a multicomponent, multitiered, comprehensive approach to schoolwide improvement. Integrating applied behavior analysis, research on effective schools, and systems change management theory, the intervention is an application of positive behavior support (PBS), a set of strategies or procedures designed to improve behavior by employing positive and systematic techniques. The intervention focuses on guiding members of an entire school staff in developing a schoolwide environment that is safe, civil, and conducive to learning. One of the core features of the Safe & Civil Schools PBIS Model is its emphasis on staff communication, collaboration, and cohesion. The intervention provides tools and strategies to help educators in elementary, middle, and high schools establish proactive, positive (nonpunitive), and instructional schoolwide discipline policies, manage student misbehavior and foster student motivation, and create a positive and productive school climate. It also aims to boost teacher satisfaction, contributing to increased teacher retention, and to engage students in the educational process, increasing their connectedness to the school community.

Implementation involves delivery of professional development services (e.g., in-service training, workshops, conferences), ongoing on-site coaching and support, and materials (e.g., books, DVDs, CDs) by the program developer to all members of a school's staff, typically over a 1- to 3-year period. The various components address student behavior in the school, classroom, and individual student levels. The core component, Foundations, guides staff through the process of designing a positive and proactive schoolwide discipline plan affecting all students in all the school's settings. Other components are supplemental to Foundations and are used in various degrees based on a school's need. Components developed for the classroom guide teachers in improving their current classroom management plan, while other components are designed to help educators plan and implement tailored strategies to increase motivation and improve the behavior of individual students.

Descriptive Information

Areas of Interest Mental health promotion
Outcomes
1: Academic achievement
2: School suspensions
3: Classroom disruption
4: Teacher professional self-efficacy
5: School discipline procedures
Outcome Categories Education
Social functioning
Physical aggression and violence-related behavior
Ages 6-12 (Childhood)
13-17 (Adolescent)
Genders Male
Female
Races/Ethnicities American Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other Pacific Islander
White
Settings School
Geographic Locations Urban
Suburban
Rural and/or frontier
Implementation History An estimated 5,000 schools in 44 States have received training on the Safe & Civil Schools Positive Behavioral Interventions and Supports Model. The intervention has also been implemented in Canada.
NIH Funding/CER Studies Partially/fully funded by National Institutes of Health: No
Evaluated in comparative effectiveness research studies: No
Adaptations Some program materials have been translated into Cambodian, Russian, Somali, Spanish, and Vietnamese.
Adverse Effects No adverse effects, concerns, or unintended consequences were identified by the developer.
IOM Prevention Categories Universal

Quality of Research

Documents Reviewed

The documents below were reviewed for Quality of Research. The research point of contact can provide information regarding the studies reviewed and the availability of additional materials, including those from more recent studies that may have been conducted.

Study 1

Madigan, K., & Cross, R. Determining the impact of Safe & Civil Schools' schoolwide positive behavior supports program on academic achievement: A nine-year study. Bethesda, MD: Accountability Works.

Study 2

Ward, B., & Gersten, R. A randomized evaluation of the effectiveness of Randy Sprick's Safe & Civil Schools' Foundations model for positive behavior support at elementary schools in a large urban school district: Interim results. Portland, OR: ECONorthwest.

Study 3

Barnoski, R. Foundations for learning: Safe & Civil Schools analysis documentation. Olympia, WA: Washington State Institute for Public Policy.

Cross, R. Reanalyzing significance levels: Addendum to Robert Barnoski's evaluation of Safe & Civil Schools model. Bethesda, MD: Accountability Works.

Supplementary Materials

Kentucky Department of Education. Reliability. In Kentucky Core Content Tests, technical report (pp. 1-11). Frankfort: Kentucky Department of Education.

Kentucky Department of Education. Valid use and interpretation of Kentucky Core Content Test Scores. In Kentucky Core Content Tests, technical report (pp. 1-29). Frankfort: Kentucky Department of Education.

McCausland, S. G., Hales, L. W., & Reinhardtsen, J. M. Positive discipline, safe school surveys: CREST technical report. Vancouver, WA: Educational Service District 112.

Safran, S. P. Using the Effective Behavior Supports Survey to guide development of schoolwide positive behavior support. Journal of Positive Behavior Interventions, 8(3), 3-9.

Outcomes

Outcome 1: Academic achievement
Description of Measures Academic achievement was assessed using two measures:

  • An Academic Index based on student performance on custom, criterion-referenced Kentucky Core Content Tests in reading, mathematics, science, social studies, arts and humanities, practical living/vocational studies, and writing
  • The California Standards Test in Mathematics; specifically, the number of students that scored at or above the proficient level, as collected from student administrative records
Key Findings One study conducted in Kentucky compared elementary, middle, and high schools receiving the intervention with control schools matched on characteristics known to influence academic achievement. For each school, Academic Index data were collected annually for 5 baseline years and for the 4 years of exposure to the intervention, which included 1 maintenance year. Over time, from the first baseline year through the maintenance year, students in intervention schools had an increase in academic achievement compared with students in control schools (p < .001).

In another study, 33 California elementary schools were randomly assigned to an intervention group or a wait-list control group. Records on the California Standards Test in Mathematics were collected at pre- and posttest. Compared with control schools, the five intervention schools implementing the program with high fidelity (as determined by two district-level administrators using quality assurance indicators) had a greater increase in the proportion of students rated proficient in math (p < .05).
Studies Measuring Outcome Study 1, Study 2
Study Designs Experimental, Quasi-experimental
Quality of Research Rating 3.6 (0.0-4.0 scale)
Outcome 2: School suspensions
Description of Measures School suspensions were measured using school administrative records. For each student, the schools tallied and recorded the number of suspensions delivered and the number of days suspended per 100 days enrolled.
Key Findings In one study, 33 California elementary schools were randomly assigned to an intervention group or a wait-list control group. Records on suspensions were collected at pre- and posttest. Compared with control schools, the five intervention schools implementing the program with high fidelity (as determined by two district-level administrators using quality assurance indicators) had a decrease in the proportion of students suspended at least once (p < .01) and in the average number of days suspended (p < .05).
Studies Measuring Outcome Study 2
Study Designs Experimental
Quality of Research Rating 3.5 (0.0-4.0 scale)
Outcome 3: Classroom disruption
Description of Measures Classroom disruption was assessed using two measures:

  • The Positive Behavior Support (PBS) Assessment--Staff Survey, which asked questions addressing the extent to which behaviors such as bullying, widespread disorder in the classrooms, and disrespectfulness and defiance occurred at school. Most of the survey questions asked respondents to assess the frequency of specific behaviors and the frequency of or extent to which specific school approaches or policies are implemented.
  • A teacher survey that addressed disruptive behaviors in school and related discipline practices. The survey included five sections: classroom disruptions, school areas avoided, impact of disruptions on learning, discipline practices, and supportive school environment. Yes/no response options were used to measure the presence or absence of each item. The 8 items measuring classroom disruption asked about the past 5 days to minimize recall problems.
Key Findings In one study, 33 California elementary schools were randomly assigned to an intervention group or a wait-list control group. The PBS Assessment--Staff Survey was administered to school staff at pre- and posttest. Compared with control schools, intervention schools had an increase in the proportion of teachers reporting that students followed classroom rules on a consistent basis (p < .01) and a decrease in the proportion of teachers reporting frequent problems with widespread disorder in the classroom (p < .05).

In another study, elementary, middle, and high schools in Washington State were assigned either to an intervention group or to a control group that did not receive the intervention. At posttest, a smaller proportion of teachers in intervention than control schools reported the following disruptions on the teacher survey:

  • Elementary school students exhibiting verbal intimidation (p < .01) and aggressive verbal intimidation (p < .01)
  • Elementary school students taking or damaging personal property (p < .01)
  • Elementary (p < .01) and middle school students (p < .05) pushing, grabbing, hitting, or kicking someone
  • Elementary (p < .05) and middle school students (p < .05) involved in sexual harassment
Studies Measuring Outcome Study 2, Study 3
Study Designs Experimental, Quasi-experimental
Quality of Research Rating 3.0 (0.0-4.0 scale)
Outcome 4: Teacher professional self-efficacy
Description of Measures Teacher professional self-efficacy was measured using a teacher survey that addressed disruptive behaviors in school and related discipline practices. The survey included five sections: classroom disruptions, school areas avoided, impact of disruptions on learning, discipline practices, and supportive school environment. Yes/no response options were used to measure the presence or absence of each item. The 8 items measuring teacher professional self-efficacy asked about the past 5 days to minimize recall problems.
Key Findings In one study, elementary, middle, and high schools in Washington State were assigned either to an intervention group or to a control group that did not receive the intervention. At posttest, a smaller proportion of teachers in intervention than control schools reported the following:

  • Feelings of not making an impact on elementary school student learning (p < .05)
  • Difficulties in achieving instructional objectives with their middle school students (p < .01)
  • Decreased desire to continue teaching in elementary (p < .01) and middle schools (p < .01)
  • Adverse effects on personal health in elementary (p < .01) and middle schools (p < .05)
Studies Measuring Outcome Study 3
Study Designs Quasi-experimental
Quality of Research Rating 2.0 (0.0-4.0 scale)
Outcome 5: School discipline procedures
Description of Measures School discipline procedures were measured using separate surveys for teachers, staff, and administrators that addressed disruptive behaviors in school and related discipline practices. The survey for teachers and staff included five sections: classroom (teachers) or outside classroom (staff) disruptions, school areas avoided, impact of disruptions on learning, discipline practices, and supportive school environment. Yes/no response options were used to measure the presence or absence of each item. The survey for administrators included some similar sections as well as questions about administrative intervention (e.g., phone call to parents or police, suspension). Items in the surveys addressing school discipline procedures and training (12 items for teachers, 11 for staff, and 10 for administrators) asked about the presence of, for example, written guidelines on school discipline practices; a schoolwide teacher/staff training program on discipline practices; and collaboration among school teachers, staff, and administrators. Items in the surveys addressing guidelines and rules for student behavior (8 items for teachers, 8 for staff, and 7 for administrators) asked about the presence of, for example, reports by students of consequences being unfair and the annual review of rules with students.
Key Findings In one study, elementary, middle, and high schools in Washington State were assigned either to an intervention group or to a control group that did not receive the intervention. At posttest, a greater proportion of teachers in intervention than control schools reported the presence of the following items related to school discipline procedures and training:

  • Written guidelines on school discipline practices in middle schools (p < .05)
  • Schoolwide teacher training program on discipline practices in elementary (p < .01) and high schools (p < .05)
  • Potential for improvement in the effectiveness of discipline practices in elementary (p < .01) and middle schools (p < .01)
  • Teacher collaboration with administrators to solve discipline problems in high schools (p < .01)
However, the following items were reported by a greater proportion of teachers in control than intervention schools:

  • Schoolwide teacher training program on discipline practices in middle schools (p < .05)
  • Teacher collaboration with common area supervisors to solve discipline problems in middle schools (p < .01)
  • Teacher collaboration with administrators to solve discipline problems in elementary schools (p < .01)
  • School-based resources to help students experiencing difficulty in elementary (p < .01) and middle schools (p < .05)
At posttest, a greater proportion of staff in intervention than control schools reported the presence of the following items related to school discipline procedures and training:

  • Schoolwide staff training program on discipline practices in elementary (p < .01) and high schools (p < .01)
  • Potential for improvement in the effectiveness of discipline practices in elementary (p < .05) and middle schools (p < .05)
However, the presence of a schoolwide staff training program on discipline practices was reported by a greater proportion of staff in control than intervention middle schools (p < .01).

At posttest, a greater proportion of administrators in intervention than control schools reported the presence of the following items related to school discipline procedures and training:

  • Schoolwide teacher training program on discipline practices in place now (p < .05) or in the next 2 years (p < .01) in elementary schools
  • Potential for improvement in the effectiveness of discipline practices in elementary schools (p < .05)
  • Administrator training on effective discipline practices in elementary schools (p < .05)
  • Teacher in-service training on discipline practices in the past 3 years in elementary schools (p < .01) and high schools (p < .01)
  • Administrator collaboration with common area supervisors on student discipline problems in high schools (p < .05)
Other findings at posttest reflect guidelines and rules for student behavior:

  • Within middle schools, a smaller proportion of teachers in intervention than control schools reported that students indicate the consequences for following or breaking the rules are not fair (p < .05).
  • In high schools, a greater proportion of staff in intervention than control schools reported that rules are posted in assigned areas (p < .01).
  • In high schools, a greater proportion of administrators in intervention than control schools reported that rules are reviewed more than once during the year (p < .01) and that regularly scheduled instruction takes place for students to learn proper school behaviors (p < .05).
Studies Measuring Outcome Study 3
Study Designs Quasi-experimental
Quality of Research Rating 2.5 (0.0-4.0 scale)

Study Populations

The following populations were identified in the studies reviewed for Quality of Research.

Study Age Gender Race/Ethnicity
Study 1 6-12 (Childhood)
13-17 (Adolescent) 		51% Female
49% Male 		73.2% White
19.6% Black or African American
3.7% Hispanic or Latino
3.4% American Indian or Alaska Native
Study 2 6-12 (Childhood) 51% Female
49% Male 		68.6% Hispanic or Latino
10.9% Asian
9.9% White
9% Black or African American
1.1% American Indian or Alaska Native
0.5% Native Hawaiian or other Pacific Islander
Study 3 6-12 (Childhood)
13-17 (Adolescent) 		50% Female
50% Male 		Data not reported/available

Quality of Research Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the Quality of Research for an intervention's reported results using six criteria:

  1. Reliability of measures
  2. Validity of measures
  3. Intervention fidelity
  4. Missing data and attrition
  5. Potential confounding variables
  6. Appropriateness of analysis

For more information about these criteria and the meaning of the ratings, see Quality of Research.

Outcome Reliability
of Measures 		Validity
of Measures 		Fidelity Missing
Data/Attrition 		Confounding
Variables 		Data
Analysis 		Overall
Rating
1: Academic achievement 3.5 3.5 3.8 3.8 3.3 4.0 3.6
2: School suspensions 3.5 3.5 3.8 3.5 3.0 4.0 3.5
3: Classroom disruption 2.8 2.8 2.8 3.3 2.5 4.0 3.0
4: Teacher professional self-efficacy 1.0 1.0 1.0 3.0 2.0 4.0 2.0
5: School discipline procedures 2.5 2.5 1.0 3.0 2.0 4.0 2.5

Study Strengths

The Kentucky Core Content Tests, which were used to assess academic achievement, have extensive data supporting their reliability and content and construct validity; the instruments' developers gave considerable attention to establishing reliability across years, grade levels, and individual subjects. School administrative records to document academic achievement and school suspensions are reliable and valid measures. For the Positive Behavior Support Assessment--Staff Survey, evidence of psychometrics was shown based on study sample data. Intervention fidelity was assessed in one study using a modified version of the Self-Assessment Survey (Version 2.0), which has strong psychometric properties. Another study assessed fidelity using the School-wide Benchmarks of Quality (BoQ), which has reliability and validity supported by independent investigators. Across all studies, missing data and attrition were either not an issue or were addressed appropriately with statistical techniques. One study included randomization, and in the other two studies, the investigators controlled for some confounds by matching control and intervention schools on characteristics known to influence academic achievement (e.g., percentage of students receiving free and reduced-price lunch, school eligibility for Title 1 Federal funding).

Study Weaknesses

No reliability and validity data from an independent investigation were presented for the survey instruments used to assess outcomes in the studies. Items in the teacher survey intended to measure self-efficacy appear to measure the impact of disruptive behavior on difficulty completing job tasks or teacher well-being and not self-efficacy as traditionally defined. Two studies did not use random assignment to treatment condition, potentially limiting internal validity. Specific confounds included presentation of intervention contents to control group school teachers in one study and self-selection bias in another.
Readiness for Dissemination

Materials Reviewed

The materials below were reviewed for Readiness for Dissemination. The implementation point of contact can provide information regarding implementation of the intervention and the availability of additional, updated, or new materials.

Other dissemination materials:

  • Implementation Checklists
  • Climate Audit

Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the intervention's Readiness for Dissemination using three criteria:

  1. Availability of implementation materials
  2. Availability of training and support resources
  3. Availability of quality assurance procedures

For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.

Implementation
Materials 		Training and Support
Resources 		Quality Assurance
Procedures 		Overall
Rating
4.0 4.0 4.0 4.0

Dissemination Strengths

Program materials are available in a variety of formats (print, audio, and video) and are comprehensive, practical, and of high quality. In addition to the core intervention materials, a variety of supplemental materials are available, including sample policies and lesson plans, reproducible study guides and tools, videos depicting use of proper techniques, survey instruments, and helpful desk references. The availability of multiple resources provides potential implementers with maximum flexibility to meet their specific school culture and behavior management needs. Materials describe potential obstacles to implementation and provide guidance on how to overcome them. From a 1-day workshop to multiyear training and support options, the flexibility of training capacity, length, and format offers tremendous opportunities for schools and districts to design a customized staff development plan. The developer is available to build and facilitate a workshop on any topic relating to behavior management. The intervention's quality assurance components form a comprehensive mechanism by which implementers can directly achieve and sustain high-quality implementation and fidelity to the intervention. Audit materials and recommended outcome measures support implementers' use of data-driven decisionmaking.

Dissemination Weaknesses

No weaknesses were identified by reviewers.
Costs

The cost information below was provided by the developer. Although this cost information may have been updated by the developer since the time of review, it may not reflect the current costs or availability of items (including newly developed or discontinued items). The implementation point of contact can provide current information and discuss implementation requirements.

Item Description Cost Required by Developer
Foundations: Establishing Positive Discipline Policies (3-volume set and 17 CDs) $995 Yes
Behavioral Response to Intervention: Creating a Continuum of Problem Solving & Support (book and CD) $50 No
Administrator's Desk Reference of Behavior Management (3-volume set) $95 No
START on Time: Safe Transitions and Reduced Tardies (binder, 8 CDs, and reference manual) $295 Yes
On the Playground: A Guide to Playground Management (binder, 7 CDs, and user's guide) $249 No
CHAMPS: A Proactive & Positive Approach to Classroom Management, 2nd ed. (book and CD) $49.50 Yes
CHAMPS DVD Inservice Series, 2nd ed. (2 books, 6 DVDs, facilitator's guide, and CD) $995 No
CHAMPS Teacher Planner $20 No
Discipline in the Secondary Classroom: A Positive Approach to Classroom Management, 2nd ed. (book and DVD) $32.95 Yes
Discipline in the Secondary Classroom DVD Inservice Series (2 books, 6 DVDs, and facilitator's guide) $995 No
Teacher Planner for the Secondary Classroom $19.95 No
Coaching Classroom Management: Strategies & Tools for Administrators & Coaches, 2nd ed. (book and CD) $60 Yes
Teacher's Encyclopedia of Behavior Management: 100 Problems/500 Plans $45 No
Interventions: Evidence-Based Behavioral Strategies for Individual Students, 2nd ed. (book and CD) $60 Yes
Interventions Audio: Evidence-Based Behavioral Strategies for Individual Students, 2nd ed. (8 CDs) $65 No
2-day, initial on-site training $5,000 for 1 school leadership team, or $20,000 for 15-20 school leadership teams, plus travel expenses No
2- to 12-day, on-site training in classroom management or individual intervention strategies $2,500 per day, plus travel expenses No
On-site program review, evaluation, and coaching for all staff, including district and building administrators $2,500 per day, plus travel expenses No
Phone and email technical assistance Free No
Foundations surveys and self-assessment checklists Free Yes

Additional Information

The developer offers comprehensive, multiyear training sequences that involve trainings conducted over 1 to 3 years.