•  

Intervention Summary

The CHARACTERplus Way

The CHARACTERplus Way is a comprehensive whole-school intervention that aims to positively change the school environment in order to foster students' ethical, social, and cognitive development. Focus is placed on the school environment because it can positively or negatively affect students' feelings of belonging within the school community, which in turn can affect students' academic performance and conduct. The CHARACTERplus Way seeks to influence this environment by developing staff members' and students' sense of belonging, instilling respect and trust in teachers and students, providing support for adult and student learning, and fostering a collaborative learning and working environment for students and school staff.

The CHARACTERplus Way is implemented throughout a school district (kindergarten through grade 12) by leadership teams. The intervention consists of a 3-year holistic process in which each school develops character education initiatives to meet its needs. In small districts, the process is led by a district leadership team, which includes the superintendent, and in large districts, the process is led by school leadership teams, which include the principal of each respective school. Each leadership team also includes two teachers, a counselor or social worker, and two parents or other community members. In year 1, the leadership team is trained in the 10 essentials of the intervention (i.e., community participation, character education policy, identified and defined traits, integrated curriculum, experiential learning, evaluation, adult role models, staff development, student leadership, and sustaining the process) and in creating coalitions for identifying character education initiatives. The team then facilitates school, parent, and community member involvement in developing the character education initiatives for inclusion in the school curriculum. In years 2 and 3, the leadership team helps each school implement the individualized character education program.

Descriptive Information

Areas of Interest Mental health promotion
Outcomes
1: School culture
2: School climate
3: Math and reading scores
4: Discipline referrals
Outcome Categories Education
Environmental change
Ages 6-12 (Childhood)
13-17 (Adolescent)
26-55 (Adult)
55+ (Older adult)
Genders Data were not reported/available.
Races/Ethnicities Data were not reported/available.
Settings School
Geographic Locations Urban
Suburban
Rural and/or frontier
Implementation History The CHARACTERplus Way has been implemented in more than 230 schools in Alabama, Illinois, Missouri, and South Dakota. About 100,000 students and 25,000 adults have participated in the intervention.
NIH Funding/CER Studies Partially/fully funded by National Institutes of Health: No
Evaluated in comparative effectiveness research studies: No
Adaptations No population- or culture-specific adaptations of the intervention were identified by the developer.
Adverse Effects No adverse effects, concerns, or unintended consequences were identified by the developer.
IOM Prevention Categories Universal

Quality of Research

Documents Reviewed

The documents below were reviewed for Quality of Research. The research point of contact can provide information regarding the studies reviewed and the availability of additional materials, including those from more recent studies that may have been conducted.

Study 1

Marshall, J. C., & Caldwell, S. D. Missouri Show Me CHARACTERplus implementation study: Fifth year final evaluation report. Unpublished manuscript.

Study 2

Marshall, J. C., & Caldwell, S. D. Caring School Community the CHARACTERplus Way: Fifth year final evaluation report. Unpublished manuscript.

Study 3

NREPP technical report part 1: Introduction, the CHARACTERplus Way: Improving school environment, increasing student academic achievement, reducing student problem behaviors. Unpublished manuscript.

Supplementary Materials

NREPP review: Quality research report--The CHARACTERplus Way

Outcomes

Outcome 1: School culture
Description of Measures School culture was assessed with the following measures:

  • The CHARACTERplus Implementation Survey, a study-specific 33-item questionnaire covering the 11 principles developed by the Character Education Partnership (e.g., "The school is a caring community," "To develop character, the school provides students opportunities for moral action"). The survey was administered by study personnel and completed annually by school staff who had received training as part of the CHARACTERplus Way.
  • The School Character Expectations, Parent and Staff Relations, Staff Culture of Belonging, and School Leadership content factors of the CHARACTERplus Staff Survey, a study-specific 40-item questionnaire. The survey was administered by study personnel and completed annually by all school staff, regardless of whether or not they received training as part of the CHARACTERplus Way.
  • The Caring Schools Community CHARACTERplus Implementation Survey, a study-specific 40-item questionnaire assessing 8 project constructs: Learning Community, Leadership Support, Resources, Data Utilization, School Climate, Staff Collaboration, Collaborative Classroom Practices, and Student Prosocial Behavior. The survey was administered by study personnel and completed annually by all school staff, regardless of whether or not they received training as part of the CHARACTERplus Way.
Key Findings In a 4-year study, elementary and secondary schools were randomly selected from the population of public schools in the State and then randomly assigned to the intervention condition, in which the CHARACTERplus Way was implemented for 1, 2, or 3 years (low, moderate, or high implementation, respectively) after leadership teams received training over the course of a year, or the wait-list control condition, in which schools used their standard curriculum. Findings included the following:

  • After the first year of implementation, intervention schools had an improvement in each of the 11 principles assessed by the CHARACTERplus Implementation Survey as compared with control schools (p < .01 for each principle). These changes were associated with large effect sizes (partial eta-squared = 0.26–0.54). Although the intervention schools also had improvements in the 11 principles during subsequent years of implementation, these changes were not significant. In addition, there were no significant differences between school levels (i.e., elementary vs. middle vs. high school) in regard to the improvements. Throughout the study, control schools had no improvement in the 11 principles.
  • Across the 3-year implementation period, intervention schools with high or moderate implementation had an improvement in the School Character Expectations content factor (p < .01) and the Parent and Staff Relations content factor (p < .01) of the CHARACTERplus Staff Survey as compared with control schools and schools with low implementation. These changes were associated with medium (partial eta-squared = 0.13 for the School Character Expectations content factor) and large (partial eta-squared = 0.24 for the Parent and Staff Relations content factor) effect sizes. There were no significant differences between school levels (i.e., elementary vs. middle vs. high school) in regard to the improvements.
In another 4-year study, elementary schools were randomly selected from districts in a large metropolitan area and then randomly assigned so that the same number of elementary schools within each district was assigned to each treatment condition: the intervention condition, in which the CHARACTERplus Way was implemented for 1, 2, or 3 years (low, moderate, or high implementation, respectively) after leadership teams received training over the course of a year, or the wait-list control condition, in which schools used their standard curriculum. Across the 3-year implementation period:

  • Compared with schools in the control condition, intervention schools had an improvement in four constructs of the Caring Schools Community CHARACTERplus Implementation Survey: Resources (p < .01), Staff Collaboration (p < .01), Collaborative Classroom Practices (p < .01), and Student Prosocial Behavior (p < .05). These changes were associated with medium to large effect sizes (partial eta-squared = 0.13–0.39).
  • Compared with schools with low implementation, schools with high or moderate levels of implementation had greater improvement in all four content factors of the CHARACTERplus Staff Survey: School Character Expectations (p < .01), Parent and Staff Relations (p < .01), Staff Culture of Belonging (p < .05), and School Leadership (p < .01). These changes were associated with large effect sizes (partial eta-squared = 0.17–0.44).
Studies Measuring Outcome Study 1, Study 2
Study Designs Experimental
Quality of Research Rating 2.3 (0.0-4.0 scale)
Outcome 2: School climate
Description of Measures School climate was assessed using two measures:

  • The Students' Feelings of Belonging content factor of the CHARACTERplus Staff Survey, a study-specific 40-item questionnaire. The survey was administered by study personnel and completed annually by all school staff, regardless of whether or not they received training as part of the CHARACTERplus Way.
  • The Sense of Classroom Autonomy and Influence factor of the Caring Schools Community CHARACTERplus Student Behavior Survey, a 77-item questionnaire administered annually by study personnel to 3rd- and 4th-grade students in participating schools.
Key Findings In a 4-year study, elementary and secondary schools were randomly selected from the population of public schools in the State and then randomly assigned to the intervention condition, in which the CHARACTERplus Way was implemented for 1, 2, or 3 years (low, moderate, or high implementation, respectively) after leadership teams received training over the course of a year, or the wait-list control condition, in which schools used their standard curriculum. Across the 3-year implementation period, intervention schools with high implementation had an improvement in the Students' Feelings of Belonging content factor of the CHARACTERplus Staff Survey, and control schools and schools with moderate or low implementation had a decline (p < .01). This change was associated with a large effect size (partial eta-squared = 0.18). There were no significant differences between school levels (i.e., elementary vs. middle vs. high school) in regard to the improvements.

In another 4-year study, elementary schools were randomly selected from districts in a large metropolitan area and then randomly assigned so that the same number of elementary schools within each district was assigned to each treatment condition: the intervention condition, in which the CHARACTERplus Way was implemented for 1, 2, or 3 years (low, moderate, or high implementation, respectively) after leadership teams received training over the course of a year, or the wait-list control condition, in which schools used their standard curriculum. Analyses were conducted with data for students in grades 3 or 4. Across the 3-year implementation period:

  • Intervention schools with high implementation had an improvement in the Students' Feelings of Belonging content factor of the CHARACTERplus Staff Survey, and control schools and schools with moderate or low implementation had a decline (p < .01). This change was associated with a large effect size (partial eta-squared = 0.18).
  • Intervention schools at all implementation levels had an improvement in the Sense of Classroom Autonomy and Influence factor of the Caring Schools Community CHARACTERplus Student Behavior Survey, and control schools had a decline (p < .01). This change was associated with a large effect size (partial eta-squared = 0.18).
Studies Measuring Outcome Study 1, Study 2
Study Designs Experimental
Quality of Research Rating 2.3 (0.0-4.0 scale)
Outcome 3: Math and reading scores
Description of Measures Math and reading scores were assessed from the results of State-mandated standardized achievement tests. Scores were categorized as proficient, advanced, basic, or below.
Key Findings In a 4-year study, elementary and secondary schools were randomly selected from the population of public schools in the State and then randomly assigned to the intervention condition, in which the CHARACTERplus Way was implemented for 1, 2, or 3 years (low, moderate, or high implementation, respectively) after leadership teams received training over the course of a year, or the wait-list control condition, in which schools used their standard curriculum. Analyses were conducted with data for students who were in grades 4, 8, and 11 when the study began. Across the 3-year implementation period, the percentage of schools with an increase in proficient or advanced reading scores was greater for intervention schools with high or moderate implementation than it was for control schools and schools with low implementation (p < .05). This change was associated with a medium effect size (partial eta-squared = 0.08). There were no significant differences between school levels (i.e., elementary vs. middle vs. high school) in regard to the improvements.

In another 4-year study, elementary schools were randomly selected from districts in a large metropolitan area and then randomly assigned so that the same number of elementary schools within each district was assigned to each treatment condition: the intervention condition, in which the CHARACTERplus Way was implemented for 1, 2, or 3 years (low, moderate, or high implementation, respectively) after leadership teams received training over the course of a year, or the wait-list control condition, in which schools used their standard curriculum. Analyses were conducted with data for students in grades 3 or 4. Across the 3-year implementation period, the percentage of schools with an increase in proficient or advanced reading and math scores was greater for intervention schools with high implementation than it was for control schools and schools with moderate or low implementation (p < .05 for reading scores; p < .01 for math scores). These changes were associated with large (partial eta-squared = 0.14 for reading scores) and medium (partial eta-squared = 0.12 for math scores) effect sizes.
Studies Measuring Outcome Study 1, Study 2
Study Designs Experimental
Quality of Research Rating 2.9 (0.0-4.0 scale)
Outcome 4: Discipline referrals
Description of Measures Discipline referrals were assessed using two measures:

  • A student discipline referral tracking form developed for the study. School personnel completed this form for each student who was referred to the school office for disciplinary reasons, and data were aggregated by study personnel.
  • Districtwide computerized school records. Study personnel used these records to collect data on student discipline referrals.
Key Findings In a 4-year study, elementary and secondary schools were randomly selected from the population of public schools in the State and then randomly assigned to the intervention condition, in which the CHARACTERplus Way was implemented for 1, 2, or 3 years (low, moderate, or high implementation, respectively) after leadership teams received training over the course of a year, or the wait-list control condition, in which schools used their standard curriculum. Analyses were conducted with data for students who were in grades 4, 8, and 11 when the study began. Across the 3-year implementation period, intervention schools with high or moderate implementation had a greater decrease in discipline referrals than control schools and schools with low implementation, which had a small increase in referrals (p < .01). There were no significant differences between school levels (i.e., elementary vs. middle vs. high school) in regard to the improvements.

In another 4-year study, elementary schools were randomly selected from districts in a large metropolitan area and then randomly assigned so that the same number of elementary schools within each district was assigned to each treatment condition: the intervention condition, in which the CHARACTERplus Way was implemented for 1, 2, or 3 years (low, moderate, or high implementation, respectively) after leadership teams received training over the course of a year, or the wait-list control condition, in which schools used their standard curriculum. Analyses were conducted with data for students in kindergarten through grade 5. Across the 3-year implementation period, intervention schools with high or moderate implementation had a greater decrease in discipline referrals than control schools and schools with low implementation, which had a small increase in referrals (p < .01).

In a 2-year study, elementary and secondary schools were randomly assigned to the intervention condition, in which schools participated in the CHARACTERplus Way, or the wait-list control condition, in which schools used their standard curriculum. Analyses were conducted with data for elementary, middle, and high school students. Across time, schools in the intervention condition had a greater reduction in discipline referrals than control schools (reductions of 52% versus 34% for middle schools and 36% vs. 8% for high schools) (p < .01).
Studies Measuring Outcome Study 1, Study 2, Study 3
Study Designs Experimental
Quality of Research Rating 2.1 (0.0-4.0 scale)

Study Populations

The following populations were identified in the studies reviewed for Quality of Research.

Study Age Gender Race/Ethnicity
Study 1 6-12 (Childhood)
13-17 (Adolescent)
26-55 (Adult)
55+ (Older adult)
Data not reported/available Data not reported/available
Study 2 6-12 (Childhood) Data not reported/available Data not reported/available
Study 3 13-17 (Adolescent) Data not reported/available Data not reported/available

Quality of Research Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the Quality of Research for an intervention's reported results using six criteria:

  1. Reliability of measures
  2. Validity of measures
  3. Intervention fidelity
  4. Missing data and attrition
  5. Potential confounding variables
  6. Appropriateness of analysis

For more information about these criteria and the meaning of the ratings, see Quality of Research.

Outcome Reliability
of Measures
Validity
of Measures
Fidelity Missing
Data/Attrition
Confounding
Variables
Data
Analysis
Overall
Rating
1: School culture 2.0 2.8 1.8 3.3 2.3 2.0 2.3
2: School climate 2.0 2.8 1.8 3.3 2.3 2.0 2.3
3: Math and reading scores 4.0 4.0 1.8 3.3 2.3 2.0 2.9
4: Discipline referrals 1.5 2.0 1.8 3.3 2.3 2.0 2.1

Study Strengths

In all studies, the use of factor analysis strengthened the validity of the measurement instruments. Attrition was low in all three studies. Measures such as visits from program coaches throughout the implementation period, training at multiple levels, and the development of action plans with measurable goals were put in place to support intervention fidelity. Careful attention was given to the development of a theoretical framework that guided the design, implementation, and evaluation of the program, which helped control confounding variables. Study sample sizes were large enough to permit the presentation of robust conclusions from data analysis.

Study Weaknesses

Some measures developed for the studies lacked external validation data. Information on the steps taken to maximize intervention fidelity was provided, but data to substantiate the effectiveness of these steps were lacking. Although the study designs incorporated random assignment, other procedures to assess for potential confounding variables do not appear to have been used. Higher level analytical techniques also do not appear to have been used.

Readiness for Dissemination

Materials Reviewed

The materials below were reviewed for Readiness for Dissemination. The implementation point of contact can provide information regarding implementation of the intervention and the availability of additional, updated, or new materials.

CHARACTERplus. The CHARACTERplus Way survey administration manual. St. Louis, MO: CHARACTERplus and Cooperating School Districts.

CHARACTERplus. Transforming climate and culture: The CHARACTERplus Way: A manual for K-12 leadership teams (1st ed.). St. Louis, MO: CHARACTERplus and Cooperating School Districts.

CHARACTERplus. Transforming climate and culture: The CHARACTERplus Way facilitator's guide (1st ed.). St. Louis, MO: CHARACTERplus and Cooperating School Districts.

CHARACTERplus & Cooperating School Districts. The CHARACTERplus Way replication handbook (3rd ed.). St. Louis, MO: Author.

CHARACTERplus & Cooperating School Districts. 17th annual Character Education Conference program guide. St. Louis, MO: Author.

CHARACTERplus & Cooperating School Districts. Taking stock: Transforming climate and culture: The CHARACTERplus Way assessment tool. St. Louis, MO: Author.

CHARACTERplus & Cooperating School Districts. Transforming climate and culture: The CHARACTERplus Way brochure. St. Louis, MO: Author.

DVDs:

  • Building community the CHARACTERplus Way
  • CHARACTERplus inspiring lives of integrity

Marshall, J. C., Caldwell, S. D., & Foster, J. F. Moral education the CHARACTERplus Way. Journal of Moral Education, 40(1), 51-72.

Other dissemination materials:

  • Ten Essentials observation form
  • The CHARACTERplus Way general site visit report
  • The CHARACTERplus Way meeting site visit report

Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the intervention's Readiness for Dissemination using three criteria:

  1. Availability of implementation materials
  2. Availability of training and support resources
  3. Availability of quality assurance procedures

For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.

Implementation
Materials
Training and Support
Resources
Quality Assurance
Procedures
Overall
Rating
3.9 4.0 3.7 3.8

Dissemination Strengths

The implementation materials are clearly written and well organized. The sample lessons for integrating character education within a school's curriculum are realistic and consistent with the goals of the program, and the leadership team manual and replication handbook provide easy-to-use assistance to new implementers. The historical and theoretical foundations of the program model, as well as supporting research, are also included for reference within the implementation materials. Training is delivered in a carefully paced sequence over a 30-month period through a variety of strategies (e.g., training institutes, coaching, networking opportunities between implementation sites). The facilitator's guide contains detailed instructions for every activity or event, including information on how to prepare for and set up the activity, agendas specifying the timeframe and group structure, worksheet templates, and the location of specific supporting materials (e.g., DVD chapter, manual page numbers) for every activity. The developer provides extensive support for collecting and using data. The training materials emphasize the use of data for planning, continuous program improvement, and quality assurance.

Dissemination Weaknesses

Little guidance is provided on methods to assure student confidentiality.

Costs

The cost information below was provided by the developer. Although this cost information may have been updated by the developer since the time of review, it may not reflect the current costs or availability of items (including newly developed or discontinued items). The implementation point of contact can provide current information and discuss implementation requirements.

Item Description Cost Required by Developer
District and school leadership team materials: K-12 leadership team manual, facilitator's guide, survey administration manual, replication handbook, monitoring packet, and handouts (1 hard copy and access to downloadable files) $500 per set Yes, for district leadership team and each school leadership team
3-day, off-site Initial Institute Training during the preimplementation period $7,200 per district leadership team Yes
2-day, on-site Advanced Training Institute Part 1 during implementation year 1 $2,400 per district leadership team, plus travel expenses Yes
2-day, off-site Advanced Training Institute Part 2 during implementation year 2 $2,400 per district leadership team Yes
1-day, off-site Sustainability and Certification Training Institute during implementation year 3 $1,200 per district leadership team Yes
2-day, off-site annual Character Education Conference $1,080 for a 3-person district leadership team Yes
2-day, on-site support during preimplementation period $2,400 per district, plus travel expenses Yes
2-day, on-site district and school visits $2,400 per district, plus travel expenses Yes, during implementation year 1 (optional during implementation years 2 and 3)
24 hours of phone and email support during the preimplementation period $1,800 per district leadership team Yes
72 hours of phone and email support during implementation year 1 $5,400 per district leadership team Yes
32 hours of phone and email support during implementation year 2 $2,400 per district leadership team Yes
16 hours of phone and email support during implementation year 3 $1,200 per district leadership team Yes
Additional phone or email support $75 per hour No
Annual survey administration and data collection (includes access to online survey, raw data, and reports) $500 initial setup fee and $300 maintenance fee per year per district, plus $780 per year per school Yes

Additional Information

The estimated cost for a district with 10 schools would be $66,140 for a 42-month implementation period. The preimplementation planning period ranges from 3 to 6 months, with school-level integration.