•  
Intervention Summary

Partners for Change Outcome Management System (PCOMS): The Heart and

Partners for Change Outcome Management System (PCOMS): The Heart and

Partners for Change Outcome Management System (PCOMS): The Heart and Soul of Change Project

The Partners for Change Outcome Management System (PCOMS) is a client feedback program for improving the treatment outcomes of adults and children participating in a behavioral health care intervention. PCOMS is designed to improve the retention of participants in treatment and to assist them in reaching reliable and clinically significant change. The program can be implemented by behavioral health care therapists as part of any behavioral health care intervention.

PCOMS, which is integrated into each treatment session, consists of two brief scales that measure robust predictors of therapeutic success:

  • The Outcome Rating Scale (ORS), which assesses the client's therapeutic progress (through ratings of psychological functioning and distress) and the client's perceived benefit of treatment
  • The Session Rating Scale (SRS), which assesses the client's perception of the client-therapist alliance (i.e., the quality of the relational bond with the therapist and whether the therapist shares his or her therapeutic objective)

The therapist administers the ORS at the beginning of the treatment session, and the SRS is administered toward the end of the session. Client ratings for both measures are discussed on a session-by-session basis to maintain the client's engagement in treatment, optimize the client-therapist alliance, and provide a means for transitioning into the treatment session by focusing on client-identified concerns. If client ratings are very low, the therapist may choose to modify the type and amount of treatment.

PCOMS is disseminated through the Heart and Soul of Change Project and the International Center for Clinical Excellence. (The Readiness for Dissemination of each version was reviewed separately by NREPP.) The Heart and Soul of Change Project integrates PCOMS into clinical practice through client-directed, outcome-informed (CDOI) interaction, which enhances the common factors across theories that account for therapeutic change, includes the client in determining the benefit of services and making decisions that affect care, informs the treatment with measures of the client's experience, and involves an expectation of recovery and an appreciation of social justice in the provision of care.

Descriptive Information

Areas of Interest Mental health treatment
Outcomes
1: Therapeutic progress
2: Marital status
Outcome Categories Family/relationships
Mental health
Treatment/recovery
Ages 18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
Genders Male
Female
Races/Ethnicities Asian
Black or African American
Hispanic or Latino
White
Race/ethnicity unspecified
Non-U.S. population
Settings Outpatient
Workplace
Geographic Locations Urban
Suburban
Rural and/or frontier
Implementation History The Heart and Soul of Change Project was founded to disseminate PCOMS. PCOMS has been used by hundreds of organizations and by thousands of behavioral health care professionals in all 50 States and the District of Columbia and, internationally, in 20 countries on five continents. It is estimated that PCOMS is implemented with approximately 100,000 clients each year.
NIH Funding/CER Studies Partially/fully funded by National Institutes of Health: No
Evaluated in comparative effectiveness research studies: No
Adaptations The PCOMS scales (ORS and SRS) have been translated into 23 languages: Afrikaans, Bulgarian, Chinese, Croatian, Danish, Dutch, Finnish, French, German, Greek, Hebrew, Italian, Japanese, Maori, Norwegian, Polish, Romanian, Russian, Slovak, Spanish, Swedish, Welsh, and Yiddish.
Adverse Effects No adverse effects, concerns, or unintended consequences were identified by the developer.
IOM Prevention Categories IOM prevention categories are not applicable.

Quality of Research

Documents Reviewed

The documents below were reviewed for Quality of Research. The research point of contact can provide information regarding the studies reviewed and the availability of additional materials, including those from more recent studies that may have been conducted.

Outcomes

Outcome 1: Therapeutic progress
Description of Measures Therapeutic progress was assessed using the Outcome Rating Scale. The ORS is a 4-item self-report scale that measures a participant's psychological functioning and distress by asking how the participant is doing individually (personal well-being), interpersonally (family, close relationships), socially (work, school, friendships), and overall (general sense of well-being). Depending on the method of ORS administration, participants rate each item either by marking a line to reflect the strength of their opinions or by stating the appropriate ratings. Scores for each item range from 1 to 10, with a total score ranging from 0 to 40. Lower scores indicate more severe distress, and a comparison of the change in scores over time indicates the trajectory of therapeutic progress.
Key Findings In one study, participating couples were randomly assigned to the intervention group, which received couple therapy with PCOMS, or the comparison group, which received couple therapy only. Couples in the intervention group had higher ORS scores than those in the comparison group at posttreatment (p < .001) and 6-month follow-up (p < .01), even after adjusting for pretreatment ORS score and therapist.

In another study, participants were randomly assigned to the intervention group, which received individual therapy with PCOMS, or the comparison group, which received individual therapy only. Half of the participants received services from faculty members or practicum students at a university counseling center, and half received services from practicum students at a graduate training clinic. Among participants who received services at a university counseling center, from pre- to posttest, those in the intervention group had a greater increase in ORS scores than those in the comparison group (p < .05). Among participants who received services at a graduate training clinic, from pre- to posttest, those in the intervention group had a greater increase in ORS scores than those in the comparison group (p < .01), even after adjusting for pretest ORS scores.

In a third study, participants in an employee assistance program received at least two phone-based counseling sessions that included PCOMS. The study had multiple phases, including a 6-month baseline period, when PCOMS was administered during the counseling sessions; a subsequent 6-month period, when a computer program was introduced to aid therapists in administering PCOMS and interpreting a participant's ORS and SRS scores during counseling sessions; and a 12-month period, after use of the computer program had been fully integrated into the counseling sessions. The average change in participants' ORS scores between counseling sessions was compared over these three phases. The mean increase in participants' ORS scores was larger for the subsequent 6-month period (p < .001) and 12-month period (p < .001) than it was for the baseline period.

In a fourth study, participating couples were randomly assigned to the intervention group, which received couple therapy with PCOMS, or the comparison group, which received couple therapy only. From pre- to posttest, couples in the intervention group had a greater increase in ORS scores than those in the comparison group (p < .05).
Studies Measuring Outcome Study 1, Study 2, Study 3, Study 4
Study Designs Experimental, Preexperimental
Quality of Research Rating 3.1 (0.0-4.0 scale)
Outcome 2: Marital status
Description of Measures Marital status was assessed by client self-report. Couples were categorized as intact (i.e., not divorced or separated) or not intact.
Key Findings Couples participating in the study were randomly assigned to the intervention group, which received couple therapy with PCOMS, or the comparison group, which received couple therapy only. At the 6-month follow-up, a greater proportion of couples in the intervention group were intact relative to couples in the comparison group (p = .014).
Studies Measuring Outcome Study 1
Study Designs Experimental
Quality of Research Rating 3.0 (0.0-4.0 scale)

Study Populations

The following populations were identified in the studies reviewed for Quality of Research.

Study Age Gender Race/Ethnicity
Study 1 18-25 (Young adult)
26-55 (Adult)
55+ (Older adult) 		50% Female
50% Male 		100% Non-U.S. population
Study 2 18-25 (Young adult)
26-55 (Adult)
55+ (Older adult) 		70.3% Female
26.4% Male 		79.1% White
10.8% Hispanic or Latino
5.4% Race/ethnicity unspecified
3.4% Black or African American
1.4% Asian
Study 3 18-25 (Young adult)
26-55 (Adult) 		66.7% Female
33.3% Male 		Data not reported/available
Study 4 18-25 (Young adult)
26-55 (Adult)
55+ (Older adult) 		50% Female
50% Male 		73.9% White
16.3% Hispanic or Latino
5.4% Race/ethnicity unspecified
4.3% Black or African American

Quality of Research Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the Quality of Research for an intervention's reported results using six criteria:

  1. Reliability of measures
  2. Validity of measures
  3. Intervention fidelity
  4. Missing data and attrition
  5. Potential confounding variables
  6. Appropriateness of analysis

For more information about these criteria and the meaning of the ratings, see Quality of Research.

Outcome Reliability
of Measures 		Validity
of Measures 		Fidelity Missing
Data/Attrition 		Confounding
Variables 		Data
Analysis 		Overall
Rating
1: Therapeutic progress 4.0 3.5 2.5 2.3 3.0 3.3 3.1
2: Marital status 3.5 3.5 2.5 1.5 3.0 4.0 3.0

Study Strengths

The reliability and validity of the ORS are well supported. The studies were undertaken in a variety of real-world settings and included therapists with a variety of professional qualifications. Three of the four studies included random assignment, which helped mitigate the effect of potential confounds. The use of advanced analytical techniques in the same three studies produced strong and dependable findings.

Study Weaknesses

Although all studies appear to have used mechanisms to ensure intervention fidelity, there was no formal assessment of whether or to what extent the intervention was delivered as intended. Attrition was substantial in three of the four studies and difficult to assess in the fourth. Issues arising from the use of the ORS both as part of the intervention and as the outcome measure are not addressed.
Readiness for Dissemination

Materials Reviewed

The materials below were reviewed for Readiness for Dissemination. The implementation point of contact can provide information regarding implementation of the intervention and the availability of additional, updated, or new materials.

Duncan, B. CDOI/PCOMS Confidence Rating Scale (CCRS).

MyOutcomes. MyOutcomes: Quick start guide.

Other materials:

  • Heart and Soul of Change: Becoming Better at What We Do Conference Brochure.
  • Jefferson Center for Mental Health Newsletter (Issue 3)
  • Jefferson Center for Mental Health: Staff Input Survey for CDOI Implementation
  • Main Dissemination Document
  • Southwest Behavioral Health: Initial Engagement Session
  • Southwest Behavioral Health: Outcome Rating Scale
  • Southwest Behavioral Health: Service Plans
  • Southwest Behavioral Health: Session Rating

Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the intervention's Readiness for Dissemination using three criteria:

  1. Availability of implementation materials
  2. Availability of training and support resources
  3. Availability of quality assurance procedures

For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.

Implementation
Materials 		Training and Support
Resources 		Quality Assurance
Procedures 		Overall
Rating
2.8 3.3 4.0 3.3

Dissemination Strengths

The Heart and Soul of Change Project, an initiative to disseminate PCOMS through CDOI interaction, maintains a Web site with a wide array of materials and training resources to support implementation. Program materials clearly describe the rationale, instructions for use, and scoring of the ORS and SRS. A 2-day initial training provides an orientation to the intervention. Trainer certification is available through a weeklong intensive course that includes core competency exams. Additional support resources for practitioners are available in the form of booster trainings, online Webinars, and a Web forum. The developer is also available to provide consultation and supervision via phone and email. The PCOMS Ten Readiness Checklist provides clear organizational standards and key elements that must be in place for effective implementation, which strengthens fidelity. The PCOMS Therapist Adherence Scale helps to ensure that the practitioner involves the client in his or her treatment and that the client understands the rationale for completing these measures. The ORS and SRS provide client outcome data, and the developer provides recommendations on how to use these data for program improvement.

Dissemination Weaknesses

The program materials do not include an overview that clearly identifies and prioritizes the core resources, and information is limited on how practitioners should use these resources and in what sequence. Some supporting materials contain comments regarding the ineffectiveness of other evidence-based practices, which may be distracting to potential implementers. Limited information is provided regarding the required training.
Costs

The cost information below was provided by the developer. Although this cost information may have been updated by the developer since the time of review, it may not reflect the current costs or availability of items (including newly developed or discontinued items). The implementation point of contact can provide current information and discuss implementation requirements.

Item Description Cost Required by Developer
License for use of the following scales:
  • Child Group Session Rating Scale
  • Child Outcome Rating Scale
  • Child Session Rating Scale
  • Group Session Rating Scale
  • Outcome Rating Scale
  • Session Rating Scale (v.3.0)
  • Young Child Outcome Rating Scale
  • Young Child Session Rating Scale
  • For individual use: free
  • For agencies with 2-10 providers: $99.95
  • For agencies with 11-25 providers: $199.95
  • For agencies with 26-50 providers: $399.95
  • For agencies with 51-100 providers: $795.95
  • For agencies with more than 100 providers: contact the developer
 		Yes
PCOMS: Administration, Scoring, and Interpretation Manual Update for the Outcome and Session Rating Scales $99.95 for agencywide use Yes
Heroic Clients, Heroic Agencies, Partners for Change: An Implementation Manual for PCOMS and Client-Directed, Outcome-Informed Clinical Services (2nd ed.) $39.95 for a single user; $99.95 for unlimited users No
Online implementation materials (articles, handouts, and slides) Free No
5-day, off-site biannual Heart and Soul of Change Project Training of Trainers Conference (includes certification of trainers) $1,195 per person (maximum of 20 participants) No
2-day, on-site initial training for agency staff and supervisors
  • $4,000 per agency, plus travel expenses, for training conducted by certified trainer
  • $6,000 per agency, plus travel expenses, for training conducted by program developer
 		Yes
Bimonthly phone consultation (6 calls per year, 1-1.5 hours per call)
  • $1,200 per agency for consultation provided by certified trainer
  • $2,000 per agency for consultation provided by program developer
 		Yes
Additional phone or email consultation
  • $200 per hour for consultation provided by certified trainer
  • $300 per hour for consultation provided by program developer
 		No
$449 per agency No
1-year subscription to MyOutcomes
  • For 1-5 practitioners: $24.95 per month
  • For 5 or more practitioners: visit Web site for cost quote
 		No
Access to the Heart and Soul of Change Project electronic mailing list Free No
PCOMS Ten Readiness Checklist Free Yes, must be discussed with developer prior to training
PCOMS Therapist Skill Checklist Free Yes
PCOMS Therapist Adherence Scale Free Yes