Intervention Summary
Partners for Change Outcome Management System (PCOMS): The Heart and Soul of Change Project
The Partners for Change Outcome Management System (PCOMS) is a client feedback program for improving the treatment outcomes of adults and children participating in a behavioral health care intervention. PCOMS is designed to improve the retention of participants in treatment and to assist them in reaching reliable and clinically significant change. The program can be implemented by behavioral health care therapists as part of any behavioral health care intervention. PCOMS, which is integrated into each treatment session, consists of two brief scales that measure robust predictors of therapeutic success: The therapist administers the ORS at the beginning of the treatment session, and the SRS is administered toward the end of the session. Client ratings for both measures are discussed on a session-by-session basis to maintain the client's engagement in treatment, optimize the client-therapist alliance, and provide a means for transitioning into the treatment session by focusing on client-identified concerns. If client ratings are very low, the therapist may choose to modify the type and amount of treatment. PCOMS is disseminated through the Heart and Soul of Change Project and the International Center for Clinical Excellence. (The Readiness for Dissemination of each version was reviewed separately by NREPP.) The Heart and Soul of Change Project integrates PCOMS into clinical practice through client-directed, outcome-informed (CDOI) interaction, which enhances the common factors across theories that account for therapeutic change, includes the client in determining the benefit of services and making decisions that affect care, informs the treatment with measures of the client's experience, and involves an expectation of recovery and an appreciation of social justice in the provision of care.
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
The following populations were identified in the studies reviewed for Quality of
Research.
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
The reliability and validity of the ORS are well supported. The studies were undertaken in a variety of real-world settings and included therapists with a variety of professional qualifications. Three of the four studies included random assignment, which helped mitigate the effect of potential confounds. The use of advanced analytical techniques in the same three studies produced strong and dependable findings. Although all studies appear to have used mechanisms to ensure intervention fidelity, there was no formal assessment of whether or to what extent the intervention was delivered as intended. Attrition was substantial in three of the four studies and difficult to assess in the fourth. Issues arising from the use of the ORS both as part of the intervention and as the outcome measure are not addressed.
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Duncan, B. CDOI/PCOMS Confidence Rating Scale (CCRS). MyOutcomes. MyOutcomes: Quick start guide. Other materials:
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
The Heart and Soul of Change Project, an initiative to disseminate PCOMS through CDOI interaction, maintains a Web site with a wide array of materials and training resources to support implementation. Program materials clearly describe the rationale, instructions for use, and scoring of the ORS and SRS. A 2-day initial training provides an orientation to the intervention. Trainer certification is available through a weeklong intensive course that includes core competency exams. Additional support resources for practitioners are available in the form of booster trainings, online Webinars, and a Web forum. The developer is also available to provide consultation and supervision via phone and email. The PCOMS Ten Readiness Checklist provides clear organizational standards and key elements that must be in place for effective implementation, which strengthens fidelity. The PCOMS Therapist Adherence Scale helps to ensure that the practitioner involves the client in his or her treatment and that the client understands the rationale for completing these measures. The ORS and SRS provide client outcome data, and the developer provides recommendations on how to use these data for program improvement. The program materials do not include an overview that clearly identifies and prioritizes the core resources, and information is limited on how practitioners should use these resources and in what sequence. Some supporting materials contain comments regarding the ineffectiveness of other evidence-based practices, which may be distracting to potential implementers. Limited information is provided regarding the required training.
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
Descriptive Information
Areas of Interest
Mental health treatment
Outcomes
1: Therapeutic progress
2: Marital status
Outcome Categories
Family/relationships
Mental health
Treatment/recovery
Ages
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
Genders
Male
Female
Races/Ethnicities
Asian
Black or African American
Hispanic or Latino
White
Race/ethnicity unspecified
Non-U.S. population
Settings
Outpatient
Workplace
Geographic Locations
Urban
Suburban
Rural and/or frontier
Implementation History
The Heart and Soul of Change Project was founded to disseminate PCOMS. PCOMS has been used by hundreds of organizations and by thousands of behavioral health care professionals in all 50 States and the District of Columbia and, internationally, in 20 countries on five continents. It is estimated that PCOMS is implemented with approximately 100,000 clients each year.
NIH Funding/CER Studies
Partially/fully funded by National Institutes of Health: No
Evaluated in comparative effectiveness research studies: No
Adaptations
The PCOMS scales (ORS and SRS) have been translated into 23 languages: Afrikaans, Bulgarian, Chinese, Croatian, Danish, Dutch, Finnish, French, German, Greek, Hebrew, Italian, Japanese, Maori, Norwegian, Polish, Romanian, Russian, Slovak, Spanish, Swedish, Welsh, and Yiddish.
Adverse Effects
No adverse effects, concerns, or unintended consequences were identified by the developer.
IOM Prevention Categories
IOM prevention categories are not applicable.
Documents Reviewed
Outcomes
Outcome 1: Therapeutic progress
Description of Measures
Therapeutic progress was assessed using the Outcome Rating Scale. The ORS is a 4-item self-report scale that measures a participant's psychological functioning and distress by asking how the participant is doing individually (personal well-being), interpersonally (family, close relationships), socially (work, school, friendships), and overall (general sense of well-being). Depending on the method of ORS administration, participants rate each item either by marking a line to reflect the strength of their opinions or by stating the appropriate ratings. Scores for each item range from 1 to 10, with a total score ranging from 0 to 40. Lower scores indicate more severe distress, and a comparison of the change in scores over time indicates the trajectory of therapeutic progress.
Key Findings
In one study, participating couples were randomly assigned to the intervention group, which received couple therapy with PCOMS, or the comparison group, which received couple therapy only. Couples in the intervention group had higher ORS scores than those in the comparison group at posttreatment (p < .001) and 6-month follow-up (p < .01), even after adjusting for pretreatment ORS score and therapist.
In another study, participants were randomly assigned to the intervention group, which received individual therapy with PCOMS, or the comparison group, which received individual therapy only. Half of the participants received services from faculty members or practicum students at a university counseling center, and half received services from practicum students at a graduate training clinic. Among participants who received services at a university counseling center, from pre- to posttest, those in the intervention group had a greater increase in ORS scores than those in the comparison group (p < .05). Among participants who received services at a graduate training clinic, from pre- to posttest, those in the intervention group had a greater increase in ORS scores than those in the comparison group (p < .01), even after adjusting for pretest ORS scores.
In a third study, participants in an employee assistance program received at least two phone-based counseling sessions that included PCOMS. The study had multiple phases, including a 6-month baseline period, when PCOMS was administered during the counseling sessions; a subsequent 6-month period, when a computer program was introduced to aid therapists in administering PCOMS and interpreting a participant's ORS and SRS scores during counseling sessions; and a 12-month period, after use of the computer program had been fully integrated into the counseling sessions. The average change in participants' ORS scores between counseling sessions was compared over these three phases. The mean increase in participants' ORS scores was larger for the subsequent 6-month period (p < .001) and 12-month period (p < .001) than it was for the baseline period.
In a fourth study, participating couples were randomly assigned to the intervention group, which received couple therapy with PCOMS, or the comparison group, which received couple therapy only. From pre- to posttest, couples in the intervention group had a greater increase in ORS scores than those in the comparison group (p < .05).
Studies Measuring Outcome
Study 1, Study 2, Study 3, Study 4
Study Designs
Experimental, Preexperimental
Quality of Research Rating
3.1
(0.0-4.0 scale)
Outcome 2: Marital status
Description of Measures
Marital status was assessed by client self-report. Couples were categorized as intact (i.e., not divorced or separated) or not intact.
Key Findings
Couples participating in the study were randomly assigned to the intervention group, which received couple therapy with PCOMS, or the comparison group, which received couple therapy only. At the 6-month follow-up, a greater proportion of couples in the intervention group were intact relative to couples in the comparison group (p = .014).
Studies Measuring Outcome
Study 1
Study Designs
Experimental
Quality of Research Rating
3.0
(0.0-4.0 scale)
Study Populations
Study
Age
Gender
Race/Ethnicity
Study 1
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
50% Female
50% Male
100% Non-U.S. population
Study 2
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
70.3% Female
26.4% Male
79.1% White
10.8% Hispanic or Latino
5.4% Race/ethnicity unspecified
3.4% Black or African American
1.4% Asian
Study 3
18-25 (Young adult)
26-55 (Adult)
66.7% Female
33.3% Male
Data not reported/available
Study 4
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
50% Female
50% Male
73.9% White
16.3% Hispanic or Latino
5.4% Race/ethnicity unspecified
4.3% Black or African American
Quality of Research Ratings by Criteria (0.0-4.0 scale)
Outcome
Reliability
of Measures
Validity
of Measures
Fidelity
Missing
Data/Attrition
Confounding
Variables
Data
Analysis
Overall
Rating
1: Therapeutic progress
4.0
3.5
2.5
2.3
3.0
3.3
3.1
2: Marital status
3.5
3.5
2.5
1.5
3.0
4.0
3.0
Study Strengths
Study Weaknesses
Materials Reviewed
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
Implementation
Materials
Training and Support
Resources
Quality Assurance
Procedures
Overall
Rating
2.8
3.3
4.0
3.3
Dissemination Strengths
Dissemination Weaknesses
Item Description
Cost
Required by Developer
License for use of the following scales:
Yes
PCOMS: Administration, Scoring, and Interpretation Manual Update for the Outcome and Session Rating Scales
$99.95 for agencywide use
Yes
Heroic Clients, Heroic Agencies, Partners for Change: An Implementation Manual for PCOMS and Client-Directed, Outcome-Informed Clinical Services (2nd ed.)
$39.95 for a single user; $99.95 for unlimited users
No
Online implementation materials (articles, handouts, and slides)
Free
No
5-day, off-site biannual Heart and Soul of Change Project Training of Trainers Conference (includes certification of trainers)
$1,195 per person (maximum of 20 participants)
No
2-day, on-site initial training for agency staff and supervisors
Yes
Bimonthly phone consultation (6 calls per year, 1-1.5 hours per call)
Yes
Additional phone or email consultation
No
$449 per agency
No
1-year subscription to MyOutcomes
No
Access to the Heart and Soul of Change Project electronic mailing list
Free
No
PCOMS Ten Readiness Checklist
Free
Yes, must be discussed with developer prior to training
PCOMS Therapist Skill Checklist
Free
Yes
PCOMS Therapist Adherence Scale
Free
Yes