Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Spoth, R., Guyll, M., & Day, S. X. Universal family-focused interventions in alcohol-use disorder prevention: Cost-effectiveness and cost-benefit analyses of two interventions. Journal of Studies on Alcohol, 63(2), 219-228.
Spoth, R., Randall, G. K., & Shin, C. (in press). Increasing school success through partnership-based family competency training: Experimental study of long-term outcomes. School Psychology Quarterly.
Spoth, R., Redmond, C., & Shin, C. Reducing adolescents' aggressive and hostile behaviors: Randomized trial effects of a brief family intervention 4 years past baseline. Archives of Pediatrics and Adolescent Medicine, 154(12), 1248-1257.
Spoth, R. L., Redmond, C., & Shin, C. Randomized trial of brief family interventions for general populations: Adolescent substance use outcomes 4 years following baseline. Journal of Consulting and Clinical Psychology, 69(4), 627-642.
Supplementary Materials Spoth, R. L., Clair, S., Shin, C., & Redmond, C. Long-term effects of universal preventive interventions on methamphetamine use among adolescents. Archives of Pediatrics and Adolescent Medicine, 160(9), 876-882.
Outcomes
| Outcome 1: Substance use |
|
Description of Measures
|
Substance use was evaluated with self-report measures of alcohol use, alcohol use without parental permission, drunkenness, cigarette use, and marijuana use during the lifetime and the past 30 days, as well as the frequency of these occurrences.
Methamphetamine use was measured with questions asking students if they had ever used the drug and if they had used it during the past 12 months.
|
|
Key Findings
|
Sixth-grade students participated in an intervention group receiving SFP 10-14 or a minimal contact control group receiving four mailed leaflets on adolescent development and parent-child relationships. At the 4-year follow-up, the proportion of new substance users in the intervention group was significantly lower than that in the control group for all five measures of lifetime substance use. For example:
- 50% of students who received the intervention reported having ever tried alcohol, compared with 68% of control group students, representing a relative reduction of 26.4% (p < .01).
- 40% of students who received the intervention reported using alcohol without parental permission, compared with 59% of control group students, representing a relative reduction of 32% (p < .01).
- 26% of students who received the intervention reported having ever been drunk, compared with 44% of control group students, representing a relative reduction of 40.1% (p < .01).
- 33% of students who received the intervention reported having ever smoked cigarettes, compared with 50% of control group students, representing a relative reduction of 34.8% (p < .01).
- 7% of students who received the intervention reported having ever tried marijuana, compared with 17% of control group students, representing a relative reduction of 55.4% (p < .01).
Also at the 4-year follow-up, the frequency of alcohol and cigarette use was lower among the intervention group than the control group (p < .05). The effects were small in size (Cohen's d = 0.26 and 0.31, respectively).
At the 6-year follow-up, no students who received the intervention reported using methamphetamine, whereas 3.21% of the control group reported using it (p = .04).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 2: School success |
|
Description of Measures
|
School success was evaluated using a structural model that included measures of academic success, school engagement, and two factors influencing school success: student substance use-related risk and parenting competency. Specifically:
- Academic success was measured through school grades as reported by the mother, father, and child.
- School engagement was measured using self-reported affective, behavioral, and cognitive dimensions of school engagement.
- Student substance use-related risk was measured through lifetime alcohol use (i.e., initiation), attitude toward alcohol use, and peer pressure to use alcohol.
- Parenting competency was measured as use of parental rules, parental involvement, anger management, and parental communication.
|
|
Key Findings
|
Sixth-grade students participated in an intervention group receiving SFP 10-14 or a minimal contact control group receiving four mailed leaflets on adolescent development and parent-child relationships. Participants provided data at three time points: in grades 6 (at posttest), 8, and 12. Increased parenting competency and reduced student substance use-related risk measured in the 6th grade were associated with positive effects on school engagement measured in the 8th grade (p < .05), which in turn was associated with positive effects in academic performance in the 12th grade (p < .05).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.9
(0.0-4.0 scale)
|
| Outcome 3: Aggression |
|
Description of Measures
|
Aggression was assessed using three measures:
- Observer Index of Aggressive and Hostile Behaviors (OIAHB), which comprises nine behavioral scales from the Iowa Family Interaction Rating Scales (IFIRS): Physical Attack, Verbal Attack, Hostility, Escalate Hostile, Reciprocate Hostile, Angry Coercion, Contempt, External Negative, and Antisocial.
- Parent-Adolescent Report of Aggressive and Hostile Behaviors in Interactions (PARAHB), which includes 5 parallel parent and child questions drawn from the IFIRS dealing with the following areas of parent-child interactions: hit, pushed, or shoved them; shouted or yelled at them; argued when they disagreed; criticized their ideas; and got angry at them. Parent and child scores were averaged to enhance accuracy.
- Adolescent-Reported Index of Aggressive and Destructive Conduct (ARIADC), which includes 4 items from the National Youth Survey: fighting with someone, throwing rocks or bottles to cause injury, purposely damaging property, and breaking into a building.
|
|
Key Findings
|
Sixth-grade students participated in an intervention group receiving SFP 10-14 or a minimal contact control group receiving four mailed leaflets on adolescent development and parent-child relationships. At the 4-year follow-up, students who received the intervention exhibited fewer aggressive and hostile behaviors on the OIAHB (39.7%) than control group students (49.2%, p < .05). The effect size was small (Cohen's d = 0.33). In addition, students in the intervention group exhibited significantly less aggressive and hostile behavior toward their mothers compared with those in the control group (p = .04).
Also at the 4-year follow-up, students who received the intervention reported lower levels of aggressive and destructive conduct on the ARIADC (14.6%) than control group students (24.5%). The effect size was small (Cohen's d = 0.35). The percentage of students reporting breaking and entering was higher among the control group than the intervention group (7.9% vs. 2.0%), a difference that represents a relative reduction of 74.7%. The relative reduction rates for physical fighting, throwing items to cause injury, and purposely damaging property were 31.7%, 53.5%, and 77.0%, respectively.
There were no statistically significant findings on aggressive or destructive behaviors as measured by the PARAHB.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 4: Cost effectiveness |
|
Description of Measures
|
The cost-effectiveness analysis was based on the premise that negative family and peer influences lead to early-onset alcohol use and eventually to adult alcohol use disorders. Cost effectiveness was measured using several estimates calculated by the researchers, including the following:
- Cost of the intervention, including direct and indirect costs, adjusted for inflation intodollars: $80,562.
- Number of cases of alcohol use disorders prevented by the intervention per 100 families treated: 5.53.
- Cost per case of alcohol use disorders prevented by the intervention: $12,459.
- Average future benefit realized by preventing a single alcohol disorder case in the intervention group: $119,633. This figure is based on the societal cost of alcohol disorders, estimated at $148 billion by H. Harwood, D. Fountain, and G. Livermore (The Economic Costs of Alcohol and Drug Abuse in the United States, report prepared for the National Institutes of Health). Researchers calculated the average future benefit realized at a particular year of age by preventing a single alcohol disorder case in adolescence and, subsequently, the value of the total lifetime benefit realized by the prevention of a single alcohol use disorder. The costs documented by Harwood et al. were based on the human capital approach, which limits benefit estimates to the avoidance of tangible economic burdens such as lost wages, decreased productivity, and spent resources.
|
|
Key Findings
|
Sixth-grade students participated in an intervention group receiving SFP 10-14 or a minimal contact control group receiving four mailed leaflets on adolescent development and parent-child relationships. Data were analyzed for age of alcohol use onset at each of seven data collection points ending at the 12th grade. Using the estimated costs of the intervention, number of cases of alcohol use disorders prevented by the intervention, cost per case of alcohol use disorders prevented by the intervention, and average benefit realized by preventing one case of an alcohol use disorder, researchers estimated the SFP 10-14 benefit-cost ratio to be 9.60 (i.e., $9.60 saved for every dollar invested) and the net benefit for each participating family to be $5,923.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
26-55 (Adult)
|
53% Female
47% Male
|
98.8% White
1.2% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Substance use
|
2.8
|
2.8
|
2.8
|
3.0
|
2.5
|
3.0
|
2.8
|
|
2: School success
|
2.8
|
2.8
|
2.8
|
3.0
|
2.8
|
3.5
|
2.9
|
|
3: Aggression
|
3.0
|
3.0
|
2.8
|
2.8
|
3.3
|
3.5
|
3.0
|
|
4: Cost effectiveness
|
3.5
|
3.5
|
2.8
|
3.0
|
3.3
|
4.0
|
3.3
|
Study Strengths In-home interviews were conducted for the initial screening, which helped limit potential confounds. The studies used appropriate measures and systems for ensuring fidelity of implementation. The selection of measures and analyses of cost effectiveness were appropriate. The use of parental reports of grades in addition to student reports was a notable strength of the school success outcome.
Study Weaknesses The inclusion of objective measures and collateral reports of drug use would have further strengthened confidence in the results. While the measures of school success were generally good, examination of actual grades and school attachment and involvement would have been useful additions to the study of this outcome.
|
|
Readiness for Dissemination
|
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Institute for Social and Behavioral Research for PROJECT FAMILY & Iowa State University Extension (Producers). Strengthening Families Program: For Parents and Youth 10-14: What it's all about [Motion picture]. Ames, IA: Iowa State University Research Foundation.
Iowa State University Extension (Producer). Strengthening Families Program: For Parents and Youth 10-14 [Motion picture]. Ames, IA: Iowa State University Research Foundation.
Molgaard, V. K., Kumpfer, K., & Fleming, E. Strengthening Families Program: For Parents and Youth 10-14: Leader guide for booster sessions. Ames, IA: Iowa State University Research Foundation.
Molgaard, V. K., Kumpfer, K., & Fleming, E. Strengthening Families Program: For Parents and Youth 10-14: Leader guide. Ames, IA: Iowa State University Research Foundation.
Quality Assurance Protocol: Observation Forms To Use in Assessing Program Fidelity
Strengthening Families Program: For Parents and Youth 10-14--Training folder
Strengthening Families Program: For Parents and Youth 10-14--Training-of-trainers folder
Strengthening Families Program: For Parents and Youth 10-14--Training PowerPoint
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
4.0
|
4.0
|
4.0
|
4.0
|
Dissemination Strengths The implementation package, which includes a fully scripted curriculum with extensive details for program facilitators, is designed for use with diverse audiences. One person from each implementing organization is required to participate in the program developer's extensive training-of-trainers course to facilitate additional training within the organization. Detailed, rigorously developed, and easy-to-use fidelity and outcome measures are provided to support quality assurance.
Dissemination Weaknesses No weaknesses were identified by reviewers.
|
|