Quality of Research
Review Date: December 2007
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Morral, A. R., McCaffrey, D. F., & Ridgeway, G. (2004). Effectiveness of community-based treatment for substance-abusing adolescents: 12-month outcomes of youths entering Phoenix Academy or alternative probation dispositions. Psychology of Addictive Behaviors, 18(3), 257-268. 
Supplementary Materials Dennis, M. L., Dawud-Noursi, S., Muck, R., & McDermeit, M. (2003). The need for developing and evaluating adolescent treatment models. In S. J. Stevens & A. R. Morral (Eds.), Adolescent substance abuse treatment in the United States: Exemplary models from a national evaluation study (pp. 3-34). Binghamton, NY: Haworth Press.
Engberg, J., & Morral, A. R. (2006). Reducing substance use improves adolescents' school attendance. Addiction, 101(12), 1741-1751.
McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9(4), 403-425.
Morral, A. R., Jaycox, L. H., Smith, W., Becker, K., & Ebener, P. (2003). An evaluation of substance abuse treatment services for juvenile probationers at Phoenix Academy of Los Angeles. In S. J. Stevens & A. R. Morral (Eds.), Adolescent substance abuse treatment in the United States: Exemplary models from a national evaluation study (pp. 213-233). Binghamton, NY: Haworth Press.
Outcomes
| Outcome 1: Substance use |
|
Description of Measures
|
The Global Appraisal of Individual Needs (GAIN) was used to measure substance use along three dimensions: substance problems (symptoms from the DSM-IV) in the previous month, the extent of substance use (total days of reported use of 12 classes of drugs) in the past 3 months, and substance involvement (range of use across 12 classes of drugs) in the past 3 months. The GAIN is a standardized interview that assesses eight functional domains: background, substance use, physical health, risk behaviors, mental health, environment, legal, and vocational.
A fourth measure of substance use was recent tobacco smoking. A single question was answered on a 7-point Likert-type scale from 0 (never) to 6 (past 2 days).
|
|
Key Findings
|
Adolescent probationers assigned to Phoenix House Academy showed a larger reduction in the reported extent of substance use, substance involvement, and recent smoking from baseline to the 3-month follow-up than adolescents assigned to the comparison condition, an alternative probation disposition (p < .05). The effect sizes for the extent of substance use (Cohen's d = 0.32), substance involvement (Cohen's d = 0.29), and recent tobacco smoking (Cohen's d = 0.30) were small.
From the 3- to 12-month follow-up, the reported extent of substance use and substance involvement remained lower among adolescents assigned to Phoenix House Academy than among those in the comparison group (p < .05). The associated effect sizes for these group differences were small (Cohen's d = 0.25 and 0.24 for the extent of substance use and substance involvement, respectively). However, during the same timeframe, Phoenix House Academy adolescents showed increases in recent tobacco smoking relative to adolescents in the comparison group (p < .005). A small effect size was associated with this finding (Cohen's d = 0.45).
Adolescents in Phoenix House Academy showed a greater reduction in substance problems from baseline to the 12-month follow-up than adolescents in the comparison group (p < .05). A small effect size was associated with this group difference (Cohen's d = 0.27).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 2: Psychological functioning |
|
Description of Measures
|
Psychological functioning was measured using three subscales of the GAIN: somatic symptoms, depressive symptoms, and anxiety symptoms. The GAIN is a standardized interview that assesses eight functional domains: background, substance use, physical health, risk behaviors, mental health, environment, legal, and vocational.
Another form of psychological functioning, recent experience of psychological distress, was measured using a single item: "When was the last time (if ever) your life was disturbed by emotional, nerve, mental, or psychological problems, or that you felt you could not go on?" Responses were on a Likert-type scale ranging from 0 (never) to 6 (within the past 2 days).
|
|
Key Findings
|
Adolescent probationers assigned to Phoenix House Academy showed greater decreases in somatic (p < .05), depressive (p < .005), and anxiety (p < .005) symptoms from the 3- to 12-month follow-up than adolescents assigned to the comparison condition, an alternative probation disposition. These group differences were associated with small effect sizes (Cohen's d = 0.27, 0.34, and 0.39 for somatic, depressive, and anxiety symptoms, respectively). From baseline to the 12-month follow-up, adolescents in Phoenix House Academy showed larger reductions in somatic (p < .01) and anxiety (p < .05) symptoms than adolescents in the comparison group. A small effect size also was associated with these group differences (Cohen's d = 0.32 and 0.29 for somatic and anxiety symptoms, respectively).
Recent experience of psychological distress declined from the 3- to 12-month follow-up for adolescents in the Phoenix House Academy and comparison groups, with no significant group differences detected.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
|
87.3% Male
12.7% Female
|
55.2% Hispanic or Latino
16% White
14.7% Black or African American
14% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Substance use
|
4.0
|
4.0
|
1.5
|
4.0
|
2.5
|
4.0
|
3.3
|
|
2: Psychological functioning
|
4.0
|
4.0
|
1.5
|
4.0
|
2.5
|
4.0
|
3.3
|
Study Strengths The measures used were psychometrically sound. Use of the GAIN adds to the confidence that outcomes could be measured with reliability and internal validity. Additionally, the GAIN measures screen for symptoms associated with DSM-IV disorders. The study employed accepted techniques for analysis of observational data that adjust for pretreatment group differences when random assignment is not feasible.
Study Weaknesses The comparison condition represented different types of residential treatment using a wide range of therapeutic approaches. The fidelity of the treatment programs was not evaluated, and no information was provided about the qualifications or training of the staff. The analyses were global tests of the effects of one type of residential treatment against those of a group of other types of residential treatment, some of which were not centered on treating substance abuse.
|
|
Readiness for Dissemination
Review Date: December 2007
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Description of training and support resources
Jaycox, L. H., Marshall, G. N., & Morral, A. R. (n.d.). Phoenix Academy at Lake View Terrace, California: Clinical manual and program description of an adolescent therapeutic community. Santa Monica, CA: RAND.
Phoenix House Foundation, Inc. (1998). Therapeutic communities for adolescents: A clinical manual. New York: Author.
Phoenix House Foundation, Inc. (2006). Phoenix House residential program quality improvement instrument. New York: Author.
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.0
|
2.5
|
4.0
|
3.2
|
Dissemination Strengths Program materials include helpful information about staff selection, training, and supervision. Initial training is followed by planned inservice training sessions, and additional follow-up support is available as needed. An excellent quality assurance review process is in place to support both fidelity and outcome monitoring.
Dissemination Weaknesses Very little guidance is provided to assist a new organization in selecting and preparing to implement this model. More substantial guidance for staff coaching is needed.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Clinical manual
|
Free
|
Yes
|
|
1- to 5-week, on-site training
|
Varies depending on site needs
|
Yes
|
|
On-site posttraining technical assistance and/or phone consultation
|
Varies depending on site needs
|
No
|
Additional Information Start-up costs vary by region, facility size, resources, and implementation stage.
|
Replications
No replications were identified by the developer.
|
|
|
