Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Dennis, M., Godley, S. H., Diamond, G., Tims, F. M., Babor, T., Donaldson, J., et al. (2004). The Cannabis Youth Treatment (CYT) study: Main findings from two randomized trials. Journal of Substance Abuse Treatment, 27(3), 197-213. 
Godley, S. H. (2007). Supplemental analyses regarding the effectiveness of A-CRA using urine data. Unpublished manuscript. Study 2Godley, M. D., Godley, S. H., Dennis, M. L., Funk, R. R., & Passetti, L. L. (2007). The effect of assertive continuing care on continuing care linkage, adherence and abstinence following residential treatment for adolescents with substance use disorders. Addiction, 102(1), 81-93. Study 3Slesnick, N., Prestopnik, J. L., Meyers, R. J., & Glassman, M. (2007). Treatment outcome for street-living, homeless youth. Addictive Behaviors, 32(6), 1237-1251.
Supplementary Materials Dennis, M. L., Funk, R., Godley, S. H., Godley, M. D., & Waldron, H. (2004). Cross-validation of the alcohol and cannabis use measures in the Global Appraisal of Individual Needs (GAIN) and Timeline Followback (TLFB; Form 90) among adolescents in substance abuse treatment. Addiction, 99(Suppl. 2), 120-128.
Dennis, M. L., Titus, J. C., Diamond, G., Donaldson, J., Godley, S. H., Tims, F. M., et al. (2002). The Cannabis Youth Treatment (CYT) experiment: Rationale, study design, and analysis plans. Addiction, 97(Suppl. 1), 16-34.
Finney, J. W., & Monahan, S. C. (1996). The cost-effectiveness of treatment for alcoholism: A second approximation. Journal of Studies on Alcohol, 57(3), 229-243.
French, M. T., Roebuck, M. C., Dennis, M. L., Diamond, G., Godley, S. H., Tims, F., et al. (2002). The economic cost of outpatient marijuana treatment for adolescents: Findings from a multi-site field experiment. Addiction, 97(Suppl. 1), 84-97.
Garner, B. R., Godley, S. H., Funk, R. R., Dennis, M. L., Smith, J. E., & Godley, M. D. (2007). A treatment fidelity measure for the Adolescent Community Reinforcement Approach: Development and validation of the A-CRA Implementation Scale (AIS). Unpublished manuscript.
Garner, B. R., Godley, S. H., Funk, R. R., Dennis, M. L., Smith, J. E., & Godley, M. D. (2007). Exposure to A-CRA treatment procedures as a mediator of the relationship between adolescent substance abuse treatment retention and outcome. Unpublished manuscript.
Godley, M. D., Godley, S. H., Dennis, M. L., Funk, R., & Passetti, L. L. (2002). Preliminary outcomes from the assertive continuing care experiment for adolescents discharged from residential treatment. Journal of Substance Abuse Treatment, 23(1), 21-32.
Outcomes
| Outcome 1: Abstinence from substance use |
|
Description of Measures
|
Abstinence from substance use was measured using the Global Appraisal of Individual Needs (GAIN) and was defined by the total number of abstinent days over four 3-month follow-up periods (3, 6, 9, and 12 months after intake). Abstinence also was defined by the percentage of participants who reported, at 30 months after intake, that they had been abstinent for the entire previous month. Abstinence from marijuana use, also measured using the GAIN, was defined by either abstinence during the month prior to the 30-month follow-up or sustained abstinence across 9 months following discharge from residential care. The GAIN is a standardized, semistructured interview with eight main sections (background, substance use, physical health, risk behaviors, mental health, environment, legal, and vocational) that is designed to support the diagnosis, placement, and outcome monitoring of patients and the economic analysis of an intervention.
Self-report data were confirmed by urinalysis at intake and at the follow-up assessments 3, 6, and 30 months after intake.
|
|
Key Findings
|
In two trials of a randomized controlled study, five manual-driven treatment interventions for adolescents with cannabis-related disorders were compared.
Trial 1 compared the following interventions at two sites:
- Motivational Enhancement Therapy/Cognitive Behavior Therapy with 5 sessions (MET/CBT5)
- MET/CBT with 12 sessions (MET/CBT12)
- Family Support Network (FSN)
Trial 2 compared the following interventions at two sites:
- A-CRA
- MET/CBT5
- Multidimensional Family Therapy (MDFT)
In both trials, assessments were conducted at intake and at 3-, 6-, 9-, and 12-month follow-ups. Trial 2 additionally included a 30-month follow-up as part of the Persistent Effects of Treatment Studies--Adolescents (PETS-A) study. Findings from these trials included:
- Days of total substance abstinence reported by all participants increased 24% (from 52 to 65 days) between intake and the 3-month follow-up, with no significant differences by intervention. Similar gains were seen at each of the subsequent follow-ups (6, 9, and 12 months after intake), regardless of intervention, site, or trial participation. Across both trials, total days reported abstinent from all substances over 12 months ranged from 251 to 269 days, with no significant differences by intervention.
- At the 30-month follow-up, A-CRA participants were more likely to report total substance abstinence in the prior month and have a negative urine screen (24%) than MDFT participants (16%) and MET/CBT5 participants (21%). A-CRA participants also were more likely to report marijuana abstinence in the prior month and have a negative urine screen (36%) than MDFT participants (34%) and MET/CBT5 participants (31%). However, these group differences were not significant.
- When urine screens were excluded from the analysis, the percentage of participants reporting abstinence from all substances or from marijuana was slightly higher for all groups (29% and 51%, respectively, of A-CRA participants; 22% and 47%, respectively, of MDFT participants; and 25% and 39%, respectively, of MET/CBT5 participants). As with the results that included urine screening, the differences between groups were not significant.
In a randomized controlled trial (RCT), adolescents with substance dependence who were preparing for discharge from residential treatment were assigned to receive either usual continuing care (UCC) through a referral to 1 of 12 community outpatient substance abuse clinics or UCC plus Assertive Continuing Care (ACC) with A-CRA, which involved a case manager who provided home visits for 3 months. UCC varied with type of discharge. Adolescents discharged "against staff advice" or "at staff request" received only a letter with information on where to go for further treatment. Adolescents discharged "as planned" received a continuing care appointment with a case manager that was typically scheduled within 2 weeks of discharge. Follow-up interviews occurred 3, 6, and 9 months after discharge. Findings from this study included:
- At the 9-month follow-up, 41% of ACC participants reported sustained abstinence from marijuana, compared with 26% of UCC participants (p = .04). This difference was associated with a small effect size (Cohen's d = 0.32).
- Participants who reported sustained abstinence at the 3-month follow-up were more likely than nonabstinent participants to report sustained abstinence between the 3- and 9-month follow-ups, regardless of intervention assignment. Specifically, those reporting abstinence from marijuana at 3 months were 11 times more likely to remain abstinent, those reporting abstinence from alcohol were 5 times more likely to remain abstinent, and those reporting abstinence from substances other than marijuana and alcohol were 11 times more likely to remain abstinent (p < .05 for all analyses). The effect sizes for these findings were large (odds ratio = 11.15, 5.47, and 11.16, respectively).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 2: Recovery from substance use |
|
Description of Measures
|
Recovery from substance use was measured by the percentage of adolescents living in the community (as opposed to living in a correctional facility, inpatient treatment program, or other controlled environment) and reporting no past-month substance use, abuse, or dependence problems at the 12-month and 30-month interviews. For the 6% of adolescents who did not complete a 12-month follow-up, data from their last follow-up were used to determine their recovery status.
|
|
Key Findings
|
In two trials of a randomized controlled study, five manual-driven treatment interventions for adolescents with cannabis-related disorders were compared.
Trial 1 compared the following interventions at two sites:
- Motivational Enhancement Therapy/Cognitive Behavior Therapy with 5 sessions (MET/CBT5)
- MET/CBT with 12 sessions (MET/CBT12)
- Family Support Network (FSN)
Trial 2 compared the following interventions at two sites:
- A-CRA
- MET/CBT5
- Multidimensional Family Therapy (MDFT)
In both trials, assessments were conducted at intake and at 3-, 6-, 9-, and 12-month follow-ups. Trial 2 additionally included a 30-month follow-up as part of the Persistent Effects of Treatment Studies--Adolescents (PETS-A) study. Findings from these trials included:
- The percentage of participants who reported being in recovery at the 12-month follow-up showed small differences by intervention assignment after controlling for site and recovery status in the month prior to intake, but the differences were not significant. Thirty-four percent of A-CRA participants reported that they were in recovery at the 12-month follow-up, compared with 23% of MET/CBT5 (Trial 2) participants, 19% of MDFT participants, 28% of MET/CBT5 (Trial 1) participants, 17% of MET/CBT12 participants, and 22% of FSN participants. The higher percentage of A-CRA participants reporting that they were in recovery at the 12-month follow-up reflected a site-specific increase that was also significant by intervention. Specifically, at one of the two sites in Trial 2, a greater percentage of A-CRA participants were in recovery than MET/CBT5 or MDFT participants at 12 months (p < .05). This difference was associated with a small effect size (Cohen's f = 0.20).
- At the 30-month follow-up, 13% of A-CRA participants reported being in recovery and had a negative urine screen, compared with 3% of the MET/CBT5 participants (p = .017). When urine screens were excluded from the analysis, the percentages increased to 17% of A-CRA participants and 6% of MET/CBT5 participants (p = .025). A-CRA participants were more than 4 times more likely to be in recovery at the 30-month follow-up relative to MET/CBT5 participants, a difference that represents a medium effect size (odds ratio = 4.64).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.7
(0.0-4.0 scale)
|
| Outcome 3: Cost effectiveness |
|
Description of Measures
|
Cost effectiveness was measured by the average daily cost of an adolescent achieving abstinence from substance use and the cost of an adolescent being in recovery 12 months after intake. Abstinence from substance use was measured using the GAIN and was defined by the total number of abstinent days during the 12-month period following intake. Recovery was defined as living in the community (as opposed to living in a correctional facility, inpatient treatment program, or other controlled environment) and reporting no past-month substance use, abuse, or dependence problems at the 12-month follow-up. Cost estimates were based on data collected using the Drug Abuse Treatment Cost Analysis Program (DATCAP), which measures both the accounting and opportunity costs of a substance abuse treatment program based on standard economic principles. DATCAP was supplemented with service contact logs completed by therapists and case managers.
|
|
Key Findings
|
In two trials of a randomized controlled study, five manual-driven treatment interventions for adolescents with cannabis-related disorders were compared.
Trial 1 compared the following interventions at two sites:
- Motivational Enhancement Therapy/Cognitive Behavior Therapy with 5 sessions (MET/CBT5)
- MET/CBT with 12 sessions (MET/CBT12)
- Family Support Network (FSN)
Trial 2 compared the following interventions at two sites:
- A-CRA
- MET/CBT5
- Multidimensional Family Therapy (MDFT)
In both trials, assessments were conducted at intake and at 3-, 6-, 9-, and 12-month follow-ups. Findings from these trials included:
- Across both Trial 2 sites, the average cost per day of abstinence for A-CRA participants was $6.62, compared with $9.00 for MET/CBT5 participants and $10.38 for MDFT participants. Only one of the two sites in Trial 2 showed a significant difference between A-CRA and MDFT ($8.09 vs. $12.79, p < .05), a difference that reflects a small effect size (Cohen's f = 0.23).
- In Trial 2, the average cost per participant in recovery at the 12-month follow-up was $7,615 and varied significantly by condition after controlling for site and recovery status in the month prior to intake (p < .05). On average, the cost of an A-CRA participant in recovery at the 12-month follow-up was $4,460, compared with $6,611 for an MET/CBT5 participant and $11,775 for an MDFT participant. This difference was associated with a large effect size (Cohen's f = 0.78).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
| Outcome 4: Linkage to and participation in continuing care services |
|
Description of Measures
|
Linkage to and participation in continuing care services (i.e., sessions) following residential treatment were evaluated by supplementing the substance use measure of the GAIN with daily service contact logs completed by the case manager. The service contact logs contained fields to record each continuing care service provided, the adolescent and/or family member involved, and the time and location of the service. Daily service contact logs were reviewed by supervisory staff.
|
|
Key Findings
|
In one RCT, adolescents with substance dependence who were preparing for discharge from residential treatment were assigned to receive either usual continuing care (UCC) through a referral to 1 of 12 community outpatient substance abuse clinics or UCC plus Assertive Continuing Care (ACC) with A-CRA, which involved a case manager who provided home visits for 3 months. UCC varied with type of discharge. Adolescents discharged "against staff advice" or "at staff request" received only a letter with information on where to go for further treatment. Adolescents discharged "as planned" received a continuing care appointment with a case manager that was typically scheduled within 2 weeks of discharge. Follow-up interviews occurred 3, 6, and 9 months after discharge. Findings from this study included:
- In the 3 months after residential discharge, ACC participants were more likely than UCC participants to link to continuing care services (94% vs. 54%, p < .001). This difference was associated with a large effect size (Cohen's d = 1.07).
- On average, ACC participants received more days of continuing care sessions than UCC participants (p < .001). This group difference was associated with a medium effect size (Cohen's d = 0.64).
- The median number of continuing care sessions attended was higher for A-CRA participants than for UCC participants (15 vs. 2 sessions, p < .001). This group difference was associated with a large effect size (Cohen's d = 0.90).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 5: Substance use |
|
Description of Measures
|
The quantity and frequency of drug and alcohol use in the past 90 days were measured using Form 90.
|
|
Key Findings
|
In one RCT, street-living, homeless youth from a drop-in center received either usual care or usual care plus A-CRA. Usual care was defined as having a place to rest during the day; access to food, clothing, and showers; and case management services that linked youth with community resources at their request. The intervention condition included 12 A-CRA therapy sessions and 4 HIV education/skills practice sessions. In this study, from baseline to the 6-month follow-up:
- A-CRA participants had a greater decrease in overall reported substance use than participants who received usual care (37% vs. 17%, p < .05). This group difference was associated with a small effect size (Cohen's d = 0.35).
- A-CRA participants also had a decrease in the reported frequency of substance use. The percentage of days participants reported having used substances decreased from 67% to 43% (p < .001), a difference that reflects a large effect size (Cohen's d = 1.00).
- Frequency of substance use also declined among usual care participants from 60% to 50% of days (p < .05), a difference that reflects a small effect size (Cohen's d = 0.41).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 6: Social stability |
|
Description of Measures
|
Social stability was defined by the percentage of days spent in work, in education, being housed, and being seen for medical care in the past 90 days and was measured using Form 90.
|
|
Key Findings
|
In one RCT, street-living, homeless youth from a drop-in center received either usual care or usual care plus A-CRA. Usual care consisted of having a place to rest during the day; access to food, clothing, and showers; and case management services that linked youth with community resources at their request. The intervention condition included 12 A-CRA therapy sessions and 4 HIV education/skills practice sessions. In this study, from baseline to the 6-month follow-up:
- The percentage of reported days with social stability increased for both A-CRA and usual care participants, although the increase was significant only for the A-CRA group (40% to 68%, p < .001). The increase for A-CRA participants was associated with a medium effect size (Cohen's d = 0.63).
- Overall, A-CRA participants had a greater increase in reported social stability than usual care participants (58% vs. 13%, p = .05), and this group difference was associated with a small effect size (Cohen's d = 0.35).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 7: Depression symptoms |
|
Description of Measures
|
Depression symptoms were evaluated using the Beck Depression Inventory--II (BDI-II), a 21-item self-report instrument for measuring past-week depressive symptoms among youth (ages 13 and above) and adults. Total scores vary from 0 to 63 and indicate whether depression is minimal (0-13), mild (14-19), moderate (20-28), or severe (29-63).
|
|
Key Findings
|
In one RCT, street-living, homeless youth from a drop-in center received either usual care or usual care plus A-CRA. Usual care consisted of having a place to rest during the day; access to food, clothing, and showers; and case management services that linked youth with community resources at their request. The intervention condition included 12 A-CRA therapy sessions and 4 HIV education/skills practice sessions. In this study, from baseline to the 6-month follow-up:
- Decreases in reported depression symptoms occurred with both A-CRA (p < .001) and usual care (p < .05) participants. These decreases were associated with a large effect size for the A-CRA group (Cohen's d = 0.94) and a small effect size for the usual care group (Cohen's d = 0.41). The decreases in reported depression symptoms were greater for A-CRA participants than for usual care participants (40% vs. 23%, p < .05), a group difference associated with a small effect size (Cohen's d = 0.35).
- When the entire youth sample was split into two age groups (14-19 and 20-22 years), the decrease in reported depression symptoms was evident in both the younger (p = .001) and older (p < .001) A-CRA participants, with medium and large effect sizes for these decreases (Cohen's d = 0.59 and 0.81), respectively. Among usual care participants, only the younger group showed this decrease (p < .01), which was associated with a small effect size (Cohen's d = 0.46).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 8: Internalized behavior problems |
|
Description of Measures
|
Internalized behavior problems were measured using the internalizing scale of the Youth Self-Report (YSR). This tool is a modified version of the parent-completed Child Behavior Checklist (CBCL), a 120-item scale that measures internalizing (sadness or poor self-esteem), externalizing (verbal or physical aggression), and total behavior problems. Respondents rate how true each item is for themselves now or within the past 6 months using a 3-point response scale (0 = not true, 1 = somewhat or sometimes true, and 2 = very true or often true).
|
|
Key Findings
|
In one RCT, street-living, homeless youth from a drop-in center received either usual care or usual care plus A-CRA. Usual care consisted of having a place to rest during the day; access to food, clothing, and showers; and case management services that linked youth with community resources at their request. The intervention condition included 12 A-CRA therapy sessions and 4 HIV education/skills practice sessions. In this study, from baseline to the 6-month follow-up:
- Decreases in reported internalized behavior problems occurred with both A-CRA (p < .001) and usual care (p < .01) participants. These decreases were associated with a large effect size for the A-CRA group (Cohen's d = 1.09) and a medium effect size for the usual care group (Cohen's d = 0.51). The decreases in reported internalized behavior problems were greater for A-CRA participants than for usual care participants (p < .05), a group difference associated with a small effect size (Cohen's d = 0.41).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
|
83% Male
17% Female
|
61% White
30% Black or African American
5% Race/ethnicity unspecified
4% Hispanic or Latino
|
|
Study 2
|
13-17 (Adolescent)
|
71% Male
29% Female
|
73.2% White
18% Black or African American
8.7% Race/ethnicity unspecified
|
|
Study 3
|
13-17 (Adolescent)
18-25 (Young adult)
|
65.6% Male
34.4% Female
|
40.6% White
30% Hispanic or Latino
13.3% American Indian or Alaska Native
12.2% Race/ethnicity unspecified
3.3% Black or African American
0.6% Asian
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Abstinence from substance use
|
4.0
|
3.6
|
3.3
|
3.8
|
3.5
|
3.4
|
3.6
|
|
2: Recovery from substance use
|
3.9
|
3.8
|
3.9
|
3.9
|
3.2
|
3.7
|
3.7
|
|
3: Cost effectiveness
|
3.0
|
2.8
|
3.9
|
3.5
|
3.0
|
3.2
|
3.2
|
|
4: Linkage to and participation in continuing care services
|
3.5
|
3.5
|
2.5
|
3.8
|
3.8
|
3.0
|
3.3
|
|
5: Substance use
|
4.0
|
3.8
|
3.1
|
1.9
|
2.4
|
3.1
|
3.0
|
|
6: Social stability
|
4.0
|
3.8
|
3.1
|
1.9
|
2.4
|
3.1
|
3.0
|
|
7: Depression symptoms
|
3.8
|
3.8
|
3.1
|
1.9
|
2.4
|
3.1
|
3.0
|
|
8: Internalized behavior problems
|
4.0
|
4.0
|
3.1
|
1.9
|
2.4
|
3.1
|
3.1
|
Study Strengths The measures used in the studies had excellent reliability, and the outcomes had good support for face, content, criterion-related, and construct validity. Intervention integrity procedures were strong overall. In two of the three studies, the investigators achieved a complete dataset through excellent follow-up rates and thoroughly addressed the limited data loss in the analysis.
Study Weaknesses Cost outcome measures were restricted to a narrowly defined assessment over less than 4 months and did not take into consideration the differential costs of expert judgments, professional personnel, or region-specific cost of living expenses. Two studies did not completely separate the active intervention and control conditions. There was no documentation of other services received during the follow-up periods. Missing data on some assessments reached 14% in the homeless youth study. Also in this study, the outcome improvements reported by participants often corresponded to the level of positive feedback they provided about the therapists, an association that suggested the need to analyze therapist differences as a potential mediator of the outcome results.
|
|
