Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1De Leon, G., Sacks, S., Staines, G., & McKendrick, K. (2000). Modified therapeutic community for homeless mentally ill chemical abusers: Treatment outcomes. American Journal of Drug and Alcohol Abuse, 26(3), 461-480. 
French, M. T., McCollister, K. E., Sacks, S., McKendrick, K., & De Leon, G. (2002). Benefit-cost analysis of a modified therapeutic community for mentally ill chemical abusers. Evaluation and Program Planning, 25, 137-148. Study 2Sacks, S., Sacks, J. Y., McKendrick, K., Banks, S., & Stommel, J. (2004). Modified TC for MICA offenders: Crime outcomes. Behavioral Sciences and the Law, 22(4), 477-501.
Sullivan, C. J., McKendrick, K., Sacks, S., & Banks, S. (2007). Modified therapeutic community treatment for offenders with MICA disorders: Substance use outcomes. American Journal of Drug and Alcohol Abuse, 33(6), 823-832. Study 3Sacks, S., McKendrick, K., Sacks, J. Y., Banks, S., & Harle, M. (2008). Enhanced outpatient treatment for co-occurring disorders: Main outcomes. Journal of Substance Abuse Treatment, 34(1), 48-60.
Supplementary Materials Sacks, S. (2007). CTCR interview protocols--Baseline & follow-up. Unpublished manuscript.
Sacks, S., Banks, S., McKendrick, K., & Sacks, J. Y. (2008). Modified therapeutic community for co-occurring disorders: A summary of four studies. Journal of Substance Abuse Treatment, 34(1), 112-122.
Sacks, S., Banks, S. M., McKendrick, K., Sacks, J. Y., & Cleland, C. M. (2007). Meta-analysis for single investigators and research teams. Manuscript submitted for publication.
Sacks, S., Sacks, J. Y., & De Leon, G. (1999). Treatment for MICAs: Design and implementation of the modified TC. Journal of Psychoactive Drugs, 31(1), 19-30.
Sacks, S., Sacks, J. Y., & Stommel, J. (2003). Modified therapeutic community program for inmates with mental illness and chemical abuse disorders. Corrections Today, 65(6), 90-99.
Outcomes
| Outcome 1: Substance use |
|
Description of Measures
|
In one study, substance use was evaluated using three self-report measures: frequency of alcohol intoxication, number of different types of illegal drugs used (0-17), and highest frequency of illegal drug use on a scale from 0 (none) to 8 (more than once daily). All three reports were obtained at baseline, at 12 months after baseline, and at each client's last follow-up point (long-term follow-up), which was more than 24 months after baseline, on average.
In another study, substance use was evaluated using six self-report measures across the first 12 months after release from prison: any illegal drug use, alcohol used to intoxication, any substance use (combined measure of drug use and alcohol used to intoxication), frequency of alcohol used to intoxication, drug use severity, and days until substance use (relapse).
|
|
Key Findings
|
Homeless clients with co-occurring disorders were sequentially assigned to a low-intensity MTC condition, a moderate-intensity MTC condition, or usual care. Compared with the high-intensity MTC condition, the low-intensity condition placed fewer demands on clients (e.g., clients had more freedom to leave the facility) and provided clients with more staff assistance. Usual care incorporated a variety of treatment and nontreatment options that were typically less specific to the needs of homeless clients with co-occurring disorders, not as well organized, and less cohesive in perspective or approach relative to the MTC conditions.
- At the 12-month follow-up, low-intensity MTC clients reported significantly less frequent alcohol intoxication (p < .05), fewer types of illegal drugs used (p < .01), and less frequent drug use (p < .01) than usual care clients. These differences remained at the long-term follow-up (p < .05, p < .05, and p < .01, respectively).
- At the 12-month follow-up, low-intensity MTC clients reported significantly fewer types of illegal drugs used (p < .01) and less frequent illegal drug use (p < .01) than moderate-intensity MTC clients. These differences remained at the long-term follow-up (p < .05 and p < .01, respectively), at which time low-intensity MTC clients also reported less frequent alcohol intoxication than moderate-intensity MTC clients (p < .05). In addition, more low-intensity than moderate-intensity MTC clients were retained in treatment for 12 months (56% vs. 34%, p < .002).
- At the 12-month follow-up, MTC clients who received 12 months of treatment (treatment completers) in either the low-intensity (p < .01) or the moderate-intensity (p < .05) condition reported less substance use than clients who received usual care for at least 9 months.
- At the long-term follow-up, clients who received at least 12 months of treatment (treatment completers) in both MTC conditions reported less frequent alcohol intoxication (p < .01), fewer types of illegal drugs used (p < .01), and less frequent illegal drug use (p < .001) than clients who received usual care for at least 9 months.
In a randomized controlled trial (RCT), male prison inmates with co-occurring disorders were assigned either to a 12-15 month in-prison MTC program modified for a prison population, followed by a voluntary, 6-month aftercare MTC program in a community corrections facility after release, or to a mental health treatment condition of variable duration (11 months, on average). Adaptations to MTC included a programmatic emphasis on criminal thinking and behavior, adjustments to comply with security guidelines, inclusion of security personnel on the treatment team, psychoeducational classes, and cognitive behavioral protocols. The control condition consisted of psychiatric medication services, weekly individual therapy and counseling, and mandated cognitive behavioral and anger management group therapy.
- At the 12-month postrelease follow-up, both groups showed improvement in substance use, illegal drug use, and alcohol used to intoxication. However, relative to control group participants, a significantly smaller percentage of MTC participants reported substance use (56% vs. 31%, p < .01), illegal drug use (44% vs. 25%, p < .05), and alcohol used to intoxication (39% vs. 21%, p < .05).
- At the 12-month postrelease follow-up, compared with MTC participants, control group participants were nearly three times as likely to report substance use and alcohol used to intoxication (odds ratio = 2.94) and more than twice as likely to report illegal drug use (odds ratio = 2.33). The effect sizes were medium and small, respectively.
- On average, MTC participants relapsed later than control group participants (3.7 months vs. 2.6 months, p < .05).
- At the 12-month postrelease follow-up, MTC participants had greater decreases in reported severity of drug use (82% vs. 64%, p < .05) and alcohol used to intoxication (63% and 28%, p < .05) relative to control group participants.
- Among clients with a history of polydrug use, MTC participants had larger reductions in reported substance use (odds ratio = 4.00), illegal drug use (odds ratio = 2.63), and alcohol used to intoxication (odds ratio = 3.45) than control group participants at the 12-month postrelease follow-up. These effect sizes ranged from small to medium.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.7
(0.0-4.0 scale)
|
| Outcome 2: Criminal behavior |
|
Description of Measures
|
In one study, criminal behavior was measured by two self-report items: number of different types of crimes committed (0-16) and total number of crimes committed for each type reported on a scale from 0 (none) to 9 (more than 500). Self-reports of criminal behavior were obtained at baseline, at 12 months after baseline, and at each client's last follow-up point (long-term follow-up), which was more than 24 months after baseline, on average.
In another study, criminal behavior was measured by the following three self-report items across the first 12 months after release from prison: reincarceration, number of new illegal activities (0-17), and drug/alcohol-related offenses. Self-reports were cross-checked against department of correction records.
|
|
Key Findings
|
Homeless clients with co-occurring disorders were sequentially assigned to a low-intensity MTC condition, a moderate-intensity MTC condition, or usual care. Compared with the high-intensity MTC condition, the low-intensity condition placed fewer demands on clients (e.g., clients had more freedom to leave the facility) and provided clients with more staff assistance. Usual care incorporated a variety of treatment and nontreatment options that were typically less specific to the needs of homeless clients with co-occurring disorders, not as well organized, and less cohesive in perspective or approach relative to the MTC conditions.
- At the 12-month follow-up, clients had a decrease in reported crimes committed and crime types regardless of treatment condition (p < .01). However, low-intensity MTC clients reported fewer crimes committed than moderate-intensity clients (p < .04).
- At the long-term follow-up, low- and moderate-intensity MTC clients reported fewer crimes committed (p < .001 and p < .05, respectively) and fewer crime types (p < .001 and p < .05, respectively) than usual care clients.
- At the 12-month follow-up, MTC clients who received at least 12 months of residential treatment (treatment completers) in either the low-intensity (p < .01) or moderate-intensity (p < .05) conditions reported fewer crimes committed and fewer crime types than clients who received usual care for at least 9 months. This difference continued to the long-term follow-up (p < .001).
In an RCT, male prison inmates with co-occurring disorders were assigned either to a 12-15 month in-prison MTC program modified for a prison population, followed by a voluntary, 6-month aftercare MTC program in a community corrections facility after release, or to a mental health treatment condition of variable duration (11 months, on average). Adaptations to MTC included a programmatic emphasis on criminal thinking and behavior, adjustments to comply with security guidelines, inclusion of security personnel on the treatment team, psychoeducational classes, and cognitive behavioral protocols. The control condition consisted of psychiatric medication services, weekly individual therapy and counseling, and mandated cognitive behavioral and anger management group therapy.
- At the 12-month postrelease follow-up, MTC participants had significantly lower reincarceration rates than individuals in the control condition (9% vs. 33%, p < .01), a difference that reflects a medium effect size (odds ratio = 3.85). MTC clients who chose to participate in the aftercare program had an even lower reincarceration rate than control group participants (5% vs. 33%, p < .02), a difference that reflects a large effect size (odds ratio = 7.69).
- Time in treatment across any of the three conditions was a significant predictor of both reincarceration and criminal activity at the 12-month postrelease follow-up (p < .01). The average time to reincarceration was longest for MTC clients who participated in the aftercare program (170 days) and shortest for control group participants (108 days).
- Compared with control group participants, MTC participants who participated in the aftercare program had significantly lower rates of criminal activity in general (67% vs. 42%, p < .05) and lower rates of criminal activity related to alcohol and drug use (58% vs. 30%, p < .03) at the 12-month postrelease follow-up. These findings reflect a small effect size (odds ratio = 2.33 and 2.78, respectively).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 3: Psychological problems |
|
Description of Measures
|
In one study, psychological problems (depression and anxiety symptoms) were measured using the Beck Depression Inventory (BDI) and the Short Form of the Taylor Manifest Anxiety Scale. The BDI is a 21-item self-report instrument that measures past-week depressive symptoms. Total scores vary from 0 to 63 and indicate whether depression is minimal (0-13), mild (14-19), moderate (20-28), or severe (29-63). The Short Form of the Taylor Manifest Anxiety Scale is a 20-item, true/false, self-report questionnaire measuring past-week anxiety symptoms. Self-reports were obtained at baseline, at 12 months after baseline, and at each client's last follow-up point (long-term follow-up), which was more than 24 months after baseline, on average.
In another study, psychological problems were measured using the Global Appraisal of Individual Needs (GAIN) at baseline and the 12-month follow-up. The GAIN is a standardized, semistructured interview with eight main sections (background, substance use, physical health, risk behaviors, mental health, environment, legal, and vocational) that is designed to support the diagnosis, placement, and outcome monitoring of patients and the economic analysis of an intervention.
|
|
Key Findings
|
Homeless clients with co-occurring disorders were sequentially assigned to a low-intensity MTC condition, a moderate-intensity MTC condition, or usual care. Compared with the high-intensity MTC condition, the low-intensity condition placed fewer demands on clients (e.g., clients had more freedom to leave the facility) and provided clients with more staff assistance. Usual care incorporated a variety of treatment and nontreatment options that were typically less specific to the needs of homeless clients with co-occurring disorders, not as well organized, and less cohesive in perspective or approach relative to the MTC conditions.
- At the 12-month follow-up, low-intensity MTC clients reported fewer depression symptoms than moderate-intensity MTC clients (p < .02).
- At the long-term follow-up, low-intensity MTC clients reported fewer depression symptoms (p < .001) and fewer anxiety symptoms (p < .03) than clients who received usual care.
- At the 12-month follow-up, clients who received 12 months of treatment (treatment completers) in both MTC conditions reported fewer depression and anxiety symptoms than clients who received usual care for at least 9 months (p < .05).
In an RCT, clients with co-occurring disorders who were admitted to an outpatient substance abuse day treatment program were assigned to one of two intensive conditions: MTC modified for day treatment or usual care. Both conditions consisted of 3 hours of treatment per day, 3 days per week. The modified MTC condition incorporated community-enhancing meetings for dual recovery taken from the residential MTC model and added a psychoeducational seminar, trauma-informed addictions treatment, and case management. Usual care was a basic day treatment program that provided individual as well as group therapy and counseling that focused on substance use and relapse prevention.
- At the 12-month follow-up, MTC clients had greater decreases in reported emotional problems (p = .04) and any emotional or psychological problems (p < .001) than usual care clients.
|
|
Studies Measuring Outcome
|
Study 1, Study 3
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 4: Employment |
|
Description of Measures
|
Employment was evaluated using one self-report measure. Response options were 0 (none), 1 (part-time irregular or odd jobs), 2 (part-time regular), and 3 (full-time). Self-reports were obtained at baseline, at 12 months after baseline, and at each client's last follow-up point (long-term follow-up), which was more than 24 months after baseline, on average.
|
|
Key Findings
|
Homeless clients with co-occurring disorders were sequentially assigned to a low-intensity MTC condition, a moderate-intensity MTC condition, or usual care. Compared with the high-intensity MTC condition, the low-intensity condition placed fewer demands on clients (e.g., clients had more freedom to leave the facility) and provided clients with more staff assistance. Usual care incorporated a variety of treatment and nontreatment options that were typically less specific to the needs of homeless clients with co-occurring disorders, not as well organized, and less cohesive in perspective or approach relative to the MTC conditions.
- At the 12-month follow-up, clients in both MTC conditions reported increased employment relative to usual care clients (p < .001). This difference remained at the long-term follow-up (p < .001 for low intensity and p < .01 for moderate intensity).
- At the 12-month follow-up, MTC clients who received at least 12 months of treatment (treatment completers) in both MTC conditions had a greater increase in reported employment than clients who received usual care for at least 9 months (p < .001). This finding remained at the long-term follow-up (p < .001).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 5: Economic benefit |
|
Description of Measures
|
Economic benefit was measured as the average incremental financial benefit over the cost of each condition, the net financial benefit over the cost of each condition, and the benefit-to-cost ratio associated with each condition, calculated in 1994 dollars. Financial benefits were evaluated as the estimated cost savings to society expected to accrue from self-reported declines in criminal activity, increased productivity (employment earnings), and decreased health care utilization occurring from 12 months before to 12 months after admission (baseline). Monetary conversion factors (unit cost estimates) were applied to changes in criminal activity, employment earnings, and health care utilization. The economic benefits of treatment were defined as the dollar value associated with changes in each of these outcome domains.
Costs associated with the study conditions were calculated using the Drug Abuse Treatment Cost Analysis Program (DATCAP), an analysis package that estimates both the accounting and economic costs of program implementation, including the full value of all resources, such as donations and subsidies.
|
|
Key Findings
|
Homeless clients with co-occurring disorders were sequentially assigned to a low-intensity MTC condition, a moderate-intensity MTC condition, or usual care. Compared with the high-intensity MTC condition, the low-intensity condition placed fewer demands on clients (e.g., clients had more freedom to leave the facility) and provided clients with more staff assistance. Usual care incorporated a variety of treatment and nontreatment options that were typically less specific to the needs of homeless clients with co-occurring disorders, not as well organized, and less cohesive in perspective or approach relative to the MTC conditions.
- On the basis of increased employment reported by MTC clients compared with usual care clients at the 12-month follow-up, the economic benefit per MTC client relative to usual care client was $720 (p = .01).
- On the basis of decreased health care utilization reported by MTC clients compared with usual care clients at the 12-month follow-up, the economic benefit per MTC client relative to usual care client was $17,613 (p = .01).
- The total average cost savings to society associated with less health care utilization, less criminal activity, and more employment reported by MTC relative to usual care clients was $305,273 (p = .01) per MTC client. When adjusted for outlying MTC clients, this figure decreased to $149,851 but remained significant (p = .01).
- The average incremental economic benefit associated with less health care utilization, less criminal activity, and more employment reported by MTC relative to usual care clients was $273,698 (p = .05) per MTC client. When adjusted for outlying MTC clients, this figure decreased to $105,618 but remained significant (p = .05).
- The net benefit estimate ($253,337) and benefit-to-cost ratio (5:1) associated with a client participating in MTC relative to usual care suggested the economic benefit of MTC, but these findings were not statistically significant.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.4
(0.0-4.0 scale)
|
| Outcome 6: Housing stability |
|
Description of Measures
|
Housing stability was measured using the GAIN, a standardized, semistructured interview with eight main sections (background, substance use, physical health, risk behaviors, mental health environment, legal, and vocational) that is designed to support the diagnosis, placement, and outcome monitoring of patients and the economic analysis of an intervention.
|
|
Key Findings
|
In an RCT, clients with co-occurring disorders who were admitted to an outpatient substance abuse day treatment program were assigned to one of two intensive conditions: MTC modified for day treatment or usual care. Both conditions consisted of 3 hours of treatment per day, 3 days per week. The modified MTC condition incorporated community-enhancing meetings for dual recovery taken from the residential MTC model and added a psychoeducational seminar, trauma-informed addictions treatment, and case management. Usual care was a basic day treatment program that provided individual as well as group therapy and counseling that focused on substance use and relapse prevention.
- At the 12-month follow-up, clients in both conditions had an increase in reported days rent was paid, a decrease in reported time spent in a shelter/emergency housing, and a decrease in reported time in a voluntary housing facility (p < .05). However, MTC clients reported more days of paying rent for housing than usual care clients (p = .04).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
26-55 (Adult)
|
75.4% Male
24.6% Female
|
70.2% Black or African American
18.1% Hispanic or Latino
11.7% White
|
|
Study 2
|
26-55 (Adult)
|
100% Male
|
48.9% White
30.2% Black or African American
16.5% Hispanic or Latino
4.3% Race/ethnicity unspecified
|
|
Study 3
|
26-55 (Adult)
|
57.1% Female
42.9% Male
|
78.8% Black or African American
13.1% White
8.1% Hispanic or Latino
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Substance use
|
2.9
|
2.4
|
2.3
|
2.8
|
3.0
|
3.3
|
2.7
|
|
2: Criminal behavior
|
3.0
|
2.5
|
2.3
|
2.8
|
3.0
|
3.3
|
2.8
|
|
3: Psychological problems
|
3.8
|
3.8
|
2.0
|
2.5
|
3.0
|
2.8
|
3.0
|
|
4: Employment
|
3.0
|
2.5
|
2.0
|
2.5
|
3.0
|
3.5
|
2.8
|
|
5: Economic benefit
|
2.5
|
2.0
|
2.5
|
2.0
|
3.0
|
2.5
|
2.4
|
|
6: Housing stability
|
3.0
|
2.5
|
2.0
|
2.5
|
3.0
|
2.5
|
2.6
|
Study Strengths Standard self-report instruments and measures were used and were augmented with collateral information in some cases (e.g., urine drug screens and department of correction records in the prison study). Self-reports of reincarceration are likely to be highly valid and reliable from the prison study, as they were checked against department of correction records. In the outpatient treatment study, housing was a good index of increased stability and reduced risk for homelessness. Intervention training was carried out by experts who provided ongoing supervision. The DATCAP economic analyses were strong in the homeless study.
Study Weaknesses Reliability for the self-report of substance use and psychological problems was not specifically calculated in these study samples. In the absence of any independent verification, the validity of self-reported crime types and number of crimes committed as true index measures for criminal behavior in the homeless study is questionable. Additionally, there was no attempt to verify self-reported employment (e.g., using pay stubs) in the homeless study. Consequently, the cost-benefit analysis in the homeless study was weakened by the reduced reliability and validity of the behavioral change measures--self-reported criminal behavior and employment--on which it was based. There was no independent verification of intervention fidelity and no fidelity ratings for the usual care control groups in any of these studies.
|
|
