Quality of Research
Review Date: November 2009
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Carise, D., Gurel, O., McLellan, A. T., Dugosh, K., & Kendig, C. (2005). Getting patients the services they need using a Computer-Assisted System for Patient Assessment and Referral--CASPAR. Drug and Alcohol Dependence, 80(2), 177-189. 
Supplementary Materials Carise, D., Cornely, W., & Gurel, O. (2002). A successful researcher-practitioner collaboration in substance abuse treatment. Journal of Substance Abuse Treatment, 23(2), 157-162.
Carise, D., & Gurel, O. (2002). Benefits of integrating assessment technology with treatment: The DENS project. In J. L. Sorensen, R. Rawson, J. Guydish, & J. E. Zweben (Eds.), Drug abuse treatment through collaboration: Practice and research partnerships that work (pp. 181-195). Washington, DC: American Psychological Association.
Gurel, O., Carise, D., Kendig, C., & McLellan, A. T. (2005). Developing CASPAR: A Computer-Assisted System for Patient Assessment and Referral. Journal of Substance Abuse Treatment, 28(3), 281-289.
McLellan, A. T., Alterman, A. I., Cacciola, J., Metzger, D., & O'Brien, C. P. (1992). A new measure of substance abuse treatment: Initial studies of the Treatment Services Review. The Journal of Nervous and Mental Disease, 180(2), 101-110.
McLellan, A. T., Grissom, G. R., Zanis, D., Randall, M., Brill, P., & O'Brien, C. P. (1997). Problem-service "matching" in addiction treatment: A prospective study in 4 programs. Archives of General Psychiatry, 54(8), 730-735.
McLellan, A. T., Kushner, H., Metzger, D., Peters, R., Smith, I., Grissom, G., et al. (1992). The Fifth Edition of the Addiction Severity Index. Journal of Substance Abuse Treatment, 9(3), 199-213.
Outcomes
| Outcome 1: Matching of counselor treatment plan to client admission problems |
|
Description of Measures
|
This outcome assessed the match between client problems identified using the Addiction Severity Index (ASI) and the services planned in counselor treatment plans. Problems at admission were assessed using the ASI, a semistructured interview that measures types and severity of problems in seven domains: medical, employment, alcohol use, drug use, legal status, family relations, and psychiatric functioning. For each client, four trained judges blinded to condition assignment individually rated the client's problem status by ASI domain using a 3-point scale from 0 to 2 (0 = no problem, 1 = some problem, and 2 = significant problem).
Counselor-developed treatment plans derived from the ASI were also rated by the four judges. For each client, the judges rated the problems and planned services described in the treatment plan by ASI domain using a similar 3-point scale from 0 to 2 (0 = no mention of a problem; 1 = a mention or comment on the problem but no clear plan, timeframe, or service suggested; and 2 = the problem was clearly identified with a specific plan and/or timeframe for resolution).
For each client, each judge compared the ratings given to the ASI and treatment plan on corresponding domains to determine the level of matching. For an ASI rating of 1, a treatment plan rating of 1 or 2 was a match; for an ASI rating of 2, a treatment plan rating of 2 was a match. ASI domains rated zero were eliminated from the matching calculations since a zero rating meant there was no problem to be addressed.
|
|
Key Findings
|
In a randomized clinical trial, nine community-based outpatient substance abuse treatment programs were randomly assigned to one of two conditions, standard assessment (SA) or enhanced assessment (EA). Counselors in programs assigned to the SA condition received 12 hours of training on use of the Drug Evaluation Network System (DENS) ASI. Counselors in programs assigned to the EA condition received the same DENS ASI training plus an additional 2 hours of training on integrating the DENS ASI assessment and treatment plan with an electronic Resource Guide to link clients to off-site community social and health services. Data were adjusted for site differences in counselor characteristics (i.e., race, level of education) and client characteristics (i.e., gender, race, money spent on drugs in the past 30 days). The percentage of clients whose treatment plans matched problems identified at admission was higher in EA than SA treatment programs across all seven domains: alcohol use (p < .01), drug use (p < .01), psychiatric functioning (p < .01), employment (p < .001), medical (p < .0001), family relations (p < .01), and legal status (p < .01). Effect sizes were large for six of the domains (odds ratio = 7.45 for alcohol use, 13.2 for drug use, 8.32 for psychiatric functioning, 10.66 for employment, 35.93 for medical, and 11.79 for family relations). The effect size for the legal status domain was very small (odds ratio = 1.28).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 2: Matching of specialized services received to client admission problems |
|
Description of Measures
|
This outcome assessed the match between client problems identified using the ASI and the services received as measured by the Treatment Services Review (TSR). Problems at admission were assessed using the ASI, a semistructured interview that measures types and severity of problems in seven domains: medical, employment, alcohol use, drug use, legal status, family relations, and psychiatric functioning. For each client, four trained judges blinded to condition assignment individually rated the client's problem status by ASI domain using a 3-point scale from 0 to 2 (0 = no problem, 1 = some problem, and 2 = significant problem).
The TSR, a 10- to 15-minute structured client interview administered in person or over the telephone, was used to measure the number and types of treatment services received by clients either on site or off site (by referrals) while in substance abuse treatment. The TSR interview questions cover the same domains as the ASI. Within each of these domains, the TSR categorizes services as "general" (e.g., advice or peer counseling received during regular group or individual counseling sessions) or "specialized," focused on a single topic or domain (e.g., vocational counseling, case management, family therapy). For each client, the four judges rated the level of specialized services received in each domain using a similar 3-point scale from 0 to 2 (0 = no specialized services were received, 1 = one specialized service was received, and 2 = two specialized services were received). TSR data were collected for services received during the first 4 weeks of treatment. The analyses were restricted to the specialized on- and off-site services received.
For each client, each judge compared the ratings given to the ASI and TSR on corresponding domains to determine the level of matching. For an ASI rating of 1, a TSR rating of 1 or 2 was a match; for an ASI rating of 2, a TSR rating of 2 was a match. ASI domains rated zero were eliminated from the matching calculations since a zero rating meant there was no problem to be addressed.
|
|
Key Findings
|
In a randomized clinical trial, nine community-based outpatient substance abuse treatment programs were randomly assigned to one of two conditions, standard assessment (SA) or enhanced assessment (EA). Counselors in programs assigned to the SA condition received 12 hours of training on use of the DENS ASI. Counselors in programs assigned to the EA condition received the same DENS ASI training plus an additional 2 hours of training on integrating the DENS ASI assessment and treatment plan with an electronic Resource Guide to link clients to off-site community social and health services. Data were adjusted for site differences in counselor characteristics (i.e., race, level of education) and client characteristics (i.e., gender, race, money spent on drugs in the past 30 days). The percentage of clients who received specialized treatment services that matched problems identified at admission was higher in EA than SA treatment programs in the alcohol use (p < .001), drug use (p < .01), psychiatric functioning (p < .01), and medical (p < .05) domains. Effect sizes were large for three of the domains (odds ratio = 13.98 for alcohol use, 15.74 for drug use, and 11.23 for psychiatric functioning). The effect size for the medical domain was medium (odds ratio = 5.20).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
| Outcome 3: Number of services received |
|
Description of Measures
|
The number of services received was measured by the TSR, a 10- to 15-minute structured client interview administered in person or over the telephone. The TSR collects information on the number and types of services received by clients on site or off site (by referrals) while in substance abuse treatment. TSR interview questions cover services received in seven domains: medical, employment, alcohol use, drug use, legal status, family relations, and psychiatric functioning. Within each of these domains, the TSR categorizes services as "general" (e.g., advice or peer counseling received during group or individual counseling sessions) or "specialized," focused on a single topic or domain (e.g., vocational counseling, case management, family therapy). TSR data were collected for services received during the first 4 weeks of treatment.
|
|
Key Findings
|
In a randomized clinical trial, nine community-based outpatient substance abuse treatment programs were randomly assigned to one of two conditions, standard assessment (SA) or enhanced assessment (EA). Counselors in programs assigned to the SA condition received 12 hours of training on use of the DENS ASI. Counselors in programs assigned to the EA condition received the same DENS ASI training plus an additional 2 hours of training on integrating the DENS ASI assessment and treatment plan with an electronic Resource Guide to link clients to off-site community social and health services. Data were adjusted for site differences in counselor characteristics (i.e., race, level of education) and client characteristics (i.e., gender, race, money spent on drugs in the past 30 days). Among the findings from this study are the following:
- Clients in EA treatment programs averaged more on-site and off-site service sessions (both specialized and general) than clients in SA treatment programs (100 vs. 31, respectively; p < .001); the difference in services received between the two groups was more pronounced for specialized services (p < .001) than general services (p < .01).
- Clients in EA treatment programs received more on-site and off-site specialized service sessions than clients in SA treatment programs in the domains of drug and alcohol use (19 vs. 5; p < .001), psychiatric functioning (25 vs. 3; p < .001), employment (5 vs. 1; p < .01), and medical (15 vs. 3; p < .05).
- More off-site service sessions (both specialized and general) were received by clients in EA than SA treatment programs (20 vs. 6, respectively; p < .001); these differences reflected more specialized services received by EA than SA program clients (15 vs. 5, respectively; p < .01).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 4: Treatment retention and completion |
|
Description of Measures
|
Treatment retention was measured as the percentage of clients who left treatment against medical advice, and treatment completion was measured as the percentage of clients who completed the planned duration and activities of the treatment program. Client data were gathered from chart records.
|
|
Key Findings
|
In a randomized clinical trial, nine community-based outpatient substance abuse treatment programs were randomly assigned to one of two conditions, standard assessment (SA) or enhanced assessment (EA). Counselors in programs assigned to the SA condition received 12 hours of training on use of the DENS ASI. Counselors in programs assigned to the EA condition received the same DENS ASI training plus an additional 2 hours of training on integrating the DENS ASI assessment and treatment plan with an electronic Resource Guide to link clients to off-site community social and health services. Data were adjusted for site differences in counselor characteristics (i.e., race, level of education) and client characteristics (i.e., gender, race, money spent on drugs in the past 30 days). Compared with treatment programs in the SA condition, programs in the EA condition had a lower percentage of clients that left treatment against medical advice (58% vs. 34%, respectively; p < .05) and a higher percentage of clients that completed treatment (24% vs. 53%, respectively; p < .01).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
|
63% Male
37% Female
|
71% Black or African American
19% White
9% Hispanic or Latino
1% American Indian or Alaska Native
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Matching of counselor treatment plan to client admission problems
|
3.5
|
3.5
|
3.5
|
3.5
|
2.5
|
3.5
|
3.3
|
|
2: Matching of specialized services received to client admission problems
|
3.5
|
3.0
|
3.0
|
3.0
|
2.5
|
3.5
|
3.1
|
|
3: Number of services received
|
3.0
|
3.0
|
3.0
|
3.0
|
2.5
|
3.5
|
3.0
|
|
4: Treatment retention and completion
|
3.5
|
3.5
|
3.0
|
3.0
|
2.5
|
3.5
|
3.2
|
Study Strengths The ASI and TSR are widely used instruments in the field and have strong psychometric properties. The study employed excellent use of judges blinded to condition assignment to determine the match between client admission problems and treatment plan and between client admission problems and services received. The algorithm used to determine the match between admission problems and services received appeared reasonable, and the validity of the approach has been thoroughly evaluated. The judges reported nearly 100% agreement on problem and service coding. Counselors in both treatment conditions participated in extensive ASI competency training to ensure the instrument would be administered with fidelity. The statistical approach thoroughly addressed program and staff attrition. The sample size was adequate, randomization at the level of the treatment program was appropriate, and the modeling approach with covariates was comprehensive.
Study Weaknesses The absence of any convergent measure for the TSR other than medical chart data weakens the instrument's psychometric properties. Minimal description was provided on the process followed by blinded judges to determine the match between client admission problems and treatment plan ratings. Only 10 of 20 programs approached about the study agreed to participate (with 1 dropping out immediately after training), suggesting some self-selection bias. The high staff turnover rate was disruptive to the study. Baseline counselor differences were not measured prior to ASI training.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
ASI Manual
|
Free
|
Yes
|
|
DENS ASI Software Suite
|
$26
|
Yes
|
|
CASPAR-C Resource Guide Software and Resource Guide Manual on CD-ROM
|
$500
|
Yes
|
|
Half-day, off-site training: Using the CASPAR-C Software
|
$2,700-$3,000 depending on the number of participants
|
No
|
|
Half-day, on-site training: Using the CASPAR-C Software
|
$3,900-$5,600 depending on the number of participants, plus travel expenses
|
No
|
|
Half-day, off-site training: Building a Resource Guide
|
$2,700-$3,000 depending on the number of participants
|
No
|
|
Half-day, on-site training: Building a Resource Guide
|
$3,900-$5,600 depending on the number of participants, plus travel expenses
|
No
|
|
1-day, off-site Combination Training
|
$3,000-$3,700 depending on the number of participants
|
No
|
|
1-day, on-site Combination Training
|
$4,300-$6,200 depending on the number of participants, plus travel expenses
|
No
|
|
2-day, off-site ASI Training
|
$3,500-$4,900 depending on the number of participants
|
No
|
|
2-day, on-site ASI Training
|
$4,800-$7,400 depending on the number of participants, plus travel expenses
|
No
|
|
2-day, off-site ASI/DENS Training
|
$3,600-$5,000 depending on the number of participants
|
No
|
|
2-day, on-site ASI/DENS Training
|
$4,900-$7,500 depending on the number of participants, plus travel expenses
|
No
|
|
2-day, off-site ASI/DENS/CASPAR Training
|
$3,700-$5,100 depending on the number of participants
|
No
|
|
2-day, on-site ASI/DENS/CASPAR Training
|
$5,000-$7,600 depending on the number of participants, plus travel expenses
|
No
|
|
Technical assistance and consultation
|
$150 per hour
|
No
|
|
ASI Coding Certification
|
$175
|
No
|
|
ASI Interviewing Certification
|
$150 per hour, 2-4 hours to complete per interviewer
|
No
|
|
