Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Bernstein, J., Bernstein, E., Tassiopoulos, K., Heeren, T., Levenson, S., & Hingson, R. (2005). Brief motivational intervention at a clinic visit reduces cocaine and heroin use. Drug and Alcohol Dependence, 77(1), 49-59.  Study 2Academic ED SBIRT Research Collaborative. (2007). An evidence-based alcohol screening, brief intervention and referral to treatment (SBIRT) curriculum for emergency department (ED) providers improves skills and utilization. Substance Abuse, 28(4), 79-92. (Published simultaneously in Alcohol/drug screening and brief intervention: Advances in evidence-based practice, pp. 79-92, by R. Saitz & M. Galanter, Eds., 2007, New York, NY: Haworth Medical Press)
Academic ED SBIRT Research Collaborative. (2007). The impact of screening, brief intervention, and referral for treatment on emergency department patients' alcohol use. Annals of Emergency Medicine, 50(6), 699-710. Study 3Bernstein, E., Edwards, E., Dorfman, D., Heeren, T., Bliss, C., & Bernstein, J. (2009). Screening and brief intervention to reduce marijuana use among youth and young adults in a pediatric emergency department. Academic Emergency Medicine, 16(11), 1174-1185.
Supplementary Materials Bernstein, E., & Bernstein, J. (2009). Evolution of an emergency department-based collaborative intervention for excessive and dependent drinking: From one institution to nationwide dissemination, 1991-2006. In C. J. Cherpitel, G. Borges, N. Giesbrecht, D. Hungerford, M. Peden, V. Poznyak, R. Room, & T. Stockwell (Eds.), Alcohol and injuries: Emergency department studies in an international perspective (pp. 175-180). Geneva, Switzerland: World Health Organization.
Bernstein, E., Bernstein, J., & Levenson, S. (1997). Project ASSERT: An ED-based intervention to increase access to primary care, preventive services, and the substance abuse treatment system. Annals of Emergency Medicine, 30(2), 181-189.
Bernstein, E., Topp, D., Shaw, E., Girard, C., Pressman, K., Woolcock, E., et al. (2009). A preliminary report of knowledge translation: Lessons from taking screening and brief intervention techniques from the research setting into regional systems of care. Academic Emergency Medicine, 16(11), 1225-1233.
Levy, S., Sherritt, L., Harris, S. K., Gates, E. C., Holder, D. W., Kulig, J. W., et al. (2004). Test-retest reliability of adolescents' self-report of substance use. Alcoholism: Clinical and Experimental Research, 28(8), 1236-1241.
Perrine, M. W., Mundt, J. C., Searles, J. S., & Lester, L. S. (1995). Validation of daily self-reported alcohol consumption using interactive voice response (IVR) technology. Journal of Studies on Alcohol, 56(5), 487-490.
Outcomes
| Outcome 1: Cocaine and opiate abstinence |
|
Description of Measures
|
Cocaine and opiate abstinence for the prior 30 days was measured by radioimmune assay of hair samples (30-day window of detection), with confirmation of positive opiate results by gas chromatography-mass spectrometry. Negative screening results indicated 30-day abstinence. Hair samples were obtained at baseline and the 6-month follow-up.
|
|
Key Findings
|
In a randomized clinical trial, adults who presented at urban, teaching hospital-based, walk-in health clinics for routine medical care were screened for cocaine and opiate use by radioimmune assay of hair samples. Those who had a positive screening result, were out of treatment, and agreed to participate were assigned to the intervention group, which received Project ASSERT, or the comparison group, which received a handout and was read a scripted statement--"based on your screening responses, you would benefit from help with your drug use"--by the interventionist, with no discussion. The handout included a list of treatment options and harm reduction information about safe sex and needle exchange. Study results included the following:
- At the 6-month follow-up, the percentage of participants who were abstinent from cocaine use was higher for the intervention group than the comparison group (22.3% vs. 16.9%; p = .045). This difference was associated with a small effect size (odds ratio = 1.51, adjusted for baseline group differences in health insurance and homelessness).
- At the 6-month follow-up, the percentage of participants who were abstinent from opiate use was higher for the intervention group than the comparison group (40.2% vs. 30.6%; p = .050). This difference was associated with a small effect size (odds ratio = 1.57, adjusted for baseline group differences in health insurance and homelessness).
- At the 6-month follow-up, the percentage of participants who were abstinent from both cocaine and opiate use was higher for the intervention group than the comparison group (17.4% vs. 12.8%); however, this group difference was not statistically significant.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.8
(0.0-4.0 scale)
|
| Outcome 2: Alcohol use |
|
Description of Measures
|
Alcohol use was assessed with the 7-item, abridged version of the National Alcohol Screening Day screening form adapted for emergency medicine.
Three questions measured the quantity and frequency of alcohol use and were developed by the National Institute on Alcohol Abuse and Alcoholism (NIAAA):
- "On average, how many days per week do you drink alcohol (beer, wine, liquor)?"
- "On a typical day when you drink, how many drinks do you have?"
- "What is the maximum number of drinks you had on any given occasion during the last month?"
An average number of drinks per week was calculated by multiplying the participant's number of drinking days per week by the number of drinks consumed on a typical drinking day. Participants were considered at-risk drinkers (and eligible for study enrollment) if, by self-report, they exceeded the NIAAA guidelines for low-risk drinking published in Helping Patients Who Drink Too Much: A Clinician's Guide. The NIAAA low-risk limits used were ≤4 drinks per drinking occasion and ≤14 standard drinks per week for men, ≤3 drinks per drinking occasion and ≤7 drinks per week for women, and ≤1 drink per day and ≤7 drinks per week for adults 65 years and older.
Four items, from the CAGE instrument, measured the severity of alcohol use by participants who, per the NIAAA low-risk limits, were identified as being at risk:
- C--"Have you ever felt you should cut down on your drinking?"
- A--"Have people annoyed you by criticizing your drinking?"
- G--"Have you ever felt bad or guilty about your drinking?"
- E--"Eye opener: Have you ever had a drink first thing in the morning to steady your nerves or to get rid of a hangover?"
Participants responded to each question with "yes" (1 point) or "no" (0 points), and points were summed to determine a baseline score. Participants with baseline CAGE scores of >2 were defined as possibly dependent drinkers (high level), and participants with baseline CAGE scores of <2 were considered at risk but unlikely to be dependent drinkers (low level).
The National Alcohol Screening Day screening form was administered in person in the ED (baseline) and by telephone at 3 months after the ED visit (follow-up), using an automated interactive voice response system.
|
|
Key Findings
|
In a multisite clinical trial, adults who presented to 1 of 14 university-based EDs were screened for at-risk drinking (i.e., being over the low-risk limits established by NIAAA). Those who had a positive screening result at baseline and agreed to participate were assigned sequentially to the intervention group, which received Project ASSERT, or the comparison group, which received a written list of local referral resources. Study results included the following:
- From baseline to the 3-month follow-up, intervention group participants reported a greater decrease in drinks per week than comparison group participants (22.5 to 14.1 vs. 22.9 to 17.4; p < .05) and a greater decrease in maximum drinks per drinking occasion (8.6 to 6.5 vs. 8.4 to 7.2; p < .05).
- At the 3-month follow-up, the percentage of participants no longer exceeding the NIAAA low-risk drinking limits was higher for the intervention group than the comparison group (27.8% vs. 18.4%; p < .05).
- At the 3-month follow-up, the percentage of participants with baseline CAGE scores of <2 who no longer exceeded the NIAAA low-risk limits was higher for the intervention group than the comparison group (37.2% vs. 18.6%; p < .05).
- At the 3-month follow-up, for participants with baseline CAGE scores of <2, those in the intervention group reported 1.2 fewer drinks per maximum drinking occasion in the prior 30 days (p = .05) and 2.8 fewer drinks per week in the prior 30 days (p = .05) than those in the comparison group.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
| Outcome 3: Marijuana abstinence |
|
Description of Measures
|
Marijuana abstinence was assessed with the Timeline Followback method, which uses a calendar format with holidays and special events to trigger retrospective recall of daily substance use across a specified time period. Participants used this tool to self-report marijuana consumption in the prior 30-day period at enrollment (baseline) and at the 3- and 12-month follow-ups. Marijuana consumption was defined as smoke in any form, including passive smoke or a toke (i.e., a puff on a marijuana cigarette or pipe), and consumables (e.g., baked goods containing cannabis). Marijuana abstinence was defined as zero marijuana consumption reported in the prior 30 days.
|
|
Key Findings
|
In a randomized clinical study, youth and young adults (aged 14-21 years) who presented to an urban, university-based pediatric ED for acute care were screened for marijuana and alcohol use. Those who had a positive screening result for marijuana use (smoked marijuana ≥3 days in the prior 30 days) but a negative result for alcohol use and agreed to participate in the study were assigned to one of three groups: the intervention group, which received Project ASSERT; an assessed comparison group; or a nonassessed comparison group. The assessed comparison group (like the intervention group) received a battery of assessment instruments, a handout with information on the risks of marijuana use and a listing of adolescent treatment and community resources, and 3- and 12-month follow-up appointments. The nonassessed comparison group received the handout and a 12-month follow-up appointment.
At the 12-month follow-up, the percentage of participants who abstained from marijuana was higher for the intervention group than the assessed comparison group (45% vs. 22%; p < .014). This group difference was associated with a medium effect size (odds ratio = 2.89); participants in the intervention group were almost three times as likely as those in the assessed comparison group to be abstinent at the 12-month follow-up (p < .014).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 4: Marijuana use |
|
Description of Measures
|
Marijuana use was assessed with the Timeline Followback method, which uses a calendar format with holidays and special events to trigger retrospective recall of daily substance use across a specified time period. Participants used this tool to self-report the number of marijuana use days and the number of days "getting high" in the prior 30-day period at enrollment (baseline) and at the 3- and 12-month follow-ups.
|
|
Key Findings
|
In a randomized clinical study, youth and young adults (aged 14-21 years) who presented to an urban, university-based pediatric ED for acute care were screened for marijuana and alcohol use. Those who had a positive screening result for marijuana use (smoked marijuana ≥3 days in the prior 30 days) but a negative result for alcohol use and agreed to participate in the study were assigned to one of three groups: the intervention group, which received Project ASSERT; an assessed comparison group; or a nonassessed comparison group. The assessed comparison group (like the intervention group) received a battery of assessment instruments, a handout with information on the risks of marijuana use and a listing of adolescent treatment and community resources, and 3- and 12-month follow-up appointments. The nonassessed comparison group received the handout and a 12-month follow-up appointment. Study results included the following:
- At the 3-month follow-up, the number of reported marijuana use days in the prior 30 days was 4.2 days fewer for participants in the intervention group compared with those in the assessed comparison group (p = .039), after controlling for baseline marijuana use.
- At the 12-month follow-up, the number of reported marijuana use days in the prior 30 days was 5.3 days fewer for participants in the intervention group compared with those in the assessed comparison group (p = .024), after controlling for baseline marijuana use.
- At the 12-month follow-up, among participants who reported smoking marijuana at least once in the prior 30 days, those in the intervention group were less likely to report "getting high" in the prior 30-day period compared with those in the assessed comparison group (adjusted odds ratio = 0.39; p < .027), after controlling for baseline marijuana use. This group difference was associated with a very small effect size.
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
26-55 (Adult)
|
71% Male
29% Female
|
62% Black or African American
23% Hispanic or Latino
14% White
1% Race/ethnicity unspecified
|
|
Study 2
|
26-55 (Adult)
|
68% Male
32% Female
|
39% White
37% Black or African American
20% Hispanic or Latino
4% Race/ethnicity unspecified
|
|
Study 3
|
13-17 (Adolescent)
18-25 (Young adult)
|
62.4% Female
37.6% Male
|
79% Black or African American
15.7% Hispanic or Latino
4.8% White
0.5% Asian
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Cocaine and opiate abstinence
|
4.0
|
4.0
|
3.5
|
4.0
|
3.5
|
4.0
|
3.8
|
|
2: Alcohol use
|
3.0
|
3.0
|
3.5
|
2.8
|
3.5
|
3.5
|
3.2
|
|
3: Marijuana abstinence
|
3.5
|
3.5
|
3.5
|
3.5
|
2.5
|
3.5
|
3.3
|
|
4: Marijuana use
|
3.5
|
3.5
|
3.5
|
3.5
|
2.5
|
3.5
|
3.3
|
Study Strengths Hair radioimmune assay, which has a 30-day window of detection, is a reliable and well-validated biochemical marker for cocaine and opiate use, and the Timeline Followback method for retrospective recall of marijuana use in the prior 30-day period has strong psychometric properties in adolescent samples. Intervention fidelity was very strong in two studies in which the intervention was delivered by peer educators who had intensive, manual-driven training that included role-plays and supervised interviews with participants; the same two studies also included a strict role designation of the peer educators as interventionists or evaluators to prevent crossover contamination. In one study, sessions with peer educators as interventionists were audiotaped and reviewed weekly by the project coordinator and investigators using a standardized instrument to assess intervention fidelity. In a third study, which used ED staff as the interventionists, intervention experts provided a 2-hour, structured training curriculum at each of the 14 participating sites, as well as an online version of the training curriculum for use by ED staff who could not attend the on-site training. Attrition was low at follow-up in one study (18%), with very little missing data. Two of the three studies used random assignment to control for many potential confounds, and one randomized study used a sophisticated blinding procedure and included a nonassessed comparison group in the study design to evaluate assessment reactivity and regression to the mean effects on outcomes at the 12-month follow-up. The statistical modeling of the outcome data in the quasi-experimental study appropriately accounted for the clustered sampling design (i.e., participants nested within sites) and for potential confounding effects of the sequential group assignment. All three studies used sophisticated modeling of the outcome data; the two studies with sizable attrition and missing data either imputed missing values on demographic control variables by participant site, that is, race, work status, education, and health status (one study), or used multiple analyses to examine the impact of attrition and missing data on the outcomes (the other study). These same two studies approximated an intent-to-treat approach. All three studies incorporated outcome mediator analyses where the subgroup sample size was adequate for statistical power.
Study Weaknesses There was no independent validation of self-reported drinking behavior, and although self-report is recognized in the field as having strong psychometric properties, the reliability and validity of drinking behavior reports are not based on studies using a motivational interviewing counseling intervention where demand characteristics in the form of social desirability may be an issue. The attrition rate at the follow-up assessments was high by group in the two ED-based studies (38% and 39% for the intervention and comparison groups, respectively, at 3 months in one study and 31%, 22%, and 34% for the intervention, assessed comparison, and nonassessed comparison groups, respectively, at 12 months in the second study), despite being incorporated into the statistical modeling of the outcome data. Two of the three studies did not use a validated fidelity instrument. In one of the two studies using peer educators as interventionists, the booster follow-up telephone contact reached only 31% of the participants in the intervention group, which may have introduced a differential effect on the outcomes. The marijuana abstinence and marijuana use outcomes are not orthogonal; participants who were abstinent at the 12-month follow-up also reported fewer days of marijuana use, and it does not appear that the investigators removed the abstinent participants from the analysis of marijuana use.
|
|
