Quality of Research
Review Date: February 2007
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Eggert, L. L., Thompson, E. A., Herting, J. R., & Nicholas, L. J. (1995). Reducing suicide potential among high-risk youth: Tests of a school-based prevention program. Suicide and Life-Threatening Behavior, 25, 276-296.  Study 2Eggert, L. L., Thompson, E. A., Randell, B. P., & Pike, K. C. (2002). Preliminary effects of brief school-based prevention approaches for reducing youth suicide: Risk behaviors, depression, and drug involvement. Journal of Child and Adolescent Psychiatric Nursing, 15, 48-64.
Randell, B. P., Eggert, L. L., & Pike, K. C. (2001). Immediate post intervention effects of two brief youth suicide prevention interventions. Suicide and Life-Threatening Behavior, 31, 41-61.
Thompson, E. A., Eggert, L. L., Randell, B. P., & Pike, K. C. (2001). Evaluation of indicated suicide risk prevention approaches for potential high school dropouts. American Journal of Public Health, 91, 742-752.
Supplementary Materials Eggert, L. L. (1996). Psychosocial approaches in prevention science: Facing the challenge with high risk youth. Communicating Nursing Research, 29, 73-85.
Eggert, L. L. (2000). Science-based prevention approaches to promoting healthy adolescent behaviors. Communicating Nursing Research, 33, 1-13.
Eggert, L. L. (2007). CARE (Care, Assess, Respond, Empower): A suicide risk prevention program. Supporting documentation for rating category: Descriptive dimensions.
Eggert, L. L., Herting, J. R., & Thompson, E. A. (1996). The Drug Involvement Scale for Adolescents (DISA). Journal of Drug Education, 26, 101-130.
Eggert, L. L., & Randell, B. P. (2004). Response to NREPP request for additional information for Coping and Support Training (CAST) prevention program and Counselors-CARE (C-CARE) prevention program.
Eggert, L. L., Thompson, E. A., & Herting, J. R. (1994). A Measure of Adolescent Potential for Suicide (MAPS): Development and preliminary findings. Suicide and Life-Threatening Behavior, 24, 359-381.
Herting, J. R., Eggert, L. L., & Thompson, E. A. (1996). A multidimensional model of adolescent drug involvement. Journal of Research on Adolescence, 6, 325-361.
Powell-Cope, G. M., & Eggert, L. L. (1994). Psychosocial risk and protective factors: Potential high school dropouts versus typical youth. In R. C. Morris (Ed.), Using what we know about at-risk youth: Lessons from the field (pp. 23-51). Lancaster, PA: Technomic Publishing.
Thompson, E. A., & Eggert, L. L. (1999). Using the Suicide Risk Screen to identify suicidal adolescents among potential high school dropouts. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 1506-1514.
Thompson, E. A., Mazza, J. J., Herting, J. R., Randell, B. P., & Eggert, L. L. (2005). The mediating roles of anxiety, depression, and hopelessness on adolescent suicidal behaviors. Suicide and Life-Threatening Behavior, 35, 14-34.
Thompson, E. A., Moody, K. A., & Eggert, L. L. (1994). Discriminating suicide ideation among high-risk youth. Journal of School Health, 64, 361-367.
Walsh, E., Randell, B. P., & Eggert, L. L. (1997). The Measure of Adolescent Potential for Suicide (MAPS): A tool for assessment and crisis intervention. Reaching Today's Youth, 2, 22-29.
Outcomes
| Outcome 1: Suicide risk factors |
|
Description of Measures
|
Suicide risk factors were evaluated in two clinical randomized controlled trials (RCTs). The first study used the Brief Suicide Risk Behavior Scale, a 5-item scale measuring the frequency of suicide thoughts, direct and indirect suicide threats, and self-reported suicide attempts. The second study used 4 items from the High School Questionnaire: Profile of Experiences to measure attitude toward suicide, frequency of suicidal ideation, frequency of direct suicide threats, and number of suicide attempts in the past month.
|
|
Key Findings
|
In the first study, suicide risk factors decreased by at least 25% from baseline to 5- and 10-month follow-up assessments in more than 85% of the youth exposed to the CARE program (p < .001). At baseline, CARE participants scored significantly higher on suicide risk factors relative to a control group of "typical" youth, and this difference was associated with a large effect size (Cohen's d = 1.28). However, at the 10-month follow-up, CARE participants showed reduced suicide risk factors, bringing them closer to the typical youth, and the effect size for this group difference was reduced from large to medium (Cohen's d = 0.57). Male and female CARE participants were equally likely to report decreases in suicide risk behaviors and suicidal ideation over the 10-month follow-up period.
CARE participants' reduction in suicide risk factors was replicated with a specific decrease in suicidal ideation (p < .05) in a second study that compared CARE participants with a usual care control group of suicidal youth over a 9-month follow-up period. The control condition involved procedures simulating those typically used by school personnel to respond to youth who show suicidal behavior. As in the earlier study, the findings did not vary by gender.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 2: Severity of depression symptoms |
|
Description of Measures
|
Two clinical RCTs evaluated the severity of depression symptoms using 5- and 6-item scales adapted from the Center for Epidemiologic Studies Depression Scale (CES-D).
|
|
Key Findings
|
In the first study, the severity of depression symptoms decreased by at least 25% in more than 65% of the CARE participants from baseline to 5- and 10-month follow-up assessments (p < .001). At baseline, CARE participants scored significantly higher on depression symptoms relative to a control group of typical youth, and this difference was associated with a large effect size (Cohen's d = 1.12). However, at the 10-month follow-up, CARE participants showed reduced depression symptoms, bringing them closer to the typical youth, and the effect size for this group difference was reduced from large to small (Cohen's d = 0.31).
The CARE participants' decrease in depressive symptomatology was replicated in a second study that compared CARE participants with a usual care control group of suicidal youth. Improvement was found at both the 10-week (p < .03) and 9-month follow-ups (p < .01). Female CARE participants had higher baseline levels of depression relative to male CARE participants, but post hoc comparisons that controlled for these differences found significant reductions in depression ratings at the 10-week (p < .005) and 9-month follow-ups (p < .001) for males and females.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 3: Feelings of hopelessness |
|
Description of Measures
|
Two clinical RCTs evaluated feelings of hopelessness as a separate dimension because of its unique association with suicidal behaviors. In both studies, hopelessness was evaluated using 3 items that measured feelings of hopelessness about life compared with satisfaction with life. Indicators were extracted from the High School Questionnaire: Profile of Experiences.
|
|
Key Findings
|
In the first study, hopelessness levels decreased by at least 25% in more than 60% of the CARE participants (both females and males) from baseline through 10 months of follow-up (p < .001).
This improvement in feelings of hopelessness was replicated in a second study that compared CARE participants with a usual care control group of suicidal youth. Growth curve analyses showed greater rates of decline in hopelessness for CARE youth compared with controls at 10 weeks (p < .01) and 9 months (p < .05). Regarding mean levels of hopelessness, controlling for baseline levels, CARE participants' reported hopelessness was lower at both the 10-week (p < .004) and 9-month follow-up assessments (p < .05) relative to the usual care participants.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 4: Anxiety |
|
Description of Measures
|
Four items from the High School Questionnaire: Profile of Experiences were used to evaluate anxiety in a clinical RCT. The items indexed physical, emotional, and cognitive signs of anxiety.
|
|
Key Findings
|
CARE participants showed significant rates of decline in anxiety at the 10-week and 9-month follow-up assessments compared with a usual care control group of suicidal youth (p < .001 for both follow-up points). However, gender differences influenced these effects. Female participants in CARE showed decreased anxiety (p < .001) from baseline through 9 months of follow-up. Controlling for baseline levels of anxiety, female CARE participants showed greater improvement on this measure relative to the usual care control participants at 10-week follow-up (p < .001). However, at 9-month follow-up, differences between the CARE participants and those receiving usual care were not statistically significant.
Male participants in CARE showed initial declines in anxiety (p < .001) from baseline to 4-week follow-up that were sustained with little additional change through 9 months of follow-up. However, male CARE participants did not differ significantly from male usual care participants at any of the three follow-up points (4 weeks, 10 weeks, and 9 months).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 5: Anger control problems |
|
Description of Measures
|
Anger control problems were evaluated using indicators from the High School Questionnaire: Profile of Experiences. In the first clinical RCT, anger was assessed using 3 items relating to "tendencies to be easily angered and irritated," "feeling out of control when angry," and "hitting something when upset or angry." The second RCT used these items plus an additional item related to "getting into physical fights with someone."
|
|
Key Findings
|
In the first study, anger control problems decreased by at least 25% in more than 65% of CARE participants (both females and males) from baseline through 10 months of follow-up (p < .019). At baseline, CARE participants scored significantly higher on anger measures relative to a control group of typical youth, and this difference was associated with a large effect size (Cohen's d = 0.91). However, at the 10-month follow-up, CARE participants showed reduced anger control problems, bringing them closer to the typical youth, and the effect size for this group difference was reduced from large to small (Cohen's d = 0.30).
The CARE participants' decreases in anger control problems were replicated in a second study that showed significant rates of decline in anger for CARE youth (p < .01) relative to a usual care control group of suicidal youth; however, a significant gender effect was found. Among females, controlling for baseline levels of anger, the results for CARE participants were not significantly different from those of usual care youth at the 10-week and 9-month follow-ups. Male CARE participants showed a 20% or greater decrease in anger at all three follow-up points (4 weeks, 10 weeks, and 9 months), but these results also were not significantly different from those of the usual care youth.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 6: Drug involvement |
|
Description of Measures
|
Drug involvement was measured in a clinical RCT using three dimensions from the High School Questionnaire: Profile of Experiences: drug use frequency (for alcohol, marijuana, and "hard drugs"), drug use control problems, and adverse drug use consequences.
|
|
Key Findings
|
Three measures of drug use frequency--alcohol use frequency (p < .05), marijuana use frequency (p < .05), and hard drug use frequency (p < .001)--decreased for CARE participants relative to a usual care control group of suicidal youth from baseline to 4-week follow-up. Significant rebound in alcohol use frequency occurred among the CARE participants and controls at the 10-week follow-up assessment (p < .05). In contrast, reductions in marijuana use were sustained for CARE youth, and greater continued decreases in hard drug use occurred for CARE youth relative to usual care controls.
Drug use control problems also decreased (p < .001) in the CARE participants compared with the usual care youth from baseline to 4-week follow-up. CARE participants had a slight rebound in reported drug use control problems by the 10-week follow-up assessment.
Similarly, adverse drug use consequences decreased (p < .001) in the CARE participants compared with the usual care youth from baseline to 4-week follow-up. However, a subsequent rebound nearly eliminated any difference between the two groups by the 10-week follow-up assessment.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 7: Stress |
|
Description of Measures
|
A clinical RCT evaluated stress with 4 items from the High School Questionnaire: Profile of Experiences that addressed the degree of perceived stress and pressure from others.
|
|
Key Findings
|
Perceived stress decreased by at least 25% in nearly half of the CARE participants (both females and males) from baseline through 10 months of follow-up (p < .001). At baseline, CARE participants scored significantly higher on perceived stress relative to a control group of typical youth, and this difference was associated with a small effect size (Cohen's d = 0.28); however, group differences virtually disappeared (Cohen's d = 0.01) by 10-week follow-up.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 8: Sense of personal control |
|
Description of Measures
|
Two clinical RCTs evaluated the sense of personal control with measures from the High School Questionnaire: Profile of Experiences. The first RCT used a 4-item scale that addressed "self-confidence in handling problems," "ability to make good things happen for self," "ability to learn to adjust/cope with problems," and "confident about feeling better eventually." The second RCT used these items plus an additional item related to "feeling capable and in control."
|
|
Key Findings
|
CARE participants showed no significant improvement in personal control over the 10 months of follow-up in the first study; only 20% showed any increase for this dimension. However, a comparison of CARE participants with a control group of typical youth showed decreased group differences over time. At baseline, CARE participants scored significantly lower on personal control measures compared with the control group, and this difference was associated with a medium effect size (Cohen's d = 0.69). By 10-week follow-up, the difference between the groups was smaller, and the effect size was reduced from medium to small (Cohen's d = 0.45).
In the second study, CARE participants reported significant increases in their sense of personal control compared with a usual care control group of suicidal youth from baseline to 4-week follow-up (p < .001). However, these gains were not significant at the 10-week and 9-month follow-up assessments.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
18-25 (Young adult)
|
58.1% Female
41.9% Male
|
72.5% White
15.1% Race/ethnicity unspecified
5.7% Asian
2.9% Hispanic or Latino
1.9% American Indian or Alaska Native
1.9% Black or African American
|
|
Study 2
|
13-17 (Adolescent)
18-25 (Young adult)
|
52% Female
48% Male
|
49% White
19% Black or African American
18% Asian
10% Hispanic or Latino
4% American Indian or Alaska Native
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Suicide risk factors
|
3.5
|
3.5
|
3.8
|
2.8
|
3.5
|
3.9
|
3.5
|
|
2: Severity of depression symptoms
|
3.5
|
3.5
|
3.8
|
2.8
|
3.5
|
3.9
|
3.5
|
|
3: Feelings of hopelessness
|
3.3
|
3.3
|
3.7
|
2.8
|
3.5
|
3.8
|
3.4
|
|
4: Anxiety
|
3.5
|
3.5
|
4.0
|
3.0
|
3.5
|
3.5
|
3.5
|
|
5: Anger control problems
|
3.5
|
3.5
|
4.0
|
3.0
|
3.5
|
3.5
|
3.5
|
|
6: Drug involvement
|
3.5
|
3.5
|
4.0
|
2.5
|
3.5
|
4.0
|
3.5
|
|
7: Stress
|
3.5
|
3.5
|
3.5
|
2.3
|
3.3
|
3.8
|
3.3
|
|
8: Sense of personal control
|
3.5
|
3.5
|
3.9
|
3.2
|
3.5
|
3.9
|
3.6
|
Study Strengths The studies employed excellent randomized designs with good intervention fidelity protocols and appropriate analytic strategies. Both studies demonstrated intervention fidelity and used good instrumentation and data analysis methods.
Study Weaknesses The studies relied on a few selected items from well-established instruments for measurement of important outcomes, jeopardizing reliability and validity. The second study had an attrition rate of greater than 10%, and the first study had fewer than 50 subjects in each condition with refusal/attrition rates greater than 10%.
|
|
