Quality of Research
Review Date: September 2009
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Eggert, L. L., & Herting, J. R. (1991). Preventing teenage drug abuse: Exploratory effects of network social support. Youth and Society, 22(4), 482-524.
Eggert, L. L., Seyl, C. D., & Nicholas, L. J. (1990). Effects of a school-based prevention program for potential high school dropouts and drug abusers. International Journal of the Addictions, 25(7), 773-801.  Study 2Eggert, L. L., Thompson, E. A., Herting, J. R., Nicholas, L. J., & Dicker, B. G. (1994). Preventing adolescent drug abuse and high school dropout through an intensive school-based social network development program. American Journal of Health Promotion, 8(3), 202-215. Study 3Eggert, L. L., Thompson, E. A., Herting, J. R., & Nicholas, L. J. (1995). Reducing suicide potential among high-risk youth: Tests of a school-based prevention program. Suicide and Life-Threatening Behavior, 25(2), 276-296.
Thompson, E. A., Eggert, L. L., & Herting, J. R. (2000). Mediating effects of an indicated prevention program for reducing youth depression and suicide risk behaviors. Suicide and Life-Threatening Behavior, 30(3), 252-271.
Supplementary Materials Eggert, L. L., & Kumpfer, K. L. (1997). Drug abuse prevention for at-risk individuals (NIH Publication No. 97-4115). Rockville, MD: Office of Science Policy and Communication, National Institute on Drug Abuse.
RY Inc. (2009). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Quality of Research reviewer guide. Redmond, WA: RY Publications.
Outcomes
| Outcome 1: School performance |
|
Description of Measures
|
School performance was assessed using data from:
- Official school records on school achievement and class attendance. School achievement was measured using grade point average (GPA) and the number of credits earned each semester. Potential grades ranged from 0.00 to 4.00 (0.00 to 0.99 reflected a failing grade, and 1.00 and above reflected a passing grade). Potential credits per semester ranged from 0 to 9, with each passing grade counting as 1 credit. Attendance was measured by actual daily absences in each class per semester, recorded on students' academic records as number of days absent/semester, ranging from 0 to 90 days.
- The High School Questionnaire: Profile of Experiences (HSQ). Two items from the HSQ measured students' perceptions of their school performance and attendance during the prior semester: "My overall performance (grades, credits earned) last semester was…" with response options on a scale from 0 (very poor) to 6 (outstanding) and "My overall attendance (in all my classes) last semester was…" with response options ranging from 0 (rarely attended) to 6 (rarely missed).
|
|
Key Findings
|
In one study, 9th- through 12th-grade students identified as high risk for potential school dropout were assigned to an experimental group receiving one semester of RY or to an assessment-only control group. GPA, credits earned, and class absenteeism were recorded for two semesters prior to treatment (two pretests) and one semester at program exit (posttest). Results of this study included the following:
- GPAs and earned credits per semester decreased among both groups from the first to second pretest. From the second pretest to the posttest, GPAs and earned credits increased among students from the experimental group and continued to decline among students from the control group (p < .001).
- Class absenteeism increased among both groups from the first to second pretest. From the second pretest to the posttest, absenteeism decreased among students from the experimental group, while it continued to increase among students from the control group (p < .001).
In another study, 9th- through 12th-grade students identified as high risk for potential school dropout were assigned to an experimental group receiving one semester of RY (as an elective course) or to a control group with a regular school schedule. Data on GPA (excluding the grade from the RY class), absenteeism, and perceived school performance and attendance were collected at pretest, at posttest at the end of the 5-month semester, and at follow-up 5 months later. Results of this study included the following:
- GPA, as recorded in school records, increased across time in the experimental group (pretest mean = 1.25, posttest mean = 1.45, and follow-up mean = 1.47) and decreased in the control group (pretest mean = 1.74, posttest mean = 1.70, and follow-up mean = 1.64; p = .024).
- Class absences, as recorded in school records, did not differ significantly between the two groups.
- Students from the experimental group perceived improvements in their school performance across time, whereas students from the control group did not (p = .001).
- Students from the experimental group perceived improvements in their school attendance across time compared with students from the control group (p = .002).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 2: Drug involvement |
|
Description of Measures
|
Drug involvement was assessed by self-report using the Drug Involvement Scale for Adolescents (DISA) included in the HSQ. The following subscales were used:
- Adverse drug use consequences subscale. Twelve items measured psychosocial complications (e.g., problems with family and friends), biochemical consequences (e.g., blackouts, morning use of drugs), school problems (e.g., suspensions, violations), and legal problems (e.g., DWI, arrests for drunkenness). Students reported the frequency of each consequence experienced on a scale from 0 (never) to 7 (several times/day). Index scores were computed by averaging item responses; the greater the score, the greater the adverse consequences experienced due to drug involvement.
- Degree of drug use subscale. Ten items measured amount and frequency of drug use, extent of use by peers, peer pressure to use, rationale for use, and network feedback about use. Items asked about substance use in general rather than use of specific drugs. Students responded using a scale from 1 (strongly disagree) to 7 (strongly agree). Index scores were computed by averaging item responses; the greater the score, the greater the degree of drug use.
- Progression of drug use subscale. To measure the transition of use from licit drugs to illicit drugs, students were asked to describe their alcohol and drug use during the past 2 weeks using a 7-point scale. Each response category subsumed the use of alcohol and drugs in the lower categories: 0 = no use, 1 = cigarette use only, 2 = beer/wine use, 3 = hard liquor use, 4 = marijuana use, 5 = illicit drug use other than cocaine, and 6 = cocaine use. In addition, the use of "hard" drugs was measured using questions about the frequency of use of seven substances: cocaine, opiates, depressants, tranquilizers, hallucinogens, inhalants, and stimulants. Students reported on use in the past 30 days using a scale from 0 (not at all) to 7 (several times a day). Responses were summed, with higher scores indicating greater frequency of hard drug use.
- Drug control problems subscale. Eight items addressed indicators of uncontrolled drug use, intended use or abstinence, basis for use, and pervasiveness of use at home and at school. Students responded using a scale ranging from 0 (not at all) to 7 (several times per day) to report problems experienced during the past 2 weeks. In addition, a pervasiveness index was calculated using a set of true/false items and summing the number of true (1) and false (0) items endorsed.
In one study, a total drug involvement level was computed by combining the adverse drug use consequences subscale and the degree of drug use subscale. These subscales were standardized to produce the same potential ranges of 1-7 for each item (the greater the score, the greater the drug involvement).
In another study, the adverse drug use consequences subscale and drug control problems subscale were combined to form the drug control problems and consequences scale.
|
|
Key Findings
|
In one study, 9th- through 12th-grade students identified as high risk for potential school dropout were assigned to an experimental group receiving one semester of RY or to an assessment-only control group. From pretest to posttest, students from the experimental group had a significant decrease in scores on adverse drug use consequences (pretest mean = 1.45 and posttest mean = 0.87; p < .001), degree of drug use (pretest mean = 3.90 and posttest mean = 3.18; p < .001), and total drug involvement (pretest mean = 5.40 and posttest mean = 4.07; p < .001). No data on these measures were available for the control group.
In another study, 9th- through 12th-grade students identified as high risk for potential school dropout were assigned to an experimental group receiving one semester of RY (as an elective course) or to a control group with a regular school schedule. Data on progression of drug use and drug control problems and consequences were collected at pretest, at posttest at the end of the 5-month semester, and at follow-up 5 months later. Results of this study included the following:
- The experimental group had a decrease in scores on progression of drug use across time (pretest mean = 2.89, posttest mean = 2.77, and follow-up mean = 2.69) while the control group had an increase (pretest mean = 2.05, posttest mean = 2.24, and follow-up mean = 2.35), but the difference between the groups was not statistically significant (p = .66). However, there was a significant difference in the frequency of the use of hard drugs, which decreased 50% in the experimental group and increased 45% in the control group (p < .001).
- The experimental group had a significant decrease in scores on drug control problems and consequences across time (pretest mean = 0.79, posttest mean = 0.56, and follow-up mean = 0.60) compared with the control group (pretest mean = 0.33, posttest mean = 0.30, and follow-up mean = 0.39; p = .029).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.2
(0.0-4.0 scale)
|
| Outcome 3: Mental health risk and protective factors |
|
Description of Measures
|
Mental health risk and protective factors were assessed by self-report using scales included in the HSQ. Scales measuring protective factors included the following:
- Self-esteem scale. A 4-item modified version of the Rosenberg Self-Esteem Scale was used to assess internalized self-regard and self-worth. Responses were given on a frequency scale from 0 (never) to 6 (always).
- School bonding scale. School bonding was defined as the degree of attachments to teachers and commitment to conventional school goals. Three items were used to measure students' perceptions of support received from their teachers: "My teacher…encouraged and supported me," "…was someone I could count on to help me," and "…offered useful points of view about topics we discussed." For each class the student took, he or she responded using a 7-point scale from 0 (never) to 6 (always). The average across all classes was used as an estimate of school bonding.
- Sense of personal control scale. Four items measured sense of personal control, defined by self-confidence in handling problems and the ability to effect positive outcomes. Items included "I feel confident that I can handle my problems" and "When I try, I can make good things happen to me." Responses ranged from 0 (never) to 6 (always).
- Perceived social support scale. For each of six network support sources (favorite teacher, school counselor, classmates, parents, siblings, and best friend), students rated instrumental support (e.g., providing help, showing different ways to handle a problem) and expressive support (e.g., listening, motivating, encouraging) on a 21-point scale ranging from -10 (nonsupportive) to +10 (supportive), with 0 representing neither nonsupportive or supportive. Ratings across the six network support sources were averaged for one score.
Scales measuring risk factors included the following:
- Deviant peer bonding scale. Adapted from the scale developed by Elliott, Huizinga, and Ageton, this scale assessed the degree of attachment to peers involved in socially deviant behaviors. Six questionnaire items asked about the proportion of friends involved in deviant behaviors such as drug use, skipping school, and getting into trouble. Responses ranged from 0 (none) to 3 (half of my friends) to 6 (almost all of my friends) and were averaged across all items for one score.
- Depression scale. Five items capturing depressive affect (e.g., "I feel depressed," "Nobody cares," "I can't shake off feeling down and blue") were adapted from the Center for Epidemiologic Studies Depression Scale for use with adolescents. Response categories ranged from 0 (never) to 6 (always).
- Feelings of hopelessness scale. Three items addressed feelings of discouragement, lack of enjoyment in life, and a sense that there are no viable solutions to problems. Response categories ranged from 0 (never) to 6 (always).
- Anger scale. Three items measured irritability, loss of control when angry, and physically striking out. Response categories ranged from 0 (never) to 6 (always).
- Perceived stress scale. Four items measured the degree of perceived stress and pressure from others. Response categories ranged from 0 (never) to 6 (always).
|
|
Key Findings
|
In one study, 9th- through 12th-grade students identified as high risk for potential school dropout were assigned to an experimental group receiving one semester of RY (as an elective course) or to a control group with a regular school schedule. Data on self-esteem, school bonding, and deviant peer bonding were collected at pretest, at posttest at the end of the 5-month semester, and at follow-up 5 months later. Results of this study included the following:
- The experimental group had a significant increase in scores on self-esteem (pretest mean = 3.31, posttest mean = 3.78, and follow-up mean = 3.95) compared with the control group (pretest mean = 3.99, posttest mean = 4.07, and follow-up mean = 4.14; p = .005).
- The experimental group had a significant increase in scores on perceived school bonding (pretest mean = 3.20, posttest mean = 3.59, and follow-up mean = 3.69) compared with the control group (pretest mean = 3.73, posttest mean = 3.78, and follow-up mean = 3.75; p = .017).
- Females from the experimental group had a significant decrease in scores on deviant peer bonding (pretest mean = 2.92, posttest mean = 2.58, and follow-up mean = 2.25) compared with those from the control group (pretest mean = 1.90, posttest mean = 1.92, and follow-up mean = 2.22; p = .013). No significant difference was found between the males from the two groups.
In another study, 9th- through 12th-grade students with suicide risk behaviors were assigned to one of three groups:
- Group I, which received one semester of RY (including an enhanced life skills training in personal control, depression, and anger management, as well as greater drug use monitoring) and a suicide risk assessment protocol
- Group II, which received two semesters of RY (one semester similar to that of group I, plus an additional semester of the life skills training along with a positive peer group booster, relapse prevention, and enhanced school bonding activities) and a suicide risk assessment protocol
- Group III, which received a suicide risk assessment protocol only
All three groups were assessed using the Measure of Adolescent Suicide Potential (MAPS), a comprehensive, computer-assisted assessment protocol designed for ethical reasons to provide a positive, no-harm experience for all participants. Although its intended purpose is to measure suicide risk and related factors in detail, MAPS also appears to have the potential effect of reducing suicide risk. Data on depression, hopelessness, perceived stress, anger, sense of personal control, self-esteem, and perceived social support were collected at time 1 (pretest), time 2 (5-month follow-up, coinciding with group I program completion), and time 3 (10-month follow-up, coinciding with group II program completion). From time 1 to time 3:
- Depression decreased significantly for all three groups (p values < .001). In all three groups, more than 65% of the youth had at least a 25% decrease in depression scores, with groups I and III having significantly greater declines than group II.
- Hopelessness decreased significantly for all three groups (p values < .001). More than 60% of the students in each group had declines in hopelessness across time. Females from group I compared with all other youth in the study had the most dramatic decreases in feelings of hopelessness at time 2 (p < .05).
- Perceived stress decreased significantly for all three groups (p values < .001), with no significant differences between the groups. Forty-five percent of the students in each group had at least a 25% reduction in scores on perceived stress.
- Anger declined significantly more for groups I and III than for group II (p = .019). More than 65% of youth from groups I and III had at least a 25% reduction in anger scores, whereas only 45% from group II had declines at this level.
- Groups I and II had significant increases in personal control compared with group III (p = .027). More than 44% of youth in groups I and II had improvements in personal control, compared with only 20% from group III.
- Self-esteem increased significantly for all three groups (p values < .001), with no significant differences between the groups.
- Perceived social support increased significantly for all three groups (p values < .001), with no significant differences between the groups. Further analysis revealed that the favorite teacher and parents were the network support sources that accounted for increases in perceived social support.
|
|
Studies Measuring Outcome
|
Study 2, Study 3
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 4: Suicide risk behaviors |
|
Description of Measures
|
Suicide risk behaviors were assessed by self-report using the Brief Suicide Risk Behavior Scale included in the HSQ. This scale measures the frequency of suicide thoughts, direct and indirect suicide threats, and suicide attempts. Response options range from 0 (never) to 6 (many times/always).
|
|
Key Findings
|
Students in 9th through 12th grade with suicide risk behaviors were assigned to one of three groups:
- Group I, which received one semester of RY (including an enhanced life skills training in personal control, depression, and anger management, as well as greater drug use monitoring) and a suicide risk assessment protocol
- Group II, which received two semesters of RY (one semester similar to that of group I, plus an additional semester of the life skills training along with a positive peer group booster, relapse prevention, and enhanced school bonding activities) and a suicide risk assessment protocol
- Group III, which received a suicide risk assessment protocol only
All three groups were assessed using MAPS, a comprehensive, computer-assisted assessment protocol designed for ethical reasons to provide a positive, no-harm experience for all participants. Although its intended purpose is to measure suicide risk and related factors in detail, MAPS also appears to have the potential effect of reducing suicide risk. Data on suicide risk behaviors were collected at time 1 (pretest), time 2 (5-month follow-up, coinciding with group I program completion), and time 3 (10-month follow-up, coinciding with group II program completion). All three groups had a significant decline in suicide risk behaviors (p values < .001). Groups I and III had greater reductions in suicide risk behaviors from time 1 to time 3 than did group II; 85% of the students in groups I and III and 65% of those in group II decreased suicide risk behavior scores by at least 25%.
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
18-25 (Young adult)
|
53% Male
47% Female
|
100% Race/ethnicity unspecified
|
|
Study 2
|
13-17 (Adolescent)
18-25 (Young adult)
|
57% Male
43% Female
|
76% White
24% Race/ethnicity unspecified
|
|
Study 3
|
13-17 (Adolescent)
18-25 (Young adult)
|
58% Female
42% Male
|
72% White
15.6% Race/ethnicity unspecified
5.7% Asian
2.9% Hispanic or Latino
1.9% American Indian or Alaska Native
1.9% Black or African American
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: School performance
|
3.3
|
4.0
|
3.8
|
2.8
|
2.8
|
3.5
|
3.3
|
|
2: Drug involvement
|
3.5
|
4.0
|
3.8
|
2.3
|
2.5
|
3.0
|
3.2
|
|
3: Mental health risk and protective factors
|
3.5
|
4.0
|
3.8
|
2.3
|
2.8
|
3.5
|
3.3
|
|
4: Suicide risk behaviors
|
3.5
|
4.0
|
3.8
|
2.3
|
2.8
|
3.5
|
3.3
|
Study Strengths The studies reviewed used standardized measures with acceptable reliability and validity. The processes used for ensuring intervention fidelity were very strong across studies and improved over time (i.e., hiring, training, and supervision practices; use of evaluation tools to measure intervention fidelity; random observation). The studies used appropriate analyses.
Study Weaknesses All three studies had issues of self-selection bias resulting in group differences at baseline. In one study, data on drug involvement were not reported for the control group.
|
|
Readiness for Dissemination
Review Date: September 2009
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Eggert, L. L., & Nicholas, L. J. (2004). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Curriculum: Module 1--Getting started (with CD-ROM) (2nd ed.). Bloomington, IN: National Educational Service.
Eggert, L. L., & Nicholas, L. J. (2004). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Curriculum: Module 2--Self-esteem enhancement (2nd ed.). Bloomington, IN: National Educational Service.
Eggert, L. L., & Nicholas, L. J. (2004). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Curriculum: Module 3--Decision making (2nd ed.). Bloomington, IN: National Educational Service.
Eggert, L. L., & Nicholas, L. J. (2004). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Curriculum: Module 4--Personal control (2nd ed.). Bloomington, IN: National Educational Service.
Eggert, L. L., & Nicholas, L. J. (2004). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Curriculum: Module 5--Interpersonal communication (2nd ed.). Bloomington, IN: National Educational Service.
Eggert, L. L., & Nicholas, L. J. (2004). Reconnecting Youth: Student workbook. Bloomington, IN: National Educational Service.
Reconnecting Youth Web site, http://www.reconnectingyouth.com
RY Inc. (2006). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Sample leader training PowerPoint presentation slides: Day 1--Introduction to RY. Redmond, WA: RY Publications.
RY Inc. (2006). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Sample T4T training PowerPoint presentation slides: Day 1--Coaching the preparation of a lesson. Redmond, WA: RY Publications.
RY Inc. (2006). Reconnecting Youth: First 10 days agenda posters. Redmond, WA: RY Publications.
RY Inc. (2006). Reconnecting Youth: Program goals posters. Redmond, WA: RY Publications.
RY Inc. (2006). Reconnecting Youth: Training aids/materials. Redmond, WA: RY Publications.
RY Inc. (2006). Reconnecting Youth training: Leadership skills to implement the RY program. Participant guide (2nd ed.). Redmond, WA: RY Publications.
RY Inc. (2006). Reconnecting Youth training: Promoting leadership skills to implement the RY program. Trainer guide (2nd ed.). Redmond, WA: RY Publications.
RY Inc. (2006). Reconnecting Youth training: Promoting RY trainer skills for training RY leaders. RY T-4-T guide (2nd ed.). Redmond, WA: RY Publications.
RY Inc. (2008). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Administrator's handbook (2nd ed.). Redmond, WA: RY Publications.
RY Inc. (2008). Reconnecting Youth: Sample administrator/coordinator training PowerPoint presentation slides: Day 3--Using process data for RY supervision. Redmond, WA: RY Publications.
RY Inc. (2008). RY administrator & coordinator training: Building infrastructure & assuring implementation fidelity. Participant guide (2nd ed.). Redmond, WA: RY Publications.
RY Inc. (2008). RY administrator & coordinator training: Building infrastructure & assuring implementation fidelity. Trainer guide (2nd ed.). Redmond, WA: RY Publications.
RY Inc. (2009). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Descriptive dimensions. Redmond, WA: RY Publications.
RY Inc. (2009). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Program evaluation measures and the timeline for administration. Redmond, WA: RY Publications.
RY Inc. (2009). Reconnecting Youth: A Peer Group Approach to Building Life Skills. Readiness for Dissemination reviewer guide. Redmond, WA: RY Publications.
RY Inc. (2009). Reconnecting Youth: Outcome evaluation materials. Redmond, WA: RY Publications.
RY Inc. (2009). Reconnecting Youth: Process evaluation materials. Redmond, WA: RY Publications.
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
4.0
|
4.0
|
4.0
|
4.0
|
Dissemination Strengths Materials are detailed, thorough, well organized, and user friendly. Infrastructure issues relevant to implementation sites are well addressed, and good guidance for identifying and selecting participants is offered. Program information is easy to access on the Web site. Several highly interactive and comprehensive training options are available to implementers, as is solid and practical background material to prepare staff for their roles. The training materials are well staged, allowing participants to move through the content in a logical progression. Quality assurance is seen as integral to the cycle of implementation, and a good set of tools is provided to assist implementers. Questionnaires and various checklists allow a multidimensional array of input for facilitating and documenting fidelity.
Dissemination Weaknesses No weaknesses were identified by reviewers.
|
|
