Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Hansen, W. B. (1996). Pilot test results comparing the All Stars program with seventh grade D.A.R.E.: Program integrity and mediating variable analysis. Substance Use and Misuse, 31(10), 1359-1377.  Study 2Harrington, N. G., Giles, S. M., Hoyle, R. H., Feeney, G. J., & Yungbluth, S. C. (2001). Evaluation of the All Stars character education and problem behavior prevention program: Effects on mediator and outcome variables for middle school students. Health Education and Behavior, 28(5), 533-546.
McNeal, R. B., Jr., Hansen, W. B., Harrington, N. G., & Giles, S. M. (2004). How All Stars works: An examination of program effects on mediating variables. Health Education and Behavior, 31(2), 165-178.
Outcomes
| Outcome 1: Personal commitment not to use drugs |
|
Description of Measures
|
The personal commitment variable consisted of 12 survey items assessing private and public manifestations of commitments regarding substance use and other problem behaviors. Responses were on a 4-point Likert scale ranging from "strongly agree" to "strongly disagree." Items included, for example, "I have made a decision to not get high by sniffing fumes."
|
|
Key Findings
|
All Stars participants' average scores for personal commitment increased from pre- to posttest, while scores decreased among recipients of an alternative program (p < .0001). This result was replicated in a second study in which All Stars was delivered by a classroom teacher (p < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.2
(0.0-4.0 scale)
|
| Outcome 2: Lifestyle incongruence |
|
Description of Measures
|
The lifestyle incongruence variable consisted of 9 survey items assessing students' beliefs that substance use, violence, and premature sexual activity were incongruent with their ideals and desired lifestyle. Responses were on a 4-point Likert scale ranging from "strongly agree" to "strongly disagree." Items included, for example, "Getting high from sniffing glue would get in the way of what is important to me."
|
|
Key Findings
|
All Stars participants' average scores for lifestyle incongruence increased from pre- to posttest, while scores decreased among recipients of an alternative program (p < .0001). This result was replicated in a second study in which All Stars was delivered by a classroom teacher (p < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.2
(0.0-4.0 scale)
|
| Outcome 3: School bonding |
|
Description of Measures
|
The school bonding variable consisted of 8 survey items assessing how students felt received at school, with responses on a 4-point Likert scale ranging from "strongly agree" to "strongly disagree." Items included, for example, "I like the teachers at this school" and "The teachers at this school like me."
|
|
Key Findings
|
All Stars participants' average scores for school bonding increased from pre- to posttest, while scores decreased among recipients of an alternative program (p < .0001).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.2
(0.0-4.0 scale)
|
| Outcome 4: Normative beliefs |
|
Description of Measures
|
The normative beliefs variable consisted of 11 survey items assessing student perceptions about the prevalence of high-risk behaviors (e.g., "How many people your age do you think use marijuana at least once a month"), with responses on a 5-point Likert scale ranging from "none" to "all," and acceptability of these behaviors to friends (e.g., "My friends think it is OK to get drunk every now and then"), with responses on a 4-point Likert scale ranging from "strongly agree" to "strongly disagree."
|
|
Key Findings
|
All Stars participants' average scores on the normative beliefs variable increased from pre- to posttest, while scores decreased among recipients of an alternative program (p < .0002).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.2
(0.0-4.0 scale)
|
| Outcome 5: Cigarette use |
|
Description of Measures
|
Lifetime use of cigarettes (yes or no), past 30-day use (yes or no), and frequency of use in the past 30 days (not at all, 1-2 times, 3-9 times, 10-19 times, 20 or more times) was assessed by 4 survey items.
|
|
Key Findings
|
When the program was delivered by a teacher, All Stars participants reported lower average levels of cigarette use at posttest compared with students who did not receive the program (p < .05).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.2
(0.0-4.0 scale)
|
| Outcome 6: Alcohol use |
|
Description of Measures
|
Lifetime use of alcohol (yes or no), past 30-day use (yes or no), and frequency of use in the past 30-days (not at all, 1-2 times, 3-9 times, 10-19 times, 20 or more times) was assessed by 8 survey items.
|
|
Key Findings
|
When the program was delivered by a teacher, All Stars participants reported lower average levels of alcohol use at posttest compared with students who did not receive the program (p < .05).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.2
(0.0-4.0 scale)
|
| Outcome 7: Inhalant use |
|
Description of Measures
|
Lifetime use of inhalants (yes or no), past 30-day use (yes or no), and frequency of use in the past 30 days (not at all, 1-2 times, 3-9 times, 10-19 times, 20 or more times) was assessed by 4 survey items.
|
|
Key Findings
|
When the program was delivered by a teacher, All Stars participants reported lower average levels of inhalant use at posttest compared with students who did not receive the program (p < .05).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.2
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
13-17 (Adolescent)
|
62.5% Female
37.5% Male
|
46% White
42% Black or African American
8% Asian
4% Hispanic or Latino
|
|
Study 2
|
6-12 (Childhood)
13-17 (Adolescent)
|
55% Female
45% Male
|
69% White
25% Black or African American
6% Hispanic or Latino
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Personal commitment not to use drugs
|
2.5
|
2.0
|
2.5
|
2.1
|
1.8
|
2.5
|
2.2
|
|
2: Lifestyle incongruence
|
2.5
|
2.0
|
2.5
|
2.1
|
1.8
|
2.5
|
2.2
|
|
3: School bonding
|
2.5
|
2.0
|
2.5
|
2.1
|
1.8
|
2.5
|
2.2
|
|
4: Normative beliefs
|
2.5
|
2.0
|
2.5
|
2.1
|
1.8
|
2.5
|
2.2
|
|
5: Cigarette use
|
2.3
|
1.8
|
2.5
|
2.5
|
1.8
|
2.5
|
2.2
|
|
6: Alcohol use
|
2.3
|
1.8
|
2.5
|
2.5
|
1.8
|
2.5
|
2.2
|
|
7: Inhalant use
|
2.3
|
1.8
|
2.5
|
2.5
|
1.8
|
2.5
|
2.2
|
Study Strengths The program is based on a sound theoretical approach and uses measurement scales with high reported levels of internal consistency. One study examined the intervention's effects in a large sample using a longitudinal design. The authors also analyzed the effects of mediating variables that may have been responsible for the outcome variables.
Study Weaknesses Confounding variables existed in one study due to lack of rigor in the design. In addition, this study was conducted in one school, classes were not randomly assigned, and half of the subjects in the comparison group did not complete the posttest. Posttest evaluations were completed at inconsistent lengths of time after the intervention. In the second study, additional schools were added to the design after the matching process. Information concerning the effects of potential confounding variables was not provided, making it difficult to assess this criterion in the two studies.
|
|
Readiness for Dissemination
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
All Stars Core Training booklet
All Stars Survey
Brochures:
- All Stars Certification of Mastery
- All Stars: Their Future, Your Vision, Our Mission
- Evaluation Lizard
Dusenbury, L., & Hansen, W. (2006). Prevention ABCs: A professional development series [DVD set]. Greensboro, NC: Tanglewood Research, Inc.
Handout on registering for the All Stars online community
Hansen, W. (2003). All Stars Booster: Version 2.0. United States edition. Greensboro, NC: Tanglewood Research, Inc.
Hansen, W., & Tanglewood Research, Inc. (2004). All Stars Core: Version 2.4. Greensboro, NC: Tanglewood Research, Inc.
Hansen, W., & Tanglewood Research, Inc. (2004). All Stars Plus: Version 2.4. Greensboro, NC: Tanglewood Research, Inc.
Program Web site, http://www.allstarsprevention.com
Tanglewood Research, Inc. (2006). Superior All Stars commitment videos [DVD].
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.5
|
3.5
|
3.3
|
3.4
|
Dissemination Strengths Detailed implementation manuals are available for all program components. A training program is offered for both the core and booster programs, and teacher certification is available to those interested becoming a program trainer. A supplemental DVD series is also available for professional development purposes. Outcome measures and teacher/administrator satisfaction interview guides are provided to support quality assurance.
Dissemination Weaknesses It is unclear whether supplemental sessions are critical. No ongoing coaching or technical assistance is available for implementers or site administrators. While there is a quality assurance survey that implementers can complete after each session, it is generic and appears to be of limited value for ensuring fidelity.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Core Facilitator Manual
|
$100 each
|
Yes
|
|
Core student materials: Basic package (includes required worksheets only)
|
$4 per student
|
Yes (one Core student materials package is required)
|
|
Core student materials: Standard package (includes required worksheets, inserts, gift card, student pre- and posttests, and fidelity assessment forms)
|
$7 per student
|
Yes (one Core student materials package is required)
|
|
Core student materials: Complete package (includes required worksheets, inserts, gift card, student pre- and posttests, fidelity assessment forms, parenting guides, and parent CDs)
|
$10 per student
|
Yes (one Core student materials package is required)
|
|
Booster Facilitator Manual
|
$80 each
|
No (teaching Booster is optional; however, if Booster is selected for implementation, one manual per teacher is required to implement Booster curriculum)
|
|
Booster student materials: Basic package (includes required worksheets only)
|
$1 per student
|
No (teaching Booster is optional; however, if Booster is selected for implementation, one Booster student materials package is required)
|
|
Booster student materials: Standard package (includes required worksheets, inserts, gift card, student pre- and posttests, and fidelity assessment forms)
|
$3 per student
|
No (teaching Booster is optional; however, if Booster is selected for implementation, one Booster student materials package is required)
|
|
Booster student materials: Complete package (includes required worksheets, inserts, gift card, student pre- and posttests, fidelity assessment forms, parenting guides, and parent CDs)
|
$6 per student
|
No (teaching Booster is optional; however, if Booster is selected for implementation, one Booster student materials package is required)
|
|
Plus Facilitator Manual
|
$80 each
|
No (teaching Plus is optional; however, if Plus is selected for implementation, one Plus manual is required per teacher to implement Plus curriculum)
|
|
Plus student materials: Basic package (includes required worksheets only)
|
$2 per student
|
No (teaching Plus is optional; however, if Plus is selected for implementation, one Plus student materials package is required)
|
|
Plus student materials: Standard package (includes required worksheets, inserts, gift card, student pre- and posttests, and fidelity assessment forms)
|
$4 per student
|
No (teaching Plus is optional; however, if Plus is selected for implementation, one Plus student materials package is required)
|
|
Plus student materials: Complete package (includes required worksheets, inserts, gift card, student pre- and posttests, fidelity assessment forms, parenting guides, and parent CDs)
|
$7 per student
|
No (teaching Plus is optional; however, if Plus is selected for implementation, one Plus student materials package is required)
|
|
1- to 2-day, on-site training
|
$125 per person per day or $3,000 per group (up to 20 people per group)
|
No
|
|
2- to 4-session live online training
|
$75 per person per session
|
No
|
|
Telephone consultation for training participants
|
Free
|
No
|
|
Online support at http://onlinecommunity.tanglewood.net
|
Free
|
No
|
|
Student pre- and posttest surveys (targeted mediators and behaviors; includes an online report)
|
$1 per student (included in the cost of standard and complete packages)
|
No
|
|
Fidelity assessment forms for teachers and on-site observers (includes an online report)
|
$5 per class or group (included in the cost of student pre- and posttest surveys)
|
No
|
|
Fidelity assessment service (certified rater analysis of implementer-submitted video recordings; includes an online report)
|
$85 per session
|
No
|
Additional Information Items can be purchased in various sets that include supplementary materials. Most materials are available in Spanish, and training is also available in Spanish. Current pricing is available on the program Web site.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
Hansen, W., & Dusenbury, L. (2004). All Stars Plus: A competence and motivation enhancement approach to prevention. Health Education, 104(6), 371-381.
Slater, M. D., Kelly, K. J., Edwards, R. W., Thurman, P. J., Plested, B. A., Keefe, T. J., et al. (2006). Combining in-school and community-based media efforts: Reducing marijuana and alcohol uptake among younger adolescents. Health Education Research, 21(1), 157-167.
|
|
|
