Quality of Research
Review Date: January 2008
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Bacon, T. P. (2001). Impact on high school students' behaviors and protective factors: A pilot study of the Too Good for Drugs and Violence prevention program. Florida Educational Research Council, Inc., Research Bulletin, 32(3 & 4), 1-40. Study 2Bacon, T. P. (2000). The effects of the Too Good for Drugs II drug prevention program on students' substance use intentions and risk and protective factors. Florida Educational Research Council, Inc., Research Bulletin, 31(3 & 4), 1-25. Study 3Bacon, T. P. (2003). Technical report: Evaluation of the Too Good for Drugs--elementary school prevention program. A report produced for a project funded by the Florida Department of Education, Department of Safe and Drug-Free Schools, Tallahassee, FL.
Supplementary Materials Bacon, T. P. (2001). Evaluation of the Too Good for Drugs and Violence--high school prevention program. A report produced for a project funded by the Florida Department of Education, Department of Safe and Drug-Free Schools, Tallahassee, FL.
Bacon, T. P. (2002, April). Resiliency of students' protective factors using a school-based drug prevention program. Paper presented at the Annual Conference of the American Educational Research Association, New Orleans, LA.
Bacon, T. P. (2004). Technical report: Evaluation of the Too Good for Drugs--middle school prevention program. A report produced for a project funded by the Florida Department of Education, Department of Safe and Drug-Free Schools, Tallahassee, FL.
Bacon, T. P. (2004). Technical report: Pilot study of the Too Good for Drugs and Violence after-school activities program. A project funded by the C. E. Mendez Foundation, Inc., Tampa, FL.
Patterson, G. A. (2004). Too Good for Drugs: Elementary school fidelity of implementation. Prepared for the School District of Palm Beach County, FL, and the Florida Department of Education.
Outcomes
| Outcome 1: Intentions to use alcohol, tobacco, and marijuana and to engage in violence |
|
Description of Measures
|
A 4-item instrument developed for the study was used to gauge students' intentions to use alcohol, tobacco, and marijuana and to engage in fighting within the next 12 months. The 5-point response scale ranged from "strongly agree" to "strongly disagree." Student responses to the items were dichotomized to represent confidence in intentions to not use substances or violence versus less certainty about intentions to use (or current use).
|
|
Key Findings
|
In one study, from pre- to posttest, the proportion of students with intentions to drink alcohol was significantly reduced in the treatment group compared with the control group, which received a standard physical education curriculum (p < .05). The proportions of students with intentions to smoke, use marijuana, or engage in fighting also were reduced in the treatment group compared with the control group, but the differences between the two groups were not statistically significant.
In a second study, from pre- to posttest, the proportions of students with intentions to drink alcohol and smoke were significantly reduced for the treatment group compared with the wait-list control group (p = .02 and .04, respectively). During the same time, the intention to use marijuana also was reduced for the treatment group compared with the control group, but the difference between the two groups was not statistically significant. At 20-week follow-up, the proportions of students with intentions to drink alcohol, smoke, and use marijuana were reduced in the treatment group compared with the control group, but the differences between the two groups were not statistically significant.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 2: Risk and protective factors for substance use and violence |
|
Description of Measures
|
Attitudes and skills related to substance use and/or violence were measured using:
- An instrument developed for the study comprising 61 Likert-type, self-report items with responses ranging from "strongly agree" to "strongly disagree." These items were grouped into nine subscales representing protective factors associated with young people's resistance to substance and violence use: positive attitudes toward nondrug use, positive attitudes toward nonviolence, perceived peer normative substance and violence use, perceived peer disapproval of substance and violence use, emotional competence, goal-setting and decisionmaking skills, social and peer resistance skills, perceived harmful effects of substance use, and perceived parental attitudes toward substance use. Higher scores indicated more positive perceptions or behaviors.
- An instrument developed for the study comprising 19 Likert-type, self-report items with responses ranging from "strongly agree" to "strongly disagree." The items were grouped into six subscales representing personal risk and protective factors or mediating variables associated with young people's resistance to substance use: perceived peer resistance skills, positive attitudes toward nondrug use, perceived peer normative substance use, perceived peer disapproval of substance use, prosocial peers, and locus of control/self-efficacy. Higher scores indicated more positive perceptions or behaviors.
- The Student Survey Questionnaire, comprising 30 Likert-type, self-report items with responses ranging from "strongly disagree" to "strongly agree." Student responses were grouped into five subscales representing protective factors associated with children's resiliency to social challenges: attitudes toward drugs, emotional competency, goal-setting and decisionmaking skills, social and peer resistance skills, and perceived harmful effects of substance use. Higher scores indicated more positive levels of attitudes, perceptions, or skills.
|
|
Key Findings
|
In one study, from pre- to posttest, students in the treatment group had significantly increased scores in eight of nine protective areas compared with students in the control group, who received a standard physical education curriculum (all p values < .01). Although the treatment group had improvement in parent-child interactions and discussions, the difference between the treatment and control groups at posttest was not statistically significant.
In another study, from pre- to posttest, students in the treatment group had significantly increased scores in all six risk and protective areas compared with students in the wait-list control group (all p values < .01). At 20-week follow-up, the difference between the groups remained statistically significant in only four of the six risk and protective areas (i.e., perceived peer resistance skills, perceived peer normative substance use, perceived peer disapproval of substance use, locus of control/self-efficacy; all p values < .01).
In a third study, from pre- to posttest, students in the treatment group had significantly increased scores in four of five protective areas (i.e., emotional competency, social and peer resistance skills, goal-setting and decisionmaking skills, perceived harmful effects of substance use) compared with students in the wait-list control group (all p values < .01). At 4-month follow-up, the difference between the groups remained statistically significant in only goal-setting and decisionmaking skills (p < .01).
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.9
(0.0-4.0 scale)
|
| Outcome 3: Personal and prosocial behaviors |
|
Description of Measures
|
Personal and prosocial behaviors were measured using a teacher checklist of student behavior. Teachers responded to 23 behavioral items using a 5-point scale ranging from 1 (never) to 5 (almost always). The responses to items were grouped into three protective subscales associated with a student's social adaptability: personal and social skills, positive social behaviors, and inappropriate social behaviors.
|
|
Key Findings
|
From pre- to posttest, students in the treatment group showed significantly increased use of personal and social skills, increased engagement in prosocial behaviors, and decreased engagement in inappropriate social behaviors compared with students in the wait-list control group (all p values < .01). These results were maintained at 4-month follow-up.
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.9
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
|
51% Male
49% Female
|
68% White
20% Hispanic or Latino
9% Black or African American
2% Asian
1% American Indian or Alaska Native
|
|
Study 2
|
6-12 (Childhood)
|
52% Female
48% Male
|
48% White
33% Black or African American
13% Hispanic or Latino
6% Asian
|
|
Study 3
|
6-12 (Childhood)
|
51% Male
49% Female
|
71% White
17% Black or African American
10% Hispanic or Latino
2% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Intentions to use alcohol, tobacco, and marijuana and to engage in violence
|
2.5
|
2.3
|
3.0
|
2.9
|
2.9
|
3.0
|
2.8
|
|
2: Risk and protective factors for substance use and violence
|
2.8
|
2.6
|
3.0
|
3.0
|
3.0
|
3.1
|
2.9
|
|
3: Personal and prosocial behaviors
|
3.0
|
2.5
|
3.1
|
3.0
|
2.9
|
3.2
|
2.9
|
Study Strengths Standardized program implementation was established to guide how the program was conducted, and implementation fidelity was monitored by observations and surveys. Appropriate procedures were used to control for attrition, missing data, and potential confounding variables.
Study Weaknesses The psychometric properties of the instruments developed for the studies were not adequately presented. For some of the measures, criterion validity was not confirmed (i.e., the instruments were not compared with more established measures). Similarly, the results of factor analysis were not presented to demonstrate the instruments' validity.
|
|
Readiness for Dissemination
Review Date: January 2008
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Mendez Foundation. (n.d.). Too Good Programs catalog. Tampa, FL: Author.
Mendez Foundation kits:
- Too Good for Drugs & Violence After-School Activities Kit
- Too Good for Drugs & Violence High School Kit
- Too Good for Drugs Grade 1 Kit
- Too Good for Drugs Grade 3 Kit
- Too Good for Drugs Grade 6 Kit
Mendez Foundation training materials:
- Too Good for Drugs & Violence After-School Activities curriculum training packet
- Too Good for Drugs & Violence High School curriculum training packet
- Too Good for Drugs & Violence Staff Development Kit
- Too Good for Drugs K-8 curriculum training packet
- Too Good Programs Regional Trainings [brochure]
- Too Good Programs Training of Trainers Manual
Program Web site, http://mendezfoundation.org
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
4.0
|
4.0
|
4.0
|
4.0
|
Dissemination Strengths Program materials are clearly presented, well organized, and easy to read. Curricula are well scripted and formatted for easy use in a classroom setting. Lesson extenders greatly enhance implementation opportunities. High-quality training materials are provided to facilitate multiple training levels and formats, including on- and off-site, regional, training-of-trainers, and refresher trainings. A variety of tools are provided to support quality assurance, including scripted curricula, implementation fidelity instruments for each grade level, and outcome monitoring measures. In addition, extensive guidance explains how to administer quality assurance tools, score the data collected, and analyze the results.
Dissemination Weaknesses No weaknesses were identified by reviewers.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Too Good for Drugs K-8 grade-specific kit (each includes the teacher's manual; 50 student workbooks; and age-appropriate teaching materials including posters, puppets, CDs, DVDs, and/or games)
|
$100-$130 each
|
Yes
|
|
Too Good for Drugs & Violence high school kit (each includes the teacher's manual; 50 student notebooks; age-appropriate teaching materials including posters, puppets, CDs, DVDs, and activities; and TGFDV staff development program and 12 core infusion lessons)
|
$750 each
|
Yes
|
|
Too Good for Drugs & Violence After-School Activities kit (each includes the facilitator's guide and age-appropriate teaching materials including posters, puppets, CDs, DVDs, and games)
|
$595 each
|
Yes
|
|
Too Good for Drugs & Violence Staff Development kit (includes the facilitator's guide and 50 participant workbooks)
|
$250 each
|
Yes
|
|
1-day, on-site curriculum training
|
$2,000 for groups of 10-50 participants, plus travel expenses
|
No
|
|
Train-the-trainer session
|
$400 per person per day
|
No
|
|
1-day, off-site curriculum training
|
$295 per person per day
|
No
|
|
Implementation design and technical assistance before, during, and following program implementation
|
Free
|
No
|
|
Student behavior checklist, student outcome survey, student knowledge test, teacher implementation instrument, and classroom observation checklist
|
Included with kits
|
No
|
Additional Information Kit components may be purchased individually.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
Bacon, T. P. (2001). Evaluation of the Too Good for Drugs and Violence--high school prevention program. A report produced for a project funded by the Florida Department of Education, Department of Safe and Drug-Free Schools, Tallahassee, FL.
Bacon, T. P. (2004). Technical report: Evaluation of the Too Good for Drugs--middle school prevention program. A report produced for a project funded by the Florida Department of Education, Department of Safe and Drug-Free Schools, Tallahassee, FL.
Bacon, T. P. (2006). Evaluation of the Too Good for Drugs--middle school prevention program. In M. Harris (Ed.), Safe and Drug-Free Schools: Exemplary evaluation practices from the field. University of North Florida, Florida Institute of Education, Safe, Disciplined, and Drug-Free School Project, for the Florida Department of Education and local agencies.
|
|
|
