Quality of Research
Review Date: October 2007
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Gorman-Smith, D., Tolan, P. H., Henry, D. B., Leventhal, A., Schoeny, M., Lutovsky, K., & Quintana, E. (2002). Predictors of participation in a family-focused preventive intervention for substance use. Psychology of Addictive Behaviors, 16(Suppl. 4), S55-S64. 
Tolan, P., Gorman-Smith, D., & Henry, D. (2000). Promoting academic and social competence among urban youth: The SAFEChildren Project. Final grant report submitted to the Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Prevention.
Tolan, P., Gorman-Smith, D., & Henry, D. (2004). Supporting families in a high-risk setting: Proximal effects of the SAFEChildren preventive intervention. Journal of Consulting and Clinical Psychology, 72(5), 855-869.
Supplementary Materials Gorman-Smith, D., Tolan, P., Henry, D. B., Quintana, E., Lutovsky, K., & Leventhal, A. (2007). Schools and Families Educating Children: A preventive intervention for early elementary school children. In P. H. Tolan, J. Szapocznik, & S. Sambrano (Eds.), Preventing youth substance abuse: Science-based programs for children and adolescents (pp. 113-135). Washington, DC: American Psychological Association.
Outcomes
| Outcome 1: Reading achievement |
|
Description of Measures
|
Children's reading achievement was measured using the Woodcock Diagnostic Reading Battery (WDRB), a comprehensive set of individually administered tests that measure important dimensions of reading achievement and closely related abilities. Four subscales of the battery were administered: letter-word identification, word attack, passage comprehension, and incomplete words. Results from each subscale were combined for a total reading score.
|
|
Key Findings
|
One study analyzed the growth trajectories in reading achievement from the end of kindergarten to the middle of 2nd grade for both intervention and nonintervention children. Children who received the intervention improved in overall reading ability at a more rapid rate for the reading composite (p < .01) than those who did not receive the intervention. When the reading composite results were converted to grade-equivalent scores, the intervention participants were 0.44 grade-equivalent years ahead of the control participants by the middle of 2nd grade. In addition, the intervention participants were somewhat above the national average for mid-2nd-grade reading skills (grade equivalent = 2.9 level achievement, school month = 2.6), while the control participants were slightly below the national average (grade equivalent = 2.4, school month = 2.6). The slope effect size was very small (Cohen's d = 0.17).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 2: Child problem behaviors |
|
Description of Measures
|
Teachers and parents provided ratings of the child's aggression, hyperactivity, and concentration problems using subscales of the Teacher Observations of Classroom Adaptation--Revised (TOCA-R) and Parent Observations of Classroom Adaptation--Revised (POCA-R). In this structured interview, teachers and parents report observations of the child's behaviors that may affect adaptation to school. Higher scores reflect higher levels of aggression, hyperactivity, and problems with concentration. Parent and teacher ratings were combined for each of the subscales.
|
|
Key Findings
|
Among families designated as high risk (those with less adequate parenting skills and family relationship quality at pretest), there was a decrease over time in aggression among children who received the SAFEChildren intervention, whereas there was essentially no change among children who did not receive the intervention (p < .05). In addition, SAFEChildren participants showed an improvement in concentration (p < .05) relative to those who did not receive the intervention. The slope effect sizes were very small (Cohen's d = 0.12 and 0.13, respectively).
Among high-risk children (having high levels of problem behaviors at pretest), SAFEChildren participants showed a decrease in aggression, whereas those who did not receive the intervention had a slight increase in aggression (p < .05). The slope effect size was very small (Cohen's d = 0.10).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 3: Parenting practices |
|
Description of Measures
|
Parents completed the Parenting Practices Questionnaire, a 46-item self-report scale that measures two primary constructs, discipline practices and monitoring. Higher scores represent more effective or better parenting practices.
|
|
Key Findings
|
Among families designated as high risk (those with less adequate parenting skills and family relationship quality at pretest), there was a significantly greater improvement in parental monitoring among those who received the SAFEChildren intervention than among those who did not receive the intervention (p < .05). The slope effect size was very small (Cohen's d = 0.14).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 4: Parental involvement in child's education |
|
Description of Measures
|
Parents completed the Fast Track Parent Involvement Scales, a self-report measure of parental involvement in their child's education that contains three subscales: parent endorsement of school, parent involvement, and quality of relationship with the teacher. Higher scores on these composites indicate greater parental involvement in school.
|
|
Key Findings
|
Among families with high-risk children (having high levels of problem behaviors at pretest), those who received the SAFEChildren intervention showed a slight increase in parental involvement in the child's education, whereas families who did not receive the intervention showed a substantial decrease (p < .05). The slope effect size was very small (Cohen's d = 0.14).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
26-55 (Adult)
|
51% Male
49% Female
|
57.5% Hispanic or Latino
42.5% Black or African American
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Reading achievement
|
4.0
|
4.0
|
3.8
|
3.5
|
3.0
|
3.5
|
3.6
|
|
2: Child problem behaviors
|
4.0
|
4.0
|
3.8
|
3.5
|
3.0
|
3.5
|
3.6
|
|
3: Parenting practices
|
4.0
|
4.0
|
3.8
|
3.5
|
3.0
|
3.5
|
3.6
|
|
4: Parental involvement in child's education
|
4.0
|
4.0
|
3.8
|
3.5
|
3.0
|
3.5
|
3.6
|
Study Strengths The measures used are well established and have good psychometric properties. The methods used to ensure intervention fidelity are noteworthy. When data were missing, this was addressed with growth curve modeling. The retention was good for this type of study. Appropriate analyses were used.
Study Weaknesses The researchers employed an unbalanced design favoring the intervention group (55% were randomly assigned to the intervention condition, and 45% were randomly assigned to the control condition), which somewhat limits the confidence in these findings. The large number of measures employed raises the possibility of colinearity in the absence of a discussion of the statistical power of the multiple measures.
|
|
Readiness for Dissemination
Review Date: October 2007
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Forms and handouts:
- Description of process and fidelity protocol
- Description of training and technical support
- Family Group Sessions: Parent Process Measure
- Family Group Sessions: Process Measure--Interventionist Form
- Family intervention handouts
- Letter-Writing Sheets
- Reading Response Sheet
- Tutoring flash cards
Program Web site, http://www.psych.uic.edu/fcrg/safe.html
SAFEChildren family intervention manual. (2006).
Woo, S., Gorman-Smith, D., Gay, F., Schoeny, M., Tolan, P. H., & Henry, D. B. (n.d.). SAFEChildren tutoring manual. Revised for 7th and 8th grade student tutors. Chicago, IL: Authors.
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.0
|
2.3
|
3.0
|
2.8
|
Dissemination Strengths Implementation materials are thorough, coherent, and easy to follow. The family intervention manual includes specific guidance for family recruitment and engagement. Training and technical assistance are available upon request. Excellent fidelity and process instruments are available to support quality assurance.
Dissemination Weaknesses Little information is provided on organization-level implementation or tutor and leader qualifications. No formal training curriculum has been developed to supplement the intervention materials. No substantive guidance is provided on how new implementation sites use the information collected from quality assurance instruments.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Family intervention manual
|
$50 each
|
Yes
|
|
Tutoring manual
|
$20 each
|
Yes
|
|
Set of reproducible tutoring materials
|
$75 each
|
Yes
|
|
Basic 3-day, on-site training
|
$3,000 per site for up to 10 participants
|
No
|
|
Intensive 5-day, on-site training
|
$6,500 per site for up to 10 participants
|
No
|
|
Initial 1.5-day consultation
|
$2,500 per site
|
No
|
|
Phone consultation
|
$250 per hour
|
No
|
|
Fidelity and process measures
|
Included with manuals
|
No
|
Additional Information Group leaders are usually hired to work half-time at a salary commensurate with a master's of social work and 4-6 years of postdegree experience. College students can serve as tutors; approximately 2 hours should be budgeted for each tutee per week, which allows for both preparation and travel time.
|
Replications
No replications were identified by the developer.
|
|
|
