Quality of Research
Review Date: February 2010
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Pilgrim, C., Abbey, A., Hendrickson, P., & Lorenz, S. (1998). Implementation and impact of a family-based substance abuse prevention program in rural communities. Journal of Primary Prevention, 18(3), 341-361. Study 2Abbey, A., Pilgrim, C., Hendrickson, P., & Buresh, S. (2000). Evaluation of a family-based substance abuse prevention program targeted for the middle school years. Journal of Drug Education, 30(2), 213-228. 
Outcomes
| Outcome 1: Positive attachment to family, school, and peers |
|
Description of Measures
|
Students and parents completed a self-report survey that assessed the following:
- Family cohesion, measured with the 9-item Cohesion subscale from the Family Environment Scale. This scale assesses the degree of commitment, help, and support family members provide one another. The scale has a true/false response option. The items were averaged to obtain one family cohesion score for the student and one for the parent.
- Family fighting, measured with a 4-item scale developed for this study. A sample item is "How many times have you yelled at your child (parent)?" Scores were calculated separately for the student and the parent.
- School attachment, measured with the 10-item Attachment to School subscale from the Effective School Battery. This subscale uses a 2-point response option and assesses whether respondents "like" or "don't like" the student's school, teachers, principal, counselors, and classes. The items were averaged to obtain one school attachment score for the student and one for the parent.
- Participation in school activities, measured by asking respondents whether they are involved in different activities at the child's school (e.g., member of a club or team, attended a PTA meeting). Using a yes/no response format, students reported on their involvement in three school activities, and parents reported on five school activities. One average score was computed for the student and one for the parent.
- Students' peer attachment (completed by students only), measured with a 15-item subset of the Inventory of Peer Attachment, which uses a true/false response scale to assess perceptions of friends' supportiveness. Items were averaged to obtain an overall peer attachment score.
|
|
Key Findings
|
In one study, short-term effects of program participation were examined by comparing pretest, posttest, and 10-week follow-up data, which were collected only for intervention participants. Long-term effects were examined by comparing the intervention and control groups at 1-year follow-up. A second study was conducted with a new cohort of students and parents 1 year later. Long-term effects were examined by comparing the intervention and control groups at 1-year follow-up. Findings from these studies included the following:
- Family cohesion: In the first study, parents who participated in the intervention reported significantly greater family cohesion at posttest than they did at pretest (p < .006). This effect was not significant at the 10-week follow-up. In the second study, students who received the intervention reported greater family cohesion than students in the control group at 1-year follow-up (p = .03).
- Family fighting: In the second study, students receiving the intervention reported less family fighting at the 1-year follow-up than students in the control group (p = .002). This effect was not significant for parents.
- School attachment: In the first study, male students who received the intervention scored significantly higher than did male students in the control group on school attachment at 1-year follow-up (p < .03). This effect was not significant for female students. However, the second study found that students receiving the intervention reported greater school attachment at the 1-year follow-up than students in the control group (p = .01). No significant gender differences were found in this study. No significant differences for parents were found in either study.
- Participation in school activities: In the first study, parents receiving the intervention reported more involvement in school activities at 1-year follow-up than did nonparticipating parents (p < .002). There were no significant differences for students.
- Peer attachment: In the first study, students receiving the intervention reported significantly greater peer attachment at posttest than at pretest (p < .04). This effect was not significant at the 10-week follow-up. However, male students who received the intervention reported significantly greater peer attachment than male nonparticipants at 1-year follow-up (p < .05). This long-term program effect was not significant for female students.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
| Outcome 2: Participation in counseling |
|
Description of Measures
|
Participation in counseling was measured by a 3-item self-report scale developed for this study to assess whether the student or parent had talked with a psychologist, social worker, or school counselor. Responses were averaged to obtain one overall score for the student and one for the parent.
|
|
Key Findings
|
Students (p < .004) and parents (p < .001) who participated in the intervention reported more involvement in family counseling at 1-year follow-up compared with their counterparts in the control group, after controlling for baseline scores.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.2
(0.0-4.0 scale)
|
| Outcome 3: Attitudes toward alcohol use |
|
Description of Measures
|
To assess attitudes toward adolescent alcohol use, a 5-item alcohol attitudes scale was created by adapting items from the Parents scale in the Program Evaluation Handbook: Drug Abuse Education. The scale uses a 4-point response option ranging from "definitely yes" to "definitely no." Questions for students are phrased in terms of their friends (e.g., "Would you be upset if your friend took you to a party where alcohol was being used?"). Parents answered parallel items about their child's use of alcohol (e.g., "Would you be upset if your teenager got drunk on a special occasion like a graduation party or New Year's Eve?"). Students and parents were also asked, "What age do you think that it is O.K. to drink more than a sip of alcohol?"
|
|
Key Findings
|
In one study, short-term effects of program participation were examined by comparing pretest, posttest, and 10-week follow-up data, which were collected only for intervention participants. Long-term effects were examined by comparing the intervention and control groups at 1-year follow-up. A second study was conducted with a new cohort of students and parents 1 year later. Long-term effects were examined by comparing the intervention and control groups at 1-year follow-up. Findings from these studies included the following:
- Opposition to adolescent alcohol use: In the first study, at 1-year follow-up, male students who received the intervention reported significantly more opposition to adolescent alcohol use than did male students in the control group (p < .003). These program effects were not significant among female students. In the second study, parent participants, as compared with parents in the control group, reported stronger opposition to adolescent alcohol use (p = .04).
- Age at which it is "O.K." to drink alcohol: In the first study, the mean acceptable drinking age (in years) according to parents receiving the intervention increased from pretest to posttest (20.10 vs. 20.55; p < .02) and from pretest to 10-week follow-up (20.10 vs. 21.09; p < .04). At 1-year follow-up, the mean acceptable drinking age reported by male students receiving the intervention was higher than the age reported by male students in the control group (19.96 vs. 16.27; p < .03). No significant difference was seen among female students. In the second study, at 1-year follow-up, intervention group students reported a higher mean age than control group students (18.08 vs. 17.16; p = .04), and intervention group parents reported a higher mean age than control group parents (20.84 vs. 19.82; p = .05). No significant gender differences were found in this study.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
| Outcome 4: Self-esteem |
|
Description of Measures
|
Students' self-esteem was measured with the 16-item self-report Behavior subscale from the Piers-Harris Children's Self-Concept Scale. This subscale, which uses a yes/no response option, measures positive self-esteem regarding how well one is handling responsibilities at home and at school. Responses are averaged to obtain an overall self-esteem score.
|
|
Key Findings
|
Students who participated in the intervention reported greater self-esteem at 1-year follow-up compared with students in the control group (p = .003), after controlling for baseline scores.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.7
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
13-17 (Adolescent)
26-55 (Adult)
|
52.7% Female
47.3% Male
|
Data not reported/available
|
|
Study 2
|
6-12 (Childhood)
13-17 (Adolescent)
26-55 (Adult)
|
52% Male
48% Female
|
Data not reported/available
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Positive attachment to family, school, and peers
|
3.6
|
3.3
|
1.8
|
2.0
|
2.0
|
3.0
|
2.6
|
|
2: Participation in counseling
|
2.0
|
2.0
|
2.0
|
2.0
|
2.0
|
3.0
|
2.2
|
|
3: Attitudes toward alcohol use
|
3.8
|
3.3
|
1.8
|
2.0
|
2.0
|
2.8
|
2.6
|
|
4: Self-esteem
|
4.0
|
4.0
|
1.5
|
2.0
|
2.0
|
2.5
|
2.7
|
Study Strengths The measures in both studies were appropriate to the goals and hypotheses for the program. In general, the studies used established measures with known psychometric properties. In addition, reliability estimates were presented for each measure, including those developed by the researchers, where applicable. The scales created by the researchers have face validity. The researchers attempted to compensate for initial differences found between participants and nonparticipants by conducting ANCOVAs, using as covariates the students' and parents' demographic characteristics and baseline outcome measures for which there were differences.
Study Weaknesses Fidelity of implementation was not described in sufficient detail to demonstrate that the program was implemented as intended. The high attrition rate, though not uncommon in school-based intervention research, was considerable (greater than 20%) in both studies. These quasi-experimental studies compared volunteer families with families that elected not to participate, thus creating potential selection bias; in fact, a number of significant baseline differences were found, including socioeconomic status, which was lower among participants than nonparticipants. Because participants completed questionnaires more frequently than did nonparticipants, testing effects are also a potential confounding factor. The group sizes for the student intervention groups (all less than 50) raise serious questions about statistical power.
|
|
Readiness for Dissemination
Review Date: February 2010
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Active Parenting Publishers. (2002). Active Parenting Now & Active Parenting of Teens training video [DVD]. Atlanta, GA: Author.
Active Parenting Publishers. (2009). Training of trainers seminar. Atlanta, GA: Author.
Active Parenting Publishers. (n.d.). Active Parenting of Teens: Families in Action quality assurance protocol and handouts. Atlanta, GA: Author.
Active Parenting Publishers. (n.d.). Leader training workshop evaluation form. Atlanta, GA: Author.
Popkin, M. H. (2002). Leader training workshop participant's guide for Active Parenting Now and Active Parenting of Teens. Atlanta, GA: Active Parenting Publishers.
Popkin, M. H., & Hendrickson, P. (2002). Active Parenting of Teens: Families in Action for parents program kit. Atlanta, GA: Active Parenting Publishers.
Popkin, M. H., & Hendrickson, P. (2002). Active Parenting of Teens: Families in Action parents-only materials kit. Atlanta, GA: Active Parenting Publishers.
Popkin, M. H., & Hendrickson, P. (2002). Active Parenting of Teens: Families in Action teen edition program kit. Atlanta, GA: Active Parenting Publishers.
Program Web site, http://www.activeparenting.com
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.9
|
3.4
|
3.2
|
3.5
|
Dissemination Strengths The materials provided to guide implementation are thorough, consistent, and well presented. Instructions for preparing to implement the program are detailed and logical, and guidance on recruiting appropriate program participants is excellent. The developer offers a variety of onsite trainings for leading the intervention with teens only, parents only, or parents and teens, and also offers train-the-trainer opportunities. Some trainings are also available in Spanish. A comprehensive set of fidelity tools, outcome measures, and data collection instruments support quality assurance.
Dissemination Weaknesses The skills and qualifications needed to serve as a program leader are not fully described. There is no formalized curriculum for program leader training or the training of trainers; only an explanation of implementation materials is available. Quality assurance tools are not an integrated or emphasized component of implementation and training. No information is provided on how data collected should be used to improve program delivery.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Program Kit
|
$549 each
|
Yes
|
|
Jump-Start Package (includes Program Kit plus 15 additional copies each of the Parent's Guide, Teen's Guide, and parent and teen completion certificates, as well as PowerPoint presentations for use with parents, teens, and parents and teens combined)
|
$898 each
|
No
|
|
Additional Parent's Guide
|
$14.95 per parent
|
Yes
|
|
Additional Teen's Guide
|
$13.95 per teen
|
Yes
|
|
Additional completion certificates
|
$6 for 15
|
No
|
|
Parent handouts
|
$24 per set (reproducible)
|
No
|
|
PowerPoint presentation
|
$49 each
|
No
|
|
2-day, on-site leader training
|
$205 per participant for groups of 12 or more, plus travel expenses
|
No
|
|
Technical assistance/consultation
|
Free
|
No
|
|
Quality assurance materials
|
Free
|
No
|
Additional Information Quantity discounts are available for program guides.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
Chen, M. (2006). Active Parenting Now and Active Parenting of Teens national field studies.
CSR, Incorporated. (1994). Center for Substance Abuse Prevention High Risk Youth Demonstration Grant Program. CSR cross-site evaluation: Families in Action.
Leonardson, G. (1991). Draft report on Active Parenting of Teens project. Watertown, SD: Northeastern Drug and Alcohol Prevention Resource Center.
Mullis, F. (1999). Active Parenting: An evaluation of two Adlerian parent education programs. Journal of Individual Psychology, 55(2), 225-232.
National Prevention Implementation Program. (1989). Parenting as prevention: Preventing alcohol and other drug use problems in the family. Prepared for the Office for Substance Abuse Prevention of the Alcohol, Drug Abuse, and Mental Health Administration, U.S. Department of Health and Human Services.
Popkin, M. H. (1989). Active Parenting: A video-based program. In M. Fine (Ed.), The second handbook on parent education: Contemporary perspectives (pp. 77-98). San Diego, CA: Academic Press.
|
|
|
