Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Gross, D., Garvey, C., Julion, W., Fogg, L., Tucker, S., & Mokros, H. (2009). Efficacy of the Chicago Parent Program with low-income African American and Latino parents of young children. Prevention Science, 10(1), 54-65.  Study 2Gross, D., Breitenstein, S., Julion, W., Garvey, C., Ridge, A., & Fogg, L. (2012). Study 2: Chicago Parent Program replication report.
Supplementary Materials Breitenstein, S. M., Fogg, L., Garvey, C., Hill, C., Resnick, B., & Gross, D. (2010). Measuring implementation fidelity in a community-based parenting intervention. Nursing Research, 59(3), 158-165. 
Supporting Document for the Reliability and Validity of Outcome Measures: Chicago Parent Program
Outcomes
| Outcome 1: Child behavior problems |
|
Description of Measures
|
Child behavior problems were assessed using two measures: the Eyberg Child Behavior Inventory (ECBI) and the Dyadic Parent-Child Interactive Coding System--Revised (DPICS-R). The ECBI is a 36-item survey completed by parents of children 2-16 years old. Each item presents a behavior that is measured along two scales: the Intensity Scale and the Problem Scale. On the Intensity Scale, the parent indicates the frequency of the child behavior on a scale of 1 (the behavior never happens) to 7 (the behavior is always happening). On the Problem Scale, the parent indicates whether he or she considers the child behavior to be a problem ("yes" or "no"). The DPICS-R was used by researchers to code the frequency of seven child behaviors observed in videotaped play and clean-up sessions. Child behavior problems were measured by the summed frequencies of the following behaviors: noncompliance, destructive behavior, physically negative behavior, crying, whining, yelling, and "smart talk." Assessments were conducted at baseline, at the conclusion of the intervention (3 months after baseline), and at 6 months and 1 year after the intervention.
|
|
Key Findings
|
Daycare centers matched on multiple criteria were randomly assigned to the intervention group or a no-intervention control group. Over time, from baseline through 1-year follow-up, children of intervention group parents exhibited fewer behavior problems during play sessions (p < .01) and clean-up sessions (p < .01) than children of control group parents, findings that had small effect sizes (Cohen's d = 0.43 for play sessions and Cohen's d = 0.46 for clean-up sessions). A second set of analyses compared children of parents with different levels of exposure to the intervention: (1) those with a high dose of the intervention (i.e., attended at least 6 the 11 core sessions), (2) those with a low dose of the intervention (i.e., attended less than 6 of the 11 core sessions), and (3) those in the control group. Over time, from baseline through 1-year follow-up, the high-dose intervention group had greater improvement in behavior problems compared with the two other groups as measured by the ECBI Intensity Scale (p < .05) and DPICS-R (p < .001 for play sessions and clean-up sessions). The effect sizes for these findings were small to medium (Cohen's d = 0.31 for the ECBI Intensity Scale, Cohen's d = 0.64 for DPICS-R during play sessions, and Cohen's d = 0.46 for DPICS-R during clean-up sessions).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 2: Parenting self-efficacy |
|
Description of Measures
|
Parenting self-efficacy was assessed using the Toddler Care Questionnaire (TCQ). This 38-item scale measures parenting confidence in managing tasks and situations pertinent to raising young children (e.g., toilet training, setting limits in destructive behavior). Scores range from 38 to 190, with higher scores indicating greater parenting confidence. Assessments were conducted at baseline, at the conclusion of the intervention (3 months after baseline), and at 6 months and 1 year after the intervention.
|
|
Key Findings
|
In one study, daycare centers were randomly assigned to the intervention group or a no-intervention control group. Over time, from baseline through 1-year follow-up, no significant difference was found between intervention group and control group parents in parenting self-efficacy. A second set of analyses compared parents with different levels of exposure to the intervention: (1) those with a high dose of the intervention (i.e., attended at least 6 of the 11 core sessions), (2) those with a low dose of the intervention (i.e., attended less than 6 of the 11 core sessions), and (3) those in the control group. Over time, from baseline through 1-year follow-up, the high-dose intervention group had greater improvement in parenting self-efficacy compared with the two other groups (p < .05). This finding had a small effect size (Cohen's d = 0.37).
In another study, daycare centers were randomly assigned to the intervention group or a no-intervention control group. Over time, from baseline through 1-year follow-up, parents in the intervention group reported greater parenting self-efficacy than parents in the control group (p < .01).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.5
(0.0-4.0 scale)
|
| Outcome 3: Corporal punishment |
|
Description of Measures
|
Corporal punishment was measured using the Corporal Punishment subscale of the Parenting Questionnaire (PQ). This subscale contains 4 items measuring parent use of corporal punishment in response to a child's misbehavior. Parents rated each item on a scale from 1 (almost never) to 5 (very often). Assessments were conducted at baseline, at the conclusion of the intervention (3 months after baseline), and at 6 months and 1 year after the intervention.
|
|
Key Findings
|
In one study, daycare centers matched on multiple criteria were randomly assigned to the intervention group or a no-intervention control group. Over time, from baseline through 1-year follow-up, intervention group parents used less corporal punishment than control group parents (p < .05), a finding with a small effect size (Cohen's d = 0.24). A second set of analyses compared parents with different levels of exposure to the intervention: (1) those with a high dose of the intervention (i.e., attended at least 6 of the 11 core sessions), (2) those with a low dose of the intervention (i.e., attended less than 6 of the 11 core sessions), and (3) those in the control group. Over time, from baseline through 1-year follow-up, the high-dose intervention group had greater decreases in use of corporal punishment compared with the two other groups (p < .01). This finding had a small effect size (Cohen's d = 0.30).
In another study, daycare centers were randomly assigned to the intervention group or a no-intervention control group. Over time, from baseline through 1-year follow-up, parents in the intervention group reported less use of corporal punishment than parents in the control group (p < .001).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
| Outcome 4: Follow-through on discipline |
|
Description of Measures
|
Follow-through on discipline was measured using a modified subscale of the Parenting Questionnaire (PQ). Researchers selected 6 items from the PQ's 23-item Demandingness subscale to create the Follow Through on Discipline subscale. The selected items measure the parent's ability to set and maintain a standard of behavior for his or her toddler (e.g., "My child can talk me into letting him/her off easier than I had intended," "My child convinces me to change my mind after I have refused a request," "I threaten punishment but do not end up punishing my child"). Parents rated each item on a scale from 1 (almost never) to 5 (very often). Assessments were conducted at baseline, at the conclusion of the intervention (3 months after baseline), and at 6 months and 1 year after the intervention.
|
|
Key Findings
|
In one study, daycare centers matched on multiple criteria were randomly assigned to the intervention group or a no-intervention control group. Over time, from baseline through 1-year follow-up, no difference was found between intervention group and control group parents in follow-through on discipline. A second set of analyses compared parents with different levels of exposure to the intervention: (1) those with a high dose of the intervention (i.e., attended at least 6 of the 11 core sessions), (2) those with a low dose of the intervention (i.e., attended less than 6 of the 11 core sessions), and (3) those in the control group. Over time, from baseline through 1-year follow-up, the high-dose intervention group had greater improvement in follow-through on discipline compared with the two other groups (p < .01). This finding had a small effect size (Cohen's d = 0.29).
In another study, daycare centers were randomly assigned to the intervention group or a no-intervention control group. Over time, from baseline through 1-year follow-up, parents in the intervention group reported greater follow-through on discipline than parents in the control group (p < .01).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.4
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
0-5 (Early childhood) 18-25 (Young adult) 26-55 (Adult)
|
56% Male 44% Female
|
58.9% Black or African American 32.8% Hispanic or Latino 4.7% White 3.6% Race/ethnicity unspecified
|
|
Study 2
|
18-25 (Young adult) 26-55 (Adult)
|
52% Male 48% Female
|
55% Black or African American 45% Hispanic or Latino
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Child behavior problems
|
3.5
|
3.5
|
3.0
|
3.8
|
3.3
|
3.3
|
3.4
|
|
2: Parenting self-efficacy
|
3.5
|
3.5
|
3.5
|
3.8
|
3.3
|
3.5
|
3.5
|
|
3: Corporal punishment
|
3.3
|
3.3
|
3.5
|
3.8
|
3.3
|
3.5
|
3.4
|
|
4: Follow-through on discipline
|
3.3
|
3.3
|
3.5
|
3.8
|
3.3
|
3.5
|
3.4
|
Study Strengths The ECBI, DPICS-R, TCQ, and PQ are gold-standard instruments widely used with diverse populations. A parent advisory council reflective of the study samples' demographics reviewed the measures and provided feedback that was incorporated into the intervention and protocols. Multiple measures were used to assess changes in child behavior. Program implementation was monitored using a validated fidelity checklist, adherence scales, random observations, and review of videotaped sessions. Attrition and missing data were minimal, and attrition rates were similar in the control and intervention groups. The sample size and power were sufficient for statistical analysis. A number of well-defined statistical procedures were conducted to determine program impact.
Study Weaknesses In both studies, some potential confounding variables could have affected study findings. In one study, for example, the parents in the control group were more likely to be employed and to have a higher income than the parents in the intervention group.
|
|
Readiness for Dissemination
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Breitenstein, S. (2009). The fidelity checklist. Chicago, IL: Author.
Breitenstein, S. (2010). Manual for the fidelity checklist. Chicago, IL: Author.
The Chicago Parent Program. (2002). Unit 1: The value of your attention [DVD]. Chicago, IL: Rush University Medical Center.
The Chicago Parent Program. (2002). Unit 2: Using your authority wisely [DVD]. Chicago, IL: Rush University Medical Center.
The Chicago Parent Program. (2002). Unit 3: Managing your stress [DVD]. Chicago, IL: Rush University Medical Center.
The Chicago Parent Program. (2002). Unit 4: Sticking with the program [DVD]. Chicago, IL: Rush University Medical Center.
Gross, D., Julion, W., Garvey, C., & Breitenstein, S. (2010). The Chicago Parent Program: Group leader manual (2nd ed.). Chicago, IL: Rush University Medical Center.
Training materials:
- Chicago Parent Program: Agenda--Day 1 and 2
- Chicago Parent Program: 8 Keys of Effective Discipline
- Chicago Parent Program: Glossary of Terms
- The Chicago Parent Program: Group Leader Training [PowerPoint slides]
- The Chicago Parent Program: Group Leader Training--Attendance
- Chicago Parent Program: Group Leader Training Evaluation--Day 1 and 2
- Chicago Parent Program: Manual Components
- Chicago Parent Program: Post Assessment Test
- Chicago Parent Program: Self Assessment Survey
- Ground Rules
Other implementation materials:
- Chicago Parent Program Group Leader Certification
- The Chicago Parent Program: Group Leader Facilitation Model
- Chicago Parent Program Group Leader Training Course SurveyMonkey registration
- Chicago Parent Program: 12 Sessions
- To Submit Audio Recordings of CPP Groups for Fidelity Assessment
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.5
|
4.0
|
3.3
|
3.6
|
Dissemination Strengths The group leader manual is well organized and easy to follow. For each parent session, the manual provides detailed instructions for conducting the session, scripted discussion questions, and leader notes describing the content that should be reinforced. The videos depict real families that are representative of a culturally and socioeconomically diverse audience. The 2-day, on-site training required for group leaders is interactive and includes discussion, role-play, and a competency-based assessment. The training includes guidance on participant recruitment, engagement, and barrier reduction. The developer is available for brief technical assistance as well as indepth consultation and coaching. Interested group leaders can become certified CPP educators through a process that includes coaching and supervision. Weekly checklists for group leaders strengthen fidelity. The weekly parent satisfaction survey and homework checklists assess the application of program principals and strategies. The developer also reviews and provides feedback on audiotaped parent sessions to support quality assurance.
Dissemination Weaknesses The program lacks a cohesive document that outlines key implementation information for potential implementers. The weekly parent satisfaction surveys and end-of-program evaluation survey may not be sufficient to assess meaningful parent outcomes. In addition, no guidance is available on how to use the data collected with these instruments for program improvement.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Group leader set, which includes the group leader manual (with weekly group leader checklists, reproducible parent handouts, practice assignments, practice checklists, weekly parent satisfaction survey, and an end-of-program evaluation survey) and set of four DVDs
|
$699 each
|
Yes
|
|
Additional group leader manual
|
$85 each
|
No
|
|
Color-coded parent handouts (double-sided for English and Spanish) for all 12 sessions
|
$20 per set
|
No
|
|
2-day, on-site group leader training
|
- In the Chicago metropolitan area, $2,500 for up to 20 participants, plus travel expenses
- In other cities, $3,000 for up to 20 participants, plus travel expenses
|
Yes
|
|
Brief phone and email technical assistance
|
Free
|
No
|
|
Phone consultation and coaching
|
$70 per hour
|
No
|
|
Fidelity Checklist Competence Scale
|
Included in the group leader manual
|
No
|
|
Review of audiotaped parent sessions using the Fidelity Checklist Competence Scale
|
$175 per parent session
|
No
|
Additional Information Implementers who would like to become certified CPP educators must attend the 2-day group leader training and pass the competency exam, lead at least two CPP groups, and submit audiotapes of at least three sessions and corresponding weekly group leader checklists.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
* Gross, D., Breitenstein, S., Julion, W., Garvey, C., Ridge, A., & Fogg, L. (2012). Study 2: Chicago Parent Program replication report.
Haines, J., Mayorga, A. M., McDonald, J., O'Brien, A., Gross, D., Taveras, E. M., et al. (2012). Embedding weight-related messages within a general parenting programme: Development and feasibility evaluation of Parents and Tots Together. Early Child Development and Care, 182(8), 951-965.
|
|
|