Quality of Research
Review Date: February 2012
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Clarke, A. M., & Barry, M. M. (2010). An evaluation of the Zippy's Friends emotional wellbeing programme for primary schools in Ireland. Health Promotion Research Centre, National University of Ireland, Galway. Study 2Mishara, B. L., & Ystgaard, M. (2006). Effectiveness of a mental health promotion program to improve coping skills in young children: "Zippy's Friends." Early Childhood Research Quarterly, 21(1), 110-123. Study 3Dufour, S., Denoncourt, J., & Mishara, B. L. (2011). Improving children's adaptation: New evidence regarding the effectiveness of Zippy's Friends, a school mental health promotion program. Advances in School Mental Health Promotion, 4(3), 18-28.
Supplementary Materials Clarke, A. M. (2011). An evaluation of Zippy's Friends, an emotional wellbeing programme for children in primary schools (Unpublished doctoral thesis). National University of Ireland, Galway.
Ryan-Wenger, N. M. (1990). Development and psychometric properties of the Schoolagers' Coping Strategy Inventory. Nursing Research, 39(6), 344-349. 
Outcomes
| Outcome 1: Emotional literacy |
|
Description of Measures
|
Emotional literacy was measured using the Emotional Literacy Checklist, a 20-item instrument with 5 subscales: self-awareness, self-regulation, motivation, empathy, and social skills. Items are rated on a scale from 1 (not at all true) to 4 (very true). Items within each subscale are summed to obtain subscale scores, and an overall emotional literacy score is obtained by summing all 20 items. Higher scores indicate greater emotional literacy. Teachers completed the checklist for each student before and after the intervention.
|
|
Key Findings
|
In a study in Ireland, students who received the intervention showed a significant increase in emotional literacy from pre- to posttest compared with a control group that received the standard, compulsory social, personal, and health curriculum (p < .001).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 2: Hyperactivity |
|
Description of Measures
|
Hyperactivity was measured using:
- Strengths and Difficulties Questionnaire (SDQ). This brief, standardized questionnaire is used to measure children's emotional and behavioral well-being. The questionnaire has 25 items divided into 5 subscales (5 items each): emotional symptoms, conduct problems, hyperactivity/inattention, peer relationship problems, and prosocial behavior. All items are scored on a scale from 0 to 2; lower scores on the hyperactivity/inattention subscale indicate less hyperactivity. Teachers used the SDQ to rate each student before and after the intervention.
- Social Skills Questionnaire Teacher Form (SSQTF), Elementary Level. The questionnaire is used to obtain teacher reports on the frequency of three types of problem behavior: hyperactivity, externalizing, and internalizing. Examples of behavior measured by the hyperactivity subscale include "Interrupts conversations with others" and "Acts impulsively." Assessments occurred before and after the intervention.
|
|
Key Findings
|
In a study in Ireland, students who received the intervention showed a significant decrease in hyperactivity from pre- to posttest compared with a control group that received the standard, compulsory social, personal, and health curriculum (p = .05).
In jointly reported studies conducted in Lithuania and Denmark, Lithuanian students who received the intervention had a significant decrease in hyperactivity from pre- to posttest compared with wait-list control students (p < .001). This outcome was not consistently measured in the Denmark study.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
| Outcome 3: Coping skills |
|
Description of Measures
|
Coping skills were measured using an observation form developed for the study. The form was based on the Schoolagers Coping Strategies Inventory, a self-report instrument with 26 questions that ask children to rate on a scale of 0-3 how often they take various actions when they feel stressed, nervous, or worried, such as "ask someone for help," "hit someone," or "get mad." In the study, teachers used the form to report on children's behavior observed at school. The form asked teachers to describe the most important conflict or problem the child had recently experienced. Teachers were further asked to describe how the child behaved in that situation and to indicate which of the 26 items of the Schoolagers Coping Strategies Inventory the child used in that situation. The situations were categorized by content, and the frequencies of the individual coping skills were examined individually and together. Teachers completed the form for each student before and after the intervention.
|
|
Key Findings
|
In jointly reported studies conducted in Lithuania and Denmark, Lithuanian students who received the intervention had a significant increase in coping strategies compared with wait-list control students (p < .001). This outcome was not consistently measured in the Denmark study.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
2.3
(0.0-4.0 scale)
|
| Outcome 4: Social skills |
|
Description of Measures
|
Social skills were measured using:
- Emotional Literacy Checklist. This 20-item checklist includes 5 subscales that measure various dimensions of emotional literacy: self-awareness, self-regulation, motivation, empathy, and social skills. Each item is rated from 1 (not at all true) to 4 (very true). Items within each subscale are summed to obtain subscale scores, and an overall emotional literacy score is obtained by summing all 20 items. Higher scores indicate greater emotional literacy. This outcome was measured using scores on the social skills subscale. Teachers completed the checklist for each student before and after the intervention.
- Social Skills Questionnaire Teacher Form, (SSQTF), Elementary Level. This questionnaire involves teacher ratings of the frequency of various observed behaviors. Subscale scores were calculated for three dimensions of social skills: cooperation, assertion, and self-control. Examples of behaviors specified in the form are "Attends to your instructions," "Easily makes transition from one classroom activity to another" (cooperation); "Joins ongoing activity or group without being told to do so," "Invites others to join in activities" (assertion); and "Controls temper in conflict situations with peers," "Receives criticism well" (self-control). Assessments occurred before and after the intervention.
- Socio-Emotional Profile. This instrument includes 80 items measuring social competencies and adaptation problems on a 6-point scale (from never to always). Three of the 12 subscales in the Socio-Emotional Profile address social skills: cooperation with adults, autonomy, and internalization of behaviors. Examples of items include "helping teachers" (cooperation); "perseverance in solving their problems themselves" (autonomy); and "anxious behaviors, dependency, nervousness and symptoms of depression such as crying without any reason" (internalization of behaviors). Teachers completed the full Socio-Emotional Profile to rate students before and after the intervention.
|
|
Key Findings
|
In a study in Ireland, students who received the intervention showed significant improvement in social skills from pre- to posttest compared with a control group that received the standard, compulsory social, personal, and health curriculum, after controlling for baseline scores (p = .01)
In jointly reported studies conducted in Lithuania and Denmark, Lithuanian students who received the intervention had significantly greater improvement in social skills from pre- to posttest compared with wait-list control students (p < .001). Danish students who received the intervention had significantly greater improvement in social skills from pre- to posttest compared with historical data for students tracked the year prior to program implementation (p < .001).
In a Canadian study, students who received the intervention showed significant pre- to posttest improvements in cooperation with adults (p < .001), autonomy (p < .02), and internalization of behaviors (p = .001) compared with peers in a no-intervention control group.
|
|
Studies Measuring Outcome
|
Study 1, Study 2, Study 3
|
|
Study Designs
|
Experimental, Quasi-experimental
|
|
Quality of Research Rating
|
2.6
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
0-5 (Early childhood)
6-12 (Childhood)
|
52.3% Male
47.7% Female
|
100% Non-U.S. population
|
|
Study 2
|
0-5 (Early childhood)
6-12 (Childhood)
|
51.3% Male
48.7% Female
|
100% Non-U.S. population
|
|
Study 3
|
0-5 (Early childhood)
6-12 (Childhood)
|
52.9% Male
47.1% Female
|
100% Non-U.S. population
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Emotional literacy
|
3.0
|
2.0
|
2.0
|
3.0
|
3.0
|
3.5
|
2.8
|
|
2: Hyperactivity
|
3.3
|
2.0
|
2.0
|
2.5
|
2.8
|
3.3
|
2.6
|
|
3: Coping skills
|
3.0
|
2.0
|
2.0
|
1.5
|
2.5
|
3.0
|
2.3
|
|
4: Social skills
|
2.5
|
2.0
|
2.0
|
3.8
|
2.5
|
3.0
|
2.6
|
Study Strengths The research design of one study used random assignment. The measures have acceptable and demonstrated reliability and at least face validity. Information on intervention fidelity collected by the investigators generally showed that the program was implemented as planned, even when teachers had the freedom to do otherwise. Missing data and attrition were well described and within an acceptable range given the nature of the study. Some problems with confounding variables were present, but on the whole, these were handled appropriately in the statistical methods. The analytic methodology used by the investigators was appropriate overall.
Study Weaknesses Validity data were not explicitly presented for most measures, and where validity data were reported, the measure had been modified for use in the study. In the jointly reported Danish and Lithuanian studies, collection of data for the program evaluation was not carried out as intended in one of the two countries. Outcomes were reported using teacher ratings only, even for measures originally intended to collect student ratings. One study reported only one-tailed p values.
|
|
Readiness for Dissemination
Review Date: February 2012
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Bates Wells & Braithwaite London LLP. (2006). Partnership for Children license agreement.
Partnership for Children. (2004). Zippy's Friends training workshop agenda.
Partnership for Children. (2006). Training for trainers schedule for YCS visitors, 10-13 December 2006.
Partnership for Children. (2007). Illustrations & annexes for Module 1.
Partnership for Children. (2007). Illustrations & annexes for Module 2.
Partnership for Children. (2007). Illustrations & annexes for Module 3.
Partnership for Children. (2007). Illustrations & annexes for Module 4.
Partnership for Children. (2007). Illustrations & annexes for Module 5.
Partnership for Children. (2007). Illustrations & annexes for Module 6.
Partnership for Children. (2007). Training for teachers training schedule for YCS office, East Orange, NJ, 25-26 January 2007.
Partnership for Children. (2008). Good books for tough times: Books for children aged 5-8. Surrey, England: Author.
Partnership for Children. (2009). Zippy's Friends International Workshop 2009 programme.
Partnership for Children. (2009). Evaluation sheet. Zippy's Friends International Workshop 2009: Summary of delegates' feedback scores.
Partnership for Children. (2010). A review of 2009-2010. Surrey, England: Author.
Partnership for Children. (2010). Module 1: Feelings. Surrey, England: Author.
Partnership for Children. (2010). Module 2: Communication. Surrey, England: Author.
Partnership for Children. (2010). Module 3: Making & breaking relationships. Surrey, England: Author.
Partnership for Children. (2010). Module 4: Conflict resolution. Surrey, England: Author.
Partnership for Children. (2010). Module 5: Dealing with change and loss. Surrey, England: Author.
Partnership for Children. (2010). Module 6: We cope. Surrey, England: Author.
Partnership for Children. (2010). Notes for teachers. Surrey, England: Author.
Partnership for Children. (2010). Recent evaluation of Zippy's Friends [PowerPoint slides].
Partnership for Children. (2011). Good books for tough times: Books for children aged 9-12. Surrey, England: Author.
Partnership for Children. (2011, July). Good mental health for children--for life [Newsletter].
Partnership for Children. (n.d.). A parent's guide to Zippy's Friends. Surrey, England: Author.
Partnership for Children. (n.d.). Trainers CD [CD-ROM]. Surrey, England: Author.
Program Web site, http://www.partnershipforchildren.org.uk/zippy-s-friends.html
Other program materials:
- Activities for Parents: Helping Your Child To Cope With Difficult Situations
- An Introduction to Zippy's Friends [DVD]
- Copyright Acknowledgment Form
- Evaluation sheet for Zippy's Friends initial training course
- 40 Ideas From Around the World
- Goals of the Modules
- Logo and PowerPoint presentation of the illustrations and rules for modules 1-6 of the Zippy's Friends teaching materials [CD-ROM]
- Principles for Implementing the Activities
- Setting Up Zippy's Friends With a New Partner Organisation
- Songs for Zippy's Friends
- Support and Follow-Up of Zippy's Teachers
- Training Certificate
- Training Video [DVD]
- Using Zippy's Friends and the SEAL materials
- Zippy's Friends Coordinator Role Guidelines
- Zippy's Friends Criteria for Selecting Partner Agencies
- Zippy's Friends Module Report
- Zippy's Friends Programmes Map
- Zippy's Friends Support Meeting, Questionnaires 1-3
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
4.0
|
3.5
|
3.3
|
3.6
|
Dissemination Strengths Teacher materials are comprehensive and easy to understand and use. Multiple resources are available for parents online and in hard copy, including activities to support classroom learning. The developer also recommends additional resources for parents and teachers to extend learning. A standardized training for teachers includes skill practice and classroom observation, with accompanying PowerPoint presentations for the individual modules. Train-the-trainer events and international workshops are also available. On-site consultation is available during the first year of implementation, and phone and email support can be requested beyond the first year. Fidelity monitoring is supported through checklists and classroom observation by the site coordinator. A licensing agreement clearly outlines the responsibilities of implementing sites. Annual reports are submitted to the developer to facilitate quality improvements.
Dissemination Weaknesses It is unclear what degree of support or coaching the licensed partner organizations provide to sites after initial implementation, other than periodic visits to monitor fidelity. In addition, although partner organizations can contact the developer (based in the United Kingdom) by phone and email for support, it is not clear what that assistance might entail. No standardized protocol for outcome data collection or analysis is provided to sites.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Licensing fee
|
Varies according to duration and scope of license
|
Yes
|
|
Implementation materials (includes one teachers' folder, one set of children's items, and a set of eight posters)
|
$390 per set
|
Yes
|
|
Additional teachers' folders
|
$300 each
|
No
|
|
Additional set of eight posters
|
$40 per set
|
No
|
|
Additional set of children's items (includes 60 paper puppets, 30 paper crowns, pencils, cards, certificates, and parent guides)
|
$75 per set
|
No
|
|
1- or 2-day teacher training course
|
$3,100 for up to 25 participants, plus travel costs
|
Yes
|
|
3-day training for trainers
|
$3,100 for 2-25 participants, plus travel costs
|
Yes
|
|
On-site support
|
$930 per day, plus travel costs
|
No
|
|
Phone and email support
|
Free
|
No
|
|
Partner Organization Annual Report Form
|
Free
|
Yes
|
|
Module Report Form and questionnaires for follow-up meetings
|
Included with training materials
|
No
|
Additional Information Training and support costs shown here are based on standard charges for two Partnership for Children staff providing training or support within the United Kingdom. Costs for new implementers in other countries will vary depending on the site's particular needs and location.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
* Clarke, A. M., & Barry, M. M. (2010). An evaluation of the Zippy's Friends emotional wellbeing programme for primary schools in Ireland. Health Promotion Research Centre, National University of Ireland, Galway.
* Dufour, S., Denoncourt, J., & Mishara, B. L. (2011). Improving children's adaptation: New evidence regarding the effectiveness of Zippy's Friends, a school mental health promotion program. Advances in School Mental Health Promotion, 4(3), 18-28.
Holen, S., Waaktaar, T., Lervag A., & Ystgaard, M. (2012). Implementing a universal stress management program for young school children: Are there classroom climate or academic effects? Scandinavian Journal of Educational Research. DOI:10.1080/00313831.2012.656320 (http://dx.doi.org/10.1080/00313831.2012.656320) Holen, S., Waaktaar, T., Lervag, A., & Ystgaard, M. (2012). The effectiveness of a universal school-based program on coping and mental health: A randomized control study of Zippy's Friends. Educational Psychology, 32(5), 657-677.
* Mishara, B. L., & Ystgaard, M. (2006). Effectiveness of a mental health promotion program to improve coping skills in young children: "Zippy's Friends." Early Childhood Research Quarterly, 21(1), 110-123.
Monkeviciene, O., Mishara, B. L., & Dufour, S. (2006). Effects of the Zippy's Friends programme on children's coping abilities during the transition from kindergarten to elementary school. Early Childhood Education Journal, 34(1), 53-60.
Rowley, G., & Cook, J. (2007). Zippy's Friends: Developing curriculum resources to support the mental health needs of young people with special educational needs. In B. Carpenter & J. Egerton (Eds.), New horizons in special education: Evidence-based practice in action. Clent, Worcestershire, England: Sunfield.
Wong, M. (2008). Helping young children to develop adaptive coping strategies. Journal of Basic Education, 17(1).
|
|
|
