Quality of Research
Review Date: January 2012
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Bell, C. C., Bhana, A., Peterson, I., McKay, M. M., Gibbons, R., Bannon, W., & Amatya, A. (2008). Building protective factors to offset sexually risky behaviors among black youths: A randomized control trial. Journal of the National Medical Association, 100(8), 936-944.  Study 2McKay, M. M., Chasse, K. T., Paikoff, R., McKinney, L., Baptiste, D., Coleman, D., et al. (2004). Family-level impact of the CHAMP Family Program: A community collaborative effort to support urban families and reduce HIV risk exposure. Family Process, 43(1), 79-93. Study 3Bannon, W. M., & McKay, M. M. (2007). Addressing urban African American youth externalizing and social problem behavioral difficulties in a family-oriented prevention project. Social Work in Mental Health, 5(1/2), 217-236.
Outcomes
| Outcome 1: Family communication |
|
Description of Measures
|
In one study, two dimensions of family communication, comfort and frequency, were measured using items adapted from the Family Communication Scale.
In a second study, comfort with communication about AIDS-related issues was measured with a series of questions adapted from the Youth AIDS Prevention Project and the Aban Aya Youth Project. Items address the degree to which youth/caregivers are comfortable with discussing HIV, AIDS, sexuality, and substance abuse. Items are scored on a 4-point scale (from "strongly disagree" to "strongly agree" or "very comfortable" to "uncomfortable").
|
|
Key Findings
|
In a study conducted in South Africa, youth and their adult caregivers were randomly assigned by school to the intervention group, which received CHAMP, or to the control group, which received standard school programming. From pre- to posttest, the change in caregivers' comfort with family communication and the frequency of communication was significantly greater among those receiving the intervention (p = .0021 and p = .0412, respectively). These differences were associated with small effect sizes (Cohen's d = 0.4 and 0.2, respectively), based on parameter estimates from mixed-effects regression models.
In a single-group study conducted in the United States, caregivers participating in CHAMP showed a significant pre- to posttest improvement in their comfort with family communication (p < .01).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 2: Knowledge about HIV transmission |
|
Description of Measures
|
Knowledge about HIV transmission was measured with a series of questions adapted from the Youth AIDS Prevention Project and the Aban Aya Youth Project. Items address the degree to which respondents understand how HIV is caused and transmitted. Items are scored on a 3-point scale (as "true," "false," and "not sure").
|
|
Key Findings
|
In a study conducted in South Africa, youth and their adult caregivers were randomly assigned by school to the intervention group, which received CHAMP, or to the control group, which received standard school programming. From pre- to posttest, the change in caregivers' knowledge about HIV transmission was significantly greater among those in the intervention group (p = .0084). This difference was associated with a medium effect size (Cohen's d = 0.6), based on parameter estimates from mixed-effects regression models.
In a single-group study conducted in the United States, youth and caregivers participating in CHAMP showed a significant pre- to posttest increase in their knowledge about HIV transmission (p < .01).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental, Preexperimental
|
|
Quality of Research Rating
|
2.7
(0.0-4.0 scale)
|
| Outcome 3: Perceived stigma of HIV/AIDS |
|
Description of Measures
|
Perceived stigma of HIV/AIDS was measured with a series of questions adapted from the Youth AIDS Prevention Project and the Aban Aya Youth Project. Adult and child responses are coded differently, with low scores for adults and high scores for children indicating low levels of perceived stigma.
|
|
Key Findings
|
In a study conducted in South Africa, youth and their adult caregivers were randomly assigned by school to the intervention group, which received CHAMP, or to the control group, which received standard school programming. From pre- to posttest, the change in the perceived stigma of HIV was significantly greater among caregivers and youth in the intervention group (p = .0817 and p = .0045, respectively). These differences were associated with small and medium effect sizes (Cohen's d = 0.4 and 0.6, respectively), based on parameter estimates from mixed-effects regression models.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
| Outcome 4: Externalizing behavior |
|
Description of Measures
|
Externalizing behavior among youth participants was assessed using the Child Behavior Checklist (CBC-L/6-18), a widely used 112-item questionnaire in which eight measured characteristics are combined to produce two scales for internalizing and externalizing behavior. The checklist was administered to the parents/caregivers of youth participating in the program.
|
|
Key Findings
|
In a single-group study conducted in the United States, youth participating in CHAMPS showed a significant pre- to posttest improvement in externalizing behavior (p < .05).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Preexperimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
|
59.1% Female
40.9% Male
|
100% Non-U.S. population
|
|
Study 2
|
6-12 (Childhood)
|
60.2% Female
39.8% Male
|
99% Black or African American
1% Non-U.S. population
|
|
Study 3
|
6-12 (Childhood)
|
37.6% Female
24.8% Male
|
99% Black or African American
1% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Family communication
|
2.9
|
2.2
|
3.2
|
2.9
|
2.8
|
2.8
|
2.8
|
|
2: Knowledge about HIV transmission
|
2.4
|
2.2
|
3.2
|
2.9
|
2.9
|
2.8
|
2.7
|
|
3: Perceived stigma of HIV/AIDS
|
2.9
|
2.4
|
3.2
|
3.3
|
3.3
|
3.8
|
3.1
|
|
4: Externalizing behavior
|
3.8
|
3.3
|
2.9
|
2.0
|
2.3
|
2.4
|
2.8
|
Study Strengths Reliability and validity were enhanced by the studies' use of portions of a measure that had prior study-based testing. The investigators made careful efforts to ensure intervention fidelity, both by putting mechanisms in place to train the implementers and by conducting regular follow-ups. Study population sizes were robust, and in two of three studies reviewed, attrition was very low. The most rigorous of the three studies used a quasi-experimental design, which helped to minimize the effects of potential confounding variables.
Study Weaknesses Attrition in one of the three studies was quite high, approaching 50%. Although the steps taken to maximize intervention fidelity are described in some detail, evidence of the success of these steps is not provided. Two of the studies employed a preexperimental design that did not adequately control for potential confounding factors. Analytical procedures are not described in detail in all the studies reviewed, requiring some inferences to be made about the approach taken.
|
|
Readiness for Dissemination
Review Date: January 2012
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
CHAMP Family Program Dissemination Guidelines and Supporting Studies
Hibbert, R. H., Miranda, C., Miranda, A., & Lawrence, R. (n.d.). The CHAMP Family Program trainer manual. New York, NY: Columbia University School of Social Work.
McKay, M. M., Hibbert, R. H., McCoy, W., Grogan, G., Gamble, D., Castro, I., et al. (2001). The CHAMP Family Program facilitator manual. New York, NY: Columbia University School of Social Work.
McKay, M. M., & Paikoff, R. L. (Eds.) (2007). Community collaborative partnerships: The foundation of HIV prevention research efforts. Binghamton, NY: The Haworth Press.
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
2.0
|
2.0
|
3.3
|
2.4
|
Dissemination Strengths The implementation manual offers handouts and general information on program engagement and family processes. A thorough training manual is provided and 2-day, on-site training is available. Outcome measurement tools are provided for both children and parents.
Dissemination Weaknesses Information on organizational or facilitator requisites is not provided. The facilitator and training manuals were developed for a research project and rely heavily on direction provided by the research team. The manuals have not been revised to be used in routine community settings. The pre- and posttest instruments are lengthy, which may make them difficult to administer in real-world settings. Little guidance is provided about how to interpret the data collected from evaluation tools.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
CHAMP Facilitator Manual (includes session outlines, activity outlines, scripts, pre- and posttests, and other support materials)
|
Free
|
Yes
|
|
2-day, on-site training for supervisors and facilitators
|
$1,500 per day (cost includes travel expenses)
|
No
|
|
CHAMP Trainer Manual
|
Free
|
Yes
|
|
On-site or telephone consultation
|
$1,500 per day (cost includes travel expenses)
|
No
|
Additional Information Consultation can be provided on site or over the phone. The cost can be prorated for consultations lasting less than 1 day.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
Bhana, A., Petersen, I., Mason, A., Mahintsho, Z., Bell, C., & McKay, M. (2004). Children and youth at risk: Adaptation and pilot study of the CHAMP (Amaqhawe) programme in South Africa. African Journal of AIDS Research, 3(1), 33-41.
Gopalan, G., Cavaleri, M. A., Bannon, W. M., & McKay, M. (in press). Differential risk factors associated with the onset of externalizing behaviors within two impoverished, urban communities. Journal of HIV/AIDS and Social Services.
McKay, M., Block, M., Mellins, C., Traube, D. E., Brackis-Cott, E., Minott, D., et al. (2007). Adapting a family-based HIV prevention program for HIV-infected preadolescents and their families: Youth, families and health care providers coming together to address complex needs. Social Work in Mental Health, 5(3/4), 355-378.
|
|
|
