Quality of Research
Review Date: October 2009
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Chamberlain, P., & Reid, J. (1998). Comparison of two community alternatives to incarceration for chronic juvenile offenders. Journal of Consulting and Clinical Psychology, 66(4), 624-633. 
Eddy, J. M., Whaley, R. B., & Chamberlain, P. (2004). The prevention of violent behavior by chronic and serious male juvenile offenders: A 2-year follow-up of a randomized clinical trial. Journal of Emotional and Behavioral Disorders, 12, 2-8.
Smith, D. K., Chamberlain, P., & Eddy, J. M. (in press). Multidimensional Treatment Foster Care: Preliminary support for the treatment of substance use. Journal of Child and Adolescent Substance Abuse. Study 2Chamberlain, P., Leve, L. D., & DeGarmo, D. S. (2007). Multidimensional Treatment Foster Care for girls in the juvenile justice system: 2-year follow-up of a randomized clinical trial. Journal of Consulting and Clinical Psychology, 75(1), 187-193.
Leve, L. D., & Chamberlain, P. (2007). A randomized evaluation of Multidimensional Treatment Foster Care: Effects on school attendance and homework completion in juvenile justice girls. Research on Social Work Practice, 17(6), 657-663. Study 3Kerr, D., Leve, L. D., & Chamberlain, P. (2009). Pregnancy rates among juvenile justice girls in two randomized controlled trials of Multidimensional Treatment Foster Care. Journal of Consulting and Clinical Psychology, 77(3), 588-593.
Supplementary Materials Chamberlain, P., & Reid, J. B. (1991). Using a specialized foster care community treatment model for children and adolescents leaving the State mental hospital. Journal of Community Psychology, 19, 266-276.
Outcomes
| Outcome 1: Days in locked settings |
|
Description of Measures
|
The number of days spent in locked settings was assessed using:
- Official criminal referral data recorded by the Oregon Youth Authority. Probation and parole officer reports, obtained every 2 months, and records maintained by the MTFC programs were used to determine the total number of days each youth spent in various settings (local detention facilities, State training schools, etc.). Data were coded by research assistants who were blind to study conditions.
- Youth's self-report, verified against probation and parole officer reports, of total days spent in detention, correctional facilities, jail, or prison. Self-reports were obtained during an in-person interview with youth by research assistants who were blind to study condition.
|
|
Key Findings
|
In one study, boys 12-17 years old who had histories of chronic and serious juvenile delinquency and were referred for community placement by the juvenile justice system were randomly assigned to participate in MTFC or a comparison condition (community-based group care). Official criminal referral data showed that during the year after referral, MTFC participants spent significantly fewer days in locked settings relative to the comparison group (p = .002), with fewer days in local detention facilities (MTFC mean = 32 days; comparison group mean = 70 days) and fewer days in State training schools (MTFC mean = 21 days; comparison group mean = 59 days). Overall, MTFC participants spent 60% fewer days incarcerated during the year after referral relative to the comparison group.
In another study, girls 13-17 years old who had histories of criminal referrals and were mandated by juvenile justice to out-of-home care were randomly assigned to participate in MTFC or a comparison condition (standard services provided to delinquent girls). MTFC participants reported fewer days spent in locked settings during the 12 months after baseline (p < .05) and from 12 months after baseline to 24-month follow-up (p < .01) than did the comparison group.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
| Outcome 2: Substance use |
|
Description of Measures
|
Self-reported substance use data were collected at baseline and at 12- and 18-month follow-up using a questionnaire developed for the study. At baseline, participants reported their substance use for the prior 6 months. At the 12- and 18-month assessments, they reported their substance use since the previous assessment. Participants reported their use of tobacco, alcohol, marijuana, and other drugs (i.e., cocaine, speed, LSD, heroin, mushrooms, PCP, morphine, inhalants) according to a 5-point Likert scale from 1 (never) to 5 (used 1 or more times per day).
|
|
Key Findings
|
Boys 12-17 years old who had histories of chronic and serious juvenile delinquency and were referred for community placement by the juvenile justice system were randomly assigned to participate in MTFC or a comparison condition (community-based group care). This study found that:
- At 12-month follow-up, no statistically significant differences between groups were found for reported alcohol, tobacco, or marijuana use. However, MTFC participants did have a significant decrease in their reported use of other drugs relative to the comparison group (p < .05).
- At 18-month follow-up, MTFC participants had a significant decrease in reported tobacco use (p < .01), marijuana use (p < .01), and other drug use (p < .05) relative to the comparison group. No statistically significant difference between groups was observed for alcohol use at the 18-month assessment.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 3: Criminal and delinquent activities |
|
Description of Measures
|
Involvement in criminal and delinquent activities was assessed using:
- Official criminal referral data recorded by the Oregon Youth Authority. A criminal referral was defined as an offense for which the youth was charged in police reports in any of the following categories: assault, menacing, kidnapping, unlawful weapons use, robbery, rape, sexual abuse, attempted murder, and murder. Both misdemeanor and felony offenses were included. Data were collected for three periods: for the 1 year before placement in MTFC or the comparison condition (T1), from placement to 1 year after program discharge or expulsion (T2), and from placement to 24 months after program discharge or expulsion (T3). Data were coded by a research assistant who was blind to study condition.
- Elliot Behavior Checklist (EBC), a confidential self-report measure on criminal and delinquent activities administered at baseline and 12-month follow-up. The respondent was asked how many times he or she engaged in each of 23 criminal or delinquent behaviors during a specific period. Three subscales were used: General Delinquency, Index Offenses, and Felony Assaults. The first subscale measures overall criminal activity, while the latter two measure the rates of more serious crimes.
|
|
Key Findings
|
In one study, boys 12-17 years old who had histories of chronic and serious juvenile delinquency and were referred for community placement by the juvenile justice system were randomly assigned to participate in MTFC or a comparison condition (community-based group care). This study found that:
- MTFC participants showed a larger drop in the number of criminal referrals per year from T1 to T2 (8.5 to 2.6) relative to the comparison group (8.5 to 5.4; p = .003).
- At 12-month follow-up, MTFC participants had lower mean scores on the EBC than the comparison group on all three subscales examined: General Delinquency (12.8 vs. 28.9; p = .01), Index Offenses (3.2 vs. 8.6; p = .03) and Felony Assaults (1.2 vs. 2.7; p = .05).
In another study, girls 13-17 years old who had histories of criminal referrals and were mandated by juvenile justice to out-of-home care were randomly assigned to participate in MTFC or a comparison condition (standard services provided to delinquent girls). In this study, MTFC participants had greater reductions than the comparison group in criminal referrals at T2 (p < .05) and at T3 (p < .01). There were no statistically significant differences between groups on the EBC.
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
| Outcome 4: Homework completion and school attendance |
|
Description of Measures
|
Homework completion and school attendance data were collected through in-person interviews with youths and their current caregivers.
For homework completion, caregivers and youths reported whether the youth did homework that day (0 = no, 1 = yes) via three phone interviews conducted within a 1-week period between 3 and 6 months postbaseline. Caregivers and youths also reported at baseline and 12-months follow-up the number of days in the past week on which the youth spent at least 30 minutes on homework (range = 0 to 7 days).
For school attendance, caregivers and youths reported how often the youth attended school (1 = not attending, 2 = attending very infrequently, 3 = attending infrequently, 4 = attending more often than not, 5 = attending regularly, or 6 = attending 100% of the time). School attendance data were collected at baseline and 12-month follow-up.
The caregivers were primarily the youth's biological parents at baseline, the intervention caregivers (i.e., foster parents) at the 3- to 6-month follow-up, and the biological parents or placement setting staff at the 12-month follow-up. Composite scores were formed for each variable by aggregating caregiver and youth reports.
|
|
Key Findings
|
Female adolescents (13-17 years old) who had histories of criminal referrals and were mandated by juvenile justice to out-of-home care were randomly assigned to participate in MTFC or a comparison condition (standard services provided to delinquent girls). This study found that:
- MTFC participants had significantly higher mean levels of daily homework completion and did homework on significantly more days relative to the comparison group (p values < .05).
- MTFC participants had significantly greater school attendance relative to the comparison group (p < .05).
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 5: Pregnancy rates |
|
Description of Measures
|
Pregnancy rates were assessed by separate interviews of youths and their current caregivers.
At baseline, each girl and her caregiver were separately interviewed by program staff regarding the girl's pregnancy history, coded 0 for negative or 1 for positive. Caregiver reports were used when girls' reports were missing.
At follow-up assessments (6, 12, 18, and 24 months postbaseline), each girl and her caregiver were separately interviewed about pregnancies that had occurred since baseline. The presence or absence of a pregnancy was coded in yes/no format based on all available information. Pregnancies reported by girls were counted as positive. As at baseline, caregiver reports were used when girls' reports were missing.
The caregivers were primarily the youth's biological parents at baseline, the intervention caregivers (i.e., foster parents) at 6-month follow-up, and the biological parents or placement setting staff at 12-month follow-up and thereafter.
|
|
Key Findings
|
Girls 13-17 years old who had histories of criminal referrals and were mandated by juvenile justice to out-of-home care were randomly assigned to participate in MTFC or a comparison condition (community-based group care). This study found that MTFC participants had significantly fewer reported pregnancies relative to the comparison group (26.9% vs. 46.9%; p = .004) from baseline to 24-month follow-up.
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.1
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
|
100% Male
|
85% White
6% Black or African American
6% Hispanic or Latino
3% American Indian or Alaska Native
|
|
Study 2
|
13-17 (Adolescent)
|
100% Female
|
74% White
12% American Indian or Alaska Native
9% Hispanic or Latino
2% Black or African American
2% Race/ethnicity unspecified
1% Asian
|
|
Study 3
|
13-17 (Adolescent)
|
100% Female
|
74% White
13% Race/ethnicity unspecified
7% Hispanic or Latino
4% American Indian or Alaska Native
2% Black or African American
1% Asian
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Days in locked settings
|
3.0
|
3.0
|
3.0
|
3.4
|
3.0
|
3.3
|
3.1
|
|
2: Substance use
|
2.5
|
1.5
|
3.1
|
3.3
|
3.0
|
3.3
|
2.8
|
|
3: Criminal and delinquent activities
|
3.3
|
2.8
|
3.1
|
3.3
|
3.0
|
3.3
|
3.1
|
|
4: Homework completion and school attendance
|
2.3
|
2.0
|
3.0
|
3.3
|
3.0
|
3.5
|
2.8
|
|
5: Pregnancy rates
|
2.5
|
2.8
|
3.0
|
3.8
|
3.0
|
3.8
|
3.1
|
Study Strengths Each of the studies involved randomization of youth to intervention and comparison groups. Baseline equivalence was established on some of the relevant variables. Attrition was generally low across the studies, and the analytic procedures that were used adequately handled missing data. The data analytic strategies were appropriate for the outcomes studied.
Study Weaknesses Information on psychometrics was inadequate for some measures (e.g., substance use, homework completion, school attendance). The findings from measures of intervention fidelity were not reported, so it is difficult to judge if treatment fidelity was achieved. The authors of one study suggested a need to examine differences in the selection and training of MTFC and comparison group parents, yet data were not provided on these group differences, which may have accounted for treatment effects.
|
|
Readiness for Dissemination
Review Date: October 2009
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
OSLC Community Programs, Inc. (2003). MTFC skills coach training. Eugene, OR: Author.
OSLC Community Programs, Inc. (n.d.). MTFC foster parent recruitment brochure. Eugene, OR: Author.
OSLC Community Programs, Inc. (n.d.). Multidimensional Treatment Foster Care family therapy manual. Eugene, OR: Author.
OSLC Community Programs, Inc. (n.d.). Multidimensional Treatment Foster Care foster parent manual. Eugene, OR: Author.
OSLC Community Programs, Inc. (n.d.). Multidimensional Treatment Foster Care foster parent recruitment manual. Eugene, OR: Author.
OSLC Community Programs, Inc. (n.d.). Multidimensional Treatment Foster Care foster parent training manual. Eugene, OR: Author.
OSLC Community Programs, Inc. (n.d.). Multidimensional Treatment Foster Care individual therapy manual. Eugene, OR: Author.
OSLC Community Programs, Inc. (n.d.). Multidimensional Treatment Foster Care point level system manual. Eugene, OR: Author.
OSLC Community Programs, Inc. (n.d.). Multidimensional Treatment Foster Care program supervisor manual. Eugene, OR: Author.
OSLC Community Programs, Inc. (n.d.). Multidimensional Treatment Foster Care skills coach manual. Eugene, OR: Author.
OSLC Community Programs, Inc. (n.d.). Multidimensional Treatment Foster Care skills coach training manual. Eugene, OR: Author.
PowerPoint presentations:
- Family Therapy
- Foster Parent Recruitment
- Foster Parent Training and Role of the MTFC Parent
- Individual Therapy
- MTFC-A Case Example: Misty
- MTFC Foster Parent Training
- Overview: The MTFC Model and Supporting Research
- Parent Daily Report
- Program Supervision
- Referral Process, "Matching," and Placement
- Skills Coach
- The Point and Level System
- Welcome Program Supervisors: MTFC Clinical Training
TFC Consultants, Inc. (n.d.). MTFC-A program assessment and remedial service. Eugene, OR: Author.
TFC Consultants, Inc. (n.d.). MTFC feasibility information and review. Eugene, OR: Author.
TFC Consultants, Inc. (n.d.). MTFC fee schedule. Eugene, OR: Author.
Other materials:
- "Foster a Teen, Foster Hope" handout (case examples)
- Job Descriptions for MTFC Program Supervisor, Child and Family Therapist, Foster Parent Recruiter and Trainer/PDR Caller, and Skills Trainer
- MTFC Program Assessment and Quality Assurance Internal Review (self-assessment guide)
- MTFC Program Cost Calculator--Adolescent--February '08 (Excel spreadsheet)
- Sample information letter to potential implementers
- WebPDR screenshots (PowerPoint slides)
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.5
|
3.8
|
4.0
|
3.8
|
Dissemination Strengths The implementation materials are detailed, comprehensive, and easy to follow. Separate manuals and PowerPoint presentations and a detailed overview of MTFC are provided to all supervisors, therapists, skills coaches, and parents involved in implementation. The developer offers a feasibility assessment protocol, training, consultation, and technical assistance. During the first year of implementation, additional intensive services are provided by the developer to initiate and support new programs, including foster parent recruitment consultation, a stakeholders presentation and implementation planning meeting, foster parent training, data collection training, weekly telephone consultation, three site visits, and program reviews and assessments. The developer has established clear fidelity and practice standards. Trainers are assessed and approved by the developer, and programs are required to receive an initial 2-year certification and recertification every 3 years thereafter from the Center for Research to Practice, an independent research organization in Eugene, Oregon. Developer-approved experts assess programs every 9-11 months to monitor model drift. Consultation and technical assistance is available to address model-adherence issues detected in the assessment. In addition, the developer provides materials to assist programs in conducting internal quality assurance reviews to help evaluate their progress toward successful implementation of the model. A Web-based behavior tracking system (WebPDR) is used to monitor individual outcomes.
Dissemination Weaknesses The materials provided to aid implementation, although extensive, could be made more user friendly. The materials lack a concise introduction and overview of the model as well as of the components involved in the implementation process. Although some of this information is provided in an introductory letter to interested sites and on the developer's Web site, it is not readily accessible, and one may find it difficult to get an overall picture of the program. The requirement for all staff to travel to Eugene, Oregon, to receive training may be a significant barrier for some agencies.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
First-year training, consultation, and implementation services (includes implementation materials and self-assessment tool)
|
$46,500
|
Yes
|
|
Off-site training in Eugene, OR
|
About $12,500 per site depending on the travel distance involved
|
Yes
|
|
Consultation on site readiness
|
$1,500 per site
|
Yes
|
|
Annual program assessment service
|
Free in first year, $1,840 annually thereafter
|
Yes
|
|
Program certification
|
$2,050
|
Yes
|
Additional Information Costs vary considerably depending on local salary levels, foster parent compensation, and other differentials. Foster parent payments and salaries typically are the biggest components. In most cases in the United States, total costs for care and treatment (programmatic plus implementation costs) are between $200 and $225 per day per child.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
Westermark, P. K., Hansson, K., & Vinnerljung, B. (2007). Foster parents in Multidimensional Treatment Foster Care: How do they deal with implementing standardized treatment components? Children and Youth Services Review, 29(4), 442-459.
Westermark, P. K., Hansson, K., & Vinnerljung, B. (2008). Does Multidimensional Treatment Foster Care (MTFC) prevent breakdown in foster care? International Journal of Child and Family Welfare, 4, 155-171.
|
|
|
