Quality of Research
Review Date: January 2010
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Caldwell, M. F., & Van Rybroek, G. J. (2005). Reducing violence in serious juvenile offenders using intensive treatment. International Journal of Law and Psychiatry, 28, 622-636.  Study 2Caldwell, M. F., Skeem, J., Salekin, R., & Van Rybroek, G. J. (2006). Treatment response of adolescent offenders with psychopathy features: A 2-year follow-up. Criminal Justice and Behavior, 33(5), 571-596. Study 3Caldwell, M. F., McCormick, D. J., Umstead, D., & Van Rybroek, G. J. (2007). Evidence of treatment progress and therapeutic outcomes among adolescents with psychopathic features. Criminal Justice and Behavior, 34(5), 571-587.
Supplementary Materials Caldwell, M. F., Vitacco, M., & Van Rybroek, G. J. (2006). Are violent delinquents worth treating? A cost-benefit analysis. Journal of Research in Crime and Delinquency, 43(2), 148-168.
Outcomes
| Outcome 1: Violent recidivism |
|
Description of Measures
|
Data on violent offenses were extracted from public court records of filed charges. The study examined the type of offense (i.e., nonviolent misdemeanor, nonviolent felony, violent misdemeanor, violent felony, violent felony with injury, and homicide), the number of offenses in each category, and the days at large before each offense type. Offenses were coded as violent if they involved a direct assault on another person.
|
|
Key Findings
|
After controlling for time at risk in the community and other covariates, youth who received treatment in the MJTC program showed a significant reduction in the prevalence of recidivism compared with youth who were admitted to MJTC briefly for assessment or stabilization services and then returned to the referring secured correctional institution (p < .0005 for felony offending, violent offending, and more serious felony violent offending). The program had the greatest impact on serious violent offenses, reducing the risk of their incidence by about half. Youth in the treatment group were more than 6 times less likely to engage in felony violence than the comparison group youth (p = .0006). In addition to reducing the number of youth involved in offending, MJTC treatment contributed to longer periods of time in the community before the first offense (p < .05 for misdemeanor offenses and p < .005 for felony offenses, violent offenses, and violent felony offenses).
In another study of youth with psychopathic personality traits (as measured by the Psychopathy Checklist: Youth Version), after controlling for the treatment assignment process and release status, youth who received MJTC treatment were less likely to be involved in community violence within 2 years of release than youth who were admitted to the MJTC briefly for assessment or stabilization services and then returned to the referring secured correctional institution (p < .05). Youth who received MJTC treatment also spent significantly more days in the community before the first violent offense than youth in the comparison group (p < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Quasi-experimental
|
|
Quality of Research Rating
|
3.0
(0.0-4.0 scale)
|
| Outcome 2: Behavioral compliance |
|
Description of Measures
|
Behavioral compliance was measured using the Today-Tomorrow Program, a behavioral point system designed to closely monitor the youth's prosocial interactions and disruptive or aggressive behaviors. The system includes scales for peer and staff interactions, rule compliance, and performance in treatment groups and school classes. Frontline staff interacting with youth assigned points reflecting each youth's compliance with behavioral expectations at the end of each shift. The weekly mean of the percentage of possible points earned was calculated for each youth in treatment. The mean percentage of possible points earned during the first 3 weeks of full-treatment programming was calculated as a baseline value and compared with the mean percentage for the final 3 weeks of treatment.
|
|
Key Findings
|
Controlling for scores on the Psychopathy Checklist: Youth Version, baseline score, age, and criminal versatility (the variety of crime types committed), the number of weeks of MJTC treatment significantly predicted both baseline to posttreatment improvement in interpersonal functioning and behavioral control (p = .001) and final behavioral scores (p = .001).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Preexperimental
|
|
Quality of Research Rating
|
2.7
(0.0-4.0 scale)
|
| Outcome 3: Absence of security-based sanctions |
|
Description of Measures
|
Security-based sanctions are increased security measures, specifically isolation of a youth in his room in response to aggressive behavior. For this study, the percentage of days each youth was free of any security sanctions during each week of treatment was tabulated. The mean for the first 3 weeks of full-treatment programming was used as a security baseline and compared with the mean for the final 3 weeks of treatment. Fewer than half of the participants accrued any days of security time in the final 3 weeks of treatment, so this variable was coded dichotomously (no security time vs. any security time).
|
|
Key Findings
|
Controlling for scores on the Psychopathy Checklist: Youth Version, baseline score, age, and criminal versatility (the variety of crime types committed), the number of weeks of MJTC treatment significantly predicted both baseline to end-of-treatment change in mean percentage of days the participating youth were free from any security sanctions (p = .001) and whether MJTC participants received any days of security time during the last 3 weeks of treatment (p = .005).
|
|
Studies Measuring Outcome
|
Study 3
|
|
Study Designs
|
Preexperimental
|
|
Quality of Research Rating
|
2.9
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
13-17 (Adolescent)
|
100% Male
|
51% Black or African American
38% White
9% Hispanic or Latino
2% Race/ethnicity unspecified
|
|
Study 2
|
13-17 (Adolescent)
|
100% Male
|
59% Black or African American
31% White
10% Race/ethnicity unspecified
|
|
Study 3
|
13-17 (Adolescent)
|
100% Male
|
51% Black or African American
43% White
6% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Violent recidivism
|
3.0
|
3.0
|
1.5
|
3.9
|
2.8
|
3.8
|
3.0
|
|
2: Behavioral compliance
|
2.3
|
2.3
|
1.5
|
3.8
|
2.8
|
3.5
|
2.7
|
|
3: Absence of security-based sanctions
|
3.0
|
3.0
|
1.5
|
3.8
|
2.8
|
3.5
|
2.9
|
Study Strengths Although the studies provided limited information on reliability coefficients and validity, official records of charges filed in a State circuit court are considered a reliable and valid source of recidivism data. MJTC's behavioral point program has face validity. Behavioral ratings assigned by treatment staff after each shift are discussed with the supervising licensed psychologist, resulting in a consensus point value. The training that staff received on the point system facilitated more consistent ratings across observers. Staff were provided extensive orientation and training on the program, which offered some consistency in implementation. Missing data and attrition were generally not an issue. Missing data on youth characteristics used as covariates in analyses were imputed using either random values or mean substitution for variables missing 10% of the data or less. Analyses were appropriate, and statistical techniques were used to minimize the impact of nonrandomization.
Study Weaknesses A weakness of all three studies is the lack of documentation of the nature and intensity of the services provided to youth in the MJTC and comparison groups. The MJTC program was frequently modified during the study period. The nonrandomized selection used in the two studies with comparison groups suggests selection bias. Some youth were excluded from one of the studies because of missing data. No information was provided as to how many youth were excluded, and there is no way to determine whether and how they differed from the study participants.
|
|
Readiness for Dissemination
Review Date: January 2010
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Mendota Juvenile Treatment Center. (2009). MJTC student handbook. Madison, WI: Author.
MJTC Additional Information
MJTC Training PowerPoint
MJTC Treatment Components
Participant handouts
Today-Tomorrow Behavioral Program materials:
- Behavior Rating Scales (treatment groups, school, peers, limits, adults)
- Coping skill tracking sheet
- MJTC Database Setup
- MJTC Points Sheet
- MJTC Privileges
- MJTC Today-Tomorrow Program overview
- Notes on Today-Tomorrow Programming
- Participant training worksheets
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
2.5
|
1.8
|
2.0
|
2.1
|
Dissemination Strengths Descriptions and examples of the behavior management system are well developed and articulated. A range of participant worksheets and handouts are provided to support implementation. Training and support from the developer are required for implementation. The behavior management system database can generate summary reports to support treatment progress monitoring.
Dissemination Weaknesses Sufficient step-by-step implementation guidance is not provided. Additional information is needed on the supervision and organizational structure required to support successful implementation. Several handouts and guides are provided, but it is unclear who uses these materials or how they should be sequenced. Many of the participant handouts assume a high level of reading comprehension for referred youth. The content, duration, and intensity of training and support are not clearly defined. No tools are available to help monitor fidelity. No comprehensive quality assurance protocol is available.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Today-Tomorrow program worksheets
|
$250 per set
|
Yes
|
|
2-day, on-site training
|
$3,000
|
Yes
|
|
Monthly follow-up consultation for 6 months
|
$9,000
|
Yes
|
|
Outcomes assessments
|
$600
|
No
|
Additional Information Daily implementation costs are more than twice those of typical secured juvenile corrections services. However, because participation in the program typically results in a shorter length of stay, overall correctional costs increase only 4.5%, or by approximately $7,000 per youth.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
* Caldwell, M. F., McCormick, D. J., Umstead, D., & Van Rybroek, G. J. (2007). Evidence of treatment progress and therapeutic outcomes among adolescents with psychopathic features. Criminal Justice and Behavior, 34(5), 571-587.
* Caldwell, M. F., Skeem, J., Salekin, R., & Van Rybroek, G. J. (2006). Treatment response of adolescent offenders with psychopathy features: A 2-year follow-up. Criminal Justice and Behavior, 33(5), 571-596.
* Caldwell, M. F., & Van Rybroek, G. J. (2005). Reducing violence in serious juvenile offenders using intensive treatment. International Journal of Law and Psychiatry, 28, 622-636.
Caldwell, M. F., Vitacco, M., & Van Rybroek, G. J. (2006). Are violent delinquents worth treating? A cost-benefit analysis. Journal of Research in Crime and Delinquency, 43(2), 148-168.
|
|
|
