Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Herman, D., Opler, L., Felix, A., Valencia, E., Wyatt, R. J., & Susser, E. (2000). A critical time intervention with mentally ill homeless men: Impact on psychiatric symptoms. Journal of Nervous and Mental Disease, 188, 135-140. 
Jones, K., Colson, P. W., Holter, M. C., Lin, S., Valencia, E., Susser, E., et al. (2003). Cost-effectiveness of Critical Time Intervention to reduce homelessness among persons with mental illness. Psychiatric Services, 54(6), 884-890.
Susser, E., Valencia, E., Conover, S., Felix, A., Tsai, W. Y., & Wyatt, R. J. (1997). Preventing recurrent homelessness among mentally ill men: A "critical time" intervention after discharge from a shelter. American Journal of Public Health, 87(2), 256-262.
Supplementary Materials CTI Daily Contact Log
CTI Fidelity Scale
CTI Follow-Up Note
CTI Initial Note
CTI Treatment Manual
Herman, D., Conover, S., Felix, A., Nakagawa, A., & Mills, D. (n.d.). Critical Time Intervention: An empirically supported model for preventing homelessness in high risk groups. Manuscript submitted for publication.
Valencia, E., Susser, E., Torres, J., Felix, A., & Conover, S. (1997). Critical Time Intervention for homeless mentally ill individuals in transition from shelter to community living. In W. R. Breakey & J. W. Thompson (Eds.), Mentally ill and homeless: Special programs for special needs (pp. 75-94). Amsterdam: Harwood Academic Publishers.
Outcomes
| Outcome 1: Homeless nights and extended homelessness |
|
Description of Measures
|
This outcome was measured every 30 days during an 18-month follow-up period by (1) average number of homeless nights (nights spent in shelters or public spaces), (2) proportion of participants who were homeless for most of the last month, (3) major homeless episodes (lasting 30 nights or more), (4) extended homelessness (being homeless more than 54 nights), and (5) nonhomeless nights (total number of nights minus the number of nights homeless).
|
|
Key Findings
|
CTI participants averaged 30 homeless nights during a follow-up period, compared with 91 nights for men receiving usual services only (p = .003). They also averaged more nonhomeless nights (p = .01). CTI participants were less likely to be homeless during the past month (p = .05) and to experience extended homelessness (p = .045).
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.3
(0.0-4.0 scale)
|
| Outcome 2: Negative symptoms of psychopathology |
|
Description of Measures
|
Severity of negative symptoms was assessed using the Positive and Negative Syndrome Scale (PANSS). The scale measures (1) positive symptoms (delusions, hallucinations, grandiosity, suspiciousness, hostility, etc.); (2) negative symptoms (social withdrawal, emotional withdrawal, poor rapport, difficulty in abstract thinking, etc.); and (3) general psychopathology symptoms (anxiety, guilt feelings, tension, depression, etc.). Symptom ratings were performed by trained raters blind to group assignment.
|
|
Key Findings
|
CTI participants experienced a significant decrease in negative symptoms over a 6-month follow-up period, compared with recipients of usual services (p = .02). No statistically significant effect was reported on positive or general symptoms of psychopathology.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.8
(0.0-4.0 scale)
|
| Outcome 3: Number of homeless nights as a function of cost |
|
Description of Measures
|
Cost-effectiveness was measured by comparing the cost of housing (defined as the total average costs for acute care services, outpatient services, housing and shelter services, criminal justice services, and transfer income) with willingness-to-pay values (the additional price society is willing to spend for an additional nonhomeless night greater than $152).
|
|
Key Findings
|
Over the course of an evaluation, the housing cost for CTI participants was $52,374, compared with $51,649 for recipients of usual services. The CTI participants experienced greater net housing stability benefit for each willingness-to-pay value.
|
|
Studies Measuring Outcome
|
Study 1
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
2.5
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
18-25 (Young adult)
26-55 (Adult)
55+ (Older adult)
|
100% Male
|
74% Black or African American
26% Race/ethnicity unspecified
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Homeless nights and extended homelessness
|
4.0
|
3.8
|
2.5
|
4.0
|
2.0
|
3.5
|
3.3
|
|
2: Negative symptoms of psychopathology
|
3.5
|
3.5
|
2.5
|
3.5
|
2.0
|
2.0
|
2.8
|
|
3: Number of homeless nights as a function of cost
|
2.0
|
2.0
|
2.0
|
3.5
|
2.0
|
3.5
|
2.5
|
Study Strengths The developer provided a clear operational definition for each measure of homeless status. Most of the measures were reliable and had some evidence of validity, most of the data were complete, and measurement was frequent. Interviewers regularly documented the housing status of study participants. The authors used an intervention fidelity measure to monitor whether clinicians were providing specific intervention elements. The study had a particularly high follow-up rate, which was an important study strength. Patients who withdrew from the study were tracked, and the reasons for withdrawal were identified. Appropriate analyses were used to assess group differences for each measure of homelessness. The large effect size for outcome 1 (prevention of recurrent homelessness) offsets concerns regarding the study sample size. The retention rate for outcome 2 (reduction of negative symptoms) was 80%, which was respectable given the target population. While problems may exist in the study used to measure cost-effectiveness (outcome 3), it is commendable that the authors conducted it.
Study Weaknesses For outcome 1 (preventing recurrent homelessness), some measures had face validity but little evidence of some other measures of validity, including the proportion of participants who were homeless, major homeless episodes, and extended homelessness. The authors did not report data on a potential confounding variable; specifically, housing placements at discharge may have impacted homelessness. Specifics regarding the randomization process were not provided. For outcome 2 (reduction of negative symptoms), the authors did not indicate if reports on reliability were done by independent reviewers. Documentation of the validity of the measures used was not provided. There was little discussion of attempts to adjust for potential confounding variables. Data were not collected on other factors, such as psychiatric medication or alcohol/drug use severity, that are associated with symptom reduction, particularly negative symptoms. Analyses might have been impacted by the small study sample. Specifics regarding the randomization process were not provided. The appropriateness of the analysis used to evaluate cost-effectiveness (outcome 3) is not clear.
|
|
Readiness for Dissemination
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Conover, S., & Herman, D. (n.d.). Critical Time Intervention fidelity scale protocol.
CTI Daily Contact Log
CTI Fidelity Assessor Worksheets
CTI Fidelity Scale
CTI Follow-Up Note
CTI Initial Note
CTI Service Plan
CTI Treatment Manual
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
2.0
|
0.5
|
1.5
|
1.3
|
Dissemination Strengths Implementation materials include examples of treatment plans and activity logs. The materials describe the model and provide a variety of tools necessary to implement the program. A comprehensive fidelity scale and manual are provided.
Dissemination Weaknesses The implementation materials assume that the individuals delivering the program will be highly skilled clinicians or case managers who are knowledgeable about substance abuse, motivational interviewing, assessment of symptom severity, and system resources. Very little information is provided on ongoing supervision and consultation, although it is also not clear how much training and support services are in fact available to implementers. While a fidelity instrument is provided, the intervention's complexity demands further discussion of ongoing training, coaching, supervision, and clear evaluation tools to support quality assurance.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Implementation materials
|
Free
|
Yes
|
|
2-day, in-person training provided by Center for Urban Community Services, Inc.
|
Contact the developer
|
No
|
|
Web-based training provided by Center for Social Innovation
|
Contact the developer
|
No
|
|
In-person technical assistance provided by Center for Urban Community Services, Inc.
|
Contact the developer
|
No
|
|
CTI Fidelity Scale
|
Free
|
No
|
Additional Information The cost of CTI has been estimated in one study at $3,769 per participant over a 9-month period.
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
Samuels, J., Shinn, M., Fischer, S., Thompkins, A., & Park, H. (2006). The impact of the family Critical Time Intervention on homeless children: Final report to the National Institute of Mental Health. Orangeburg, NY: Nathan Kline Institute of Psychiatric Research.
Samuels, J., Tang, D. I., O'Callaghan, S., Barrow, S., Lawinski, T., & Travers, N. (2006). Homeless families program final report: Homeless families in Westchester County, NY--Phase 2. Orangeburg, NY: Nathan Kline Institute of Psychiatric Research.
|
|
|
