Reviews & Submissions Overview
Once an intervention has been approved for review by SAMHSA, NREPP contractor staff identify the outcomes and materials that will be used by certified reviewers to complete the review. A review can take several months from submission to completion. The NREPP intervention program profile will then be approved by SAMHSA staff. Please note that since each NREPP review represents a considerable investment of time and public funds, SAMHSA reserves the right to publish all programs on the website that were reviewed and rated. The four stages of the review process are summarized below.
Pre-screening and Approval for Review
- NREPP receives programs for review in three ways:
- Nominations from the field: SAMHSA announces an open submission period. The open submission period generally lasts several months and allows developers, researchers, practitioners, and other interested parties to submit programs for review.
- Environmental scans: SAMHSA and NREPP contractor staff conduct environmental scans (including literature searches, focus groups, public input, and interviews) to identify potential interventions for review.
- Agency nomination: SAMHSA identifies programs and practices addressing specific agency priorities.
- Programs identified through the open submission period are prioritized for review.
- Programs are pre-screened to ensure that at least one evaluation study meets the minimum criteria for review.
- SAMHSA has determined that all eligible outcomes will be reviewed and all studies and outcomes that meet the standardized screening criteria will be reviewed, including those with negative or non-significant effects. Programs with reported positive impacts on outcomes and populations of interest will be prioritized for review over programs with no positive impacts.
- Pre-screened programs are submitted to SAMHSA for approval to move into the review process.
A Federal Register
notice issued July 7, 2015 announced the intent to re-review currently posted NREPP programs to comport with new review criteria and ratings.
The changes to the review process will apply to all re-reviews of programs currently posted on NREPP. Approximately 110 programs posted on NREPP prior to September 2015 will be reviewed each year over the next 3 years. Program contacts/developers will be notified at least 45 days prior to the re-review date that their program has been selected for re-review. The re-review will follow the procedures presented on the Review Process page
Literature Search and Screening
As of July 7, 2015, SAMHSA has determined that programs no longer require resources for dissemination and implementation (RFDI) materials to be reviewed (see July 7, 2015 Federal Register notice
), but that programs with such materials will be prioritized for review.
- NREPP contractor staff contact the developer to gather any additional evaluation or RFDI materials to complete the review.
- NREPP contractor staff determine which evaluation studies and outcomes are eligible for review.
- NREPP contractor staff conduct a literature search to identify other relevant evaluation studies.
- NREPP contractor staff identify the reviewers. Reviewers must complete a Conflict of Interest form to confirm no conflict exists.
- NREPP contractor staff send review packets to certified reviewers to assess the rigor of the study and the magnitude and direction of the program’s impact on eligible outcomes.
- Reviewers independently review the materials provided and calculate ratings using the NREPP Outcome Rating Instrument.
- Reviewers submit their ratings to the NREPP contractor.
- If their ratings differ by a significant margin, NREPP contractor staff may hold a consensus conference to discuss and resolve the differences.
- NREPP contractor staff compile the ratings and descriptive information into a program profile.
- NREPP contractor staff share a courtesy copy of the program profile with the developer, who has the right to suggest revisions to the profile.
- The final program profile is submitted to SAMHSA for review, approval, and posting on the NREPP website.