•  

Intervention Summary

Back to Results Start New Search

BrainTrain4Kids

BrainTrain4Kids is an interactive Web site (http://www.BrainTrain4Kids.com) that teaches children aged 7-9 years about the brain and the effects of drugs on the brain and body, building a foundation for later substance abuse prevention efforts. It also promotes positive attitudes toward science and instills knowledge of scientific inquiry. The online program is modeled after Brain Power!, a classroom-based training curriculum for children in grades 2 and 3. The developers of BrainTrain4Kids expanded the Brain Power! materials and adapted elements of the curriculum for use with interactive online media. Features of BrainTrain4Kids include activities, printable materials, and quizzes. The Web site also includes a Grown-Up Guide, which provides adults with information on the program, and a Playground, where users can play interactive health- and science-themed games.

The BrainTrain4Kids Web site, which uses a train motif to organize concepts, consists of six "stations" that correspond to the six parts of the Brain Power! curriculum: scientific inquiry, brain function, how messages travel through the nervous system, helpful and harmful effects of drugs, risks associated with smoking and what makes cigarettes addictive, and how to stay healthy. Each station includes four "buildings": Building 1 describes what users will learn in the station; buildings 2 and 3 provide educational lessons; and building 4 contains games, hands-on activities, and printable puzzles. Users must navigate through each set of four buildings sequentially. All six stations can be completed in approximately 2 hours; however, users are not limited in the amount of time they can spend interacting with the Web site. They can start or stop at any time, and they can repeat each section without limit. Adults are encouraged to monitor children as they use the site in case they need help understanding a concept or an activity; however, use of BrainTrain4Kids does not require direct adult supervision.

Descriptive Information

Areas of Interest Substance abuse prevention
Outcomes Review Date: April 2010
1: Attitudes toward science and drugs
2: Knowledge of science and drugs
Outcome Categories Alcohol
Drugs
Tobacco
Ages 6-12 (Childhood)
Genders Male
Female
Races/Ethnicities American Indian or Alaska Native
Asian
Black or African American
White
Race/ethnicity unspecified
Settings Home
Geographic Locations Urban
Suburban
Rural and/or frontier
Implementation History BrainTrain4Kids has been in use since July 2007 and has reached children, families, and class groups across Australia, Canada, and the United States.
NIH Funding/CER Studies Partially/fully funded by National Institutes of Health: Yes
Evaluated in comparative effectiveness research studies: No
Adaptations No population- or culture-specific adaptations of the intervention were identified by the developer.
Adverse Effects No adverse effects, concerns, or unintended consequences were identified by the developer.
IOM Prevention Categories Universal

Quality of Research
Review Date: April 2010

Documents Reviewed

The documents below were reviewed for Quality of Research. The research point of contact can provide information regarding the studies reviewed and the availability of additional materials, including those from more recent studies that may have been conducted.

Study 1

Metcalf, M. P., Hennis, M., Kelley, L., Sealy, S., & Workman, K. L. (2007). BrainTrain4Kids.com: An online substance abuse prevention and science education program for children. HHSN271200455234C. N44DA-4-5523. September 15, 2003 to July 15, 2007 (Final report submitted to the National Institute on Drug Abuse). Chapel Hill, NC: Clinical Tools.

Supplementary Materials

Metcalf, M. P. (2009). BrainTrain4Kids: Assessment of an interactive Internet program to educate children aged 7-9 about science, the brain and drugs. Manuscript submitted for publication.

Outcomes

Outcome 1: Attitudes toward science and drugs
Description of Measures The 16-item BrainTrain4Kids Attitude Assessment Instrument was used to assess each child's attitude toward science (6 items), health (5 items), and related health behavior issues (5 items). For example, items asked about science attitudes in general ("I think doing science is fun"), health and drug prevention ("Drinking alcohol can be harmful to your brain"), and health behavior ("I think exercise is fun"). Participants answered 13 items with a 5-point Likert scale ranging from "NO!" to "YES!," with the available response of "I'm not sure what this question means." In addition, participants responded to 3 items that were related to the health behaviors of others (e.g., "If someone I knew was smoking cigarettes, I would feel..."). These items had four possible answers: "not at all worried about them," "a little worried," "very worried," or "I don't understand this sentence." A higher score indicated greater interest in science and greater concern about drug use.
Key Findings Participants were randomly assigned to the intervention group or the wait-list control group, which had a 6-week waiting period to match the length of participation by the intervention group. The intervention group completed a pretest (initial assessment), an immediate posttest (second assessment), and a 1-week follow-up assessment (third assessment). The wait-list control group completed a pretest (initial assessment), an assessment after the 6-week waiting period (second assessment), and an immediate posttest (third assessment).

Study results included the following:

  • There was no significant difference in attitudinal scores between the intervention group and the wait-list control group at the initial assessment.
  • Results of the second assessment indicated that the two groups had significantly different attitudinal scores (p = .012). From the initial assessment to the second assessment, the intervention group's attitudinal score increased significantly (86.58 to 90.32; p < .001), but the wait-list control group's score did not (85.31 to 85.87).
  • From the second assessment to the third assessment, the wait-list control group's attitudinal score increased significantly (85.87 to 91.00; p = .001), and the intervention group's score decreased (90.32 to 88.60). From the initial assessment to the third assessment, the intervention group's attitudinal score did not have a significant increase (86.58 to 88.60).
Studies Measuring Outcome Study 1
Study Designs Experimental
Quality of Research Rating 1.7 (0.0-4.0 scale)
Outcome 2: Knowledge of science and drugs
Description of Measures The BrainTrain4Kids Knowledge Assessment Instrument was used to determine each child's knowledge of the program content, which included the basic components of scientific inquiry, the names and functions of four major parts of the brain, basic information on nerve cells, the differences between drugs and medicines, and the effects of nicotine and alcohol on the brain and lungs. The instrument was made up of 16 multiple-choice items and 5 true-false items, which also included "I don't know" as a possible response. A higher score indicated greater knowledge of the program content.
Key Findings Participants were randomly assigned to the intervention group or the wait-list control group, which had a 6-week waiting period to match the length of participation by the intervention group. The intervention group completed a pretest (initial assessment), an immediate posttest (second assessment), and a 1-week follow-up (third assessment). The wait-list control group completed a pretest (initial assessment), an assessment after the 6-week waiting period (second assessment), and an immediate posttest (third assessment).

Study results included the following:

  • There was no significant difference in knowledge scores between the intervention group and the wait-list control group at the initial assessment.
  • Results of the second assessment indicated that the two groups had significantly different knowledge scores (p < .001). From the initial assessment to the second assessment, the intervention group's knowledge score increased significantly (52.67 to 69.06; p < .001), and the wait-list control group's score decreased (53.75 to 49.18).
  • From the second assessment to the third assessment, the wait-list control group's knowledge score increased significantly (49.18 to 70.95; p < .001), and the intervention group's score decreased (69.06 to 68.33). From the initial assessment to the third assessment, the intervention group's knowledge score increased significantly (52.67 to 68.33; p < .001).
Studies Measuring Outcome Study 1
Study Designs Experimental
Quality of Research Rating 1.5 (0.0-4.0 scale)

Study Populations

The following populations were identified in the studies reviewed for Quality of Research.

Study Age Gender Race/Ethnicity
Study 1 6-12 (Childhood) 50.8% Female
49.2% Male
73.7% White
15.3% Black or African American
5.1% Asian
4.2% American Indian or Alaska Native
1.7% Race/ethnicity unspecified

Quality of Research Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the Quality of Research for an intervention's reported results using six criteria:

  1. Reliability of measures
  2. Validity of measures
  3. Intervention fidelity
  4. Missing data and attrition
  5. Potential confounding variables
  6. Appropriateness of analysis

For more information about these criteria and the meaning of the ratings, see Quality of Research.

Outcome Reliability
of Measures
Validity
of Measures
Fidelity Missing
Data/Attrition
Confounding
Variables
Data
Analysis
Overall
Rating
1: Attitudes toward science and drugs 2.0 2.0 1.5 1.3 1.8 1.5 1.7
2: Knowledge of science and drugs 1.0 2.0 1.5 1.3 1.8 1.5 1.5

Study Strengths

Use of a randomized wait-list control group reduced some threats to internal validity. The measures used had face validity.

Study Weaknesses

The documentation reviewed did not provide adequate information to support the psychometric properties of the measures used (e.g., Cronbach's alpha values were not reported for the knowledge items, items were created by the developer, there was no documentation of reliability and validity at acceptable levels by independent investigators, only face validity and content validity of the measures were assessed through consultant review and not documented by independent researchers). Intervention fidelity was assessed through usage logs, but it is unclear how they were analyzed and what the analysis indicated (e.g., no data were provided on dosage, the number of minutes children spent on each lesson, or the number of children who completed some or all of the modules). Differential attrition between the experimental and wait-list control groups was not analyzed. Although certain variables, such as age and gender, were compared, other key outcome-related variables were not used for comparison of those who did and did not complete the assessments. Missing data for the knowledge scale were treated as incorrect instead of missing. The analyses used were not strong enough to support the stated outcomes of the study; that is, it was not clear whether the analyses accounted for participants' pretest scores and any potentially confounding variables.

Readiness for Dissemination
Review Date: April 2010

Materials Reviewed

The materials below were reviewed for Readiness for Dissemination. The implementation point of contact can provide information regarding implementation of the intervention and the availability of additional, updated, or new materials.

Program Web site, http://www.BrainTrain4Kids.com

Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)

External reviewers independently evaluate the intervention's Readiness for Dissemination using three criteria:

  1. Availability of implementation materials
  2. Availability of training and support resources
  3. Availability of quality assurance procedures

For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.

Implementation
Materials
Training and Support
Resources
Quality Assurance
Procedures
Overall
Rating
2.5 1.5 2.5 2.2

Dissemination Strengths

The program's Web site is engaging and easy to navigate. Several hands-on, offline activities are available for users who want to extend and practice the skills learned online. Information provided in the Grown-Up Guide can help parents, teachers, and other adults assist children participating in the program. Pre- and posttest measures for attitudes and knowledge are available for parents or other adults to monitor children's outcomes.

Dissemination Weaknesses

No guidance is provided for implementing the intervention with users who have limited reading skills or who do not have access to a computer and a high-speed Internet connection. Some materials include typographical errors and content that does not seem logical. No guidance is provided to ensure that participants can read and comprehend the questions being asked in the self-report quality assurance tools provided. No guidance is provided to adults for addressing situations where the intervention or questionnaire content creates anxiety in young children, a possible occurrence when a family member has issues with alcohol or drugs. Although users can submit queries through a Web-based contact form, no direct access to developer staff (e.g., in-person, phone, or email contact information) is readily available as a source for technical assistance to new implementers.

Costs

The cost information below was provided by the developer. Although this cost information may have been updated by the developer since the time of review, it may not reflect the current costs or availability of items (including newly developed or discontinued items). The implementation point of contact can provide current information and discuss implementation requirements.

Item Description Cost Required by Developer
BrainTrain4Kids Web site (includes Grown-Up Guide, supplemental activity sheets, and assessment tools) Free Yes
Materials for group usage (includes hard copies of Grown-Up Guide and assessment tools, as well as aggregated data on group participants' use of the site) $25 per group No

Additional Information

Access to a computer with an Internet connection is required. In addition, access to a printer is required if the user opts to print the supplemental activities (e.g., crossword puzzles, coloring sheets).

Replications

No replications were identified by the developer.

Contact Information

To learn more about implementation or research, contact:
Mary P. Metcalf, Ph.D., M.P.H., CHES
(919) 960-8118
metcalf@clinicaltools.com; BrainTrain4Kids@clinicaltools.com

Consider these Questions to Ask (PDF, 54KB) as you explore the possible use of this intervention.

Web Site(s):