Study 1

Lange, J. B., Lauer, E. M., & Voas, R. B. (1999). A survey of the San Diego-Tijuana cross-border binging. Methods and analysis. Evaluation Review, 23(4), 378-398.  

Romano, E., Cano, S., Lauer, E., Jimenez, A., Voas, R. B., & Lange, J. E. (2004). Tijuana alcohol control policies: A response to cross-border high-risk drinking by young Americans. Prevention Science, 5(2), 127-134.  

Voas, R. B., Tippetts, A. S., Johnson, M. B., Lange, J. E., & Baker, J. (2002). Operation safe crossing: Using science within a community intervention. Addiction, 97(9), 1205-1214.  

Study Strengths

For outcome 1 (reduced arrests for alcohol-related violations), the measure used has face validity. The validity is also supported by the decline in the number of alcohol-related misdemeanors, along with the decline in the number of American visitors and the decline in the numbers of youth with high BACs. The data provide a source of criterion validity for other intervention effects found. While the reliability of the data (police arrest records) may be questionable, expected effects were documented by this external data source, and effects were specific to misdemeanors and alcohol-related misdemeanors.

For outcome 2 (reduced number of bars with a majority of American customers), reports conducted by one observer were corroborated 95% of the time by returning bar patrons, providing evidence of the reliability of the data (interrater reliability) and support validity. Similarly, Mexican intervention efforts were demonstrated by the bar surveys (e.g., changes in signage, increase in police security) and corroborated by returning visitors. The sampling strategy (visiting each bar early and late on three evenings 1 random week per month) should yield representative data over time.

For outcome 3 (reduced nighttime alcohol-related crashes), potential confounders like a broader downward trend in "had been drinking" crashes were addressed by analyzing crashes in surrounding counties; effects were isolated to San Diego. The study used innovative methods to demonstrate the effects of intervention (outcome measures and media campaign) on the reduction of youth drinking and drinking-related problems. Carefully thought-out measures were implemented to control the quality of data collected. The researchers did a great job using a variety of data sources; the convergence of findings across all of these measures supports the efficacy of the program despite any individual measurement limitations. Thus, while each individual variable has limitations, it is the collective evidence across the set of diverse indicators that converges on a convincing argument supporting causal inference.

For outcome 4 (reduced number of youth crossing into Mexico to drink), direct recording of the outcome of interest was utilized. A lengthy baseline period was also available for comparison purposes to effectively isolate changes associated with the intervention period. Time series analysis, incorporating potential covariates like seasonality, is the most appropriate method for analyzing these data. The number of observations was more than adequate.

Outcome 5 (reduced number of youths returning from Tijuana with high BACs) used the Voluntary Breath Alcohol Survey (VBAS), a very reliable measurement of BACs for youths returning from Tijuana. The outcome utilized objective physiological data on BAC, correlated with interviewer ratings. Data collection procedures and reliance on objective BAC level support the validity of this measure. The detailed system for training and maintaining high level of fidelity to the data collection system, ongoing monitoring, and correspondence with interviewer ratings support fidelity. The authors demonstrated very sophisticated methods of handling missing data for this outcome. The use of BAC (as opposed to reliance on the survey) and high participation rates are strengths of this outcome.

Overall, the study had a relatively high participation rate, and measures were included to attempt to control missing data. Analyses appear appropriate for the outcomes.

Study Weaknesses

For outcome 1 (reduced arrests for alcohol-related violations), arrest records may include non-alcohol-related violations. The measurements also heavily depend on the accurate record provided by the Tijuana police. A potential confound is whether police were either enforcing the law less or reporting arrests less over time as the novelty of the intervention and reporting wore off. It is unclear how reliable source the police records are, and there was no baseline period for comparison since data were not recorded prior to the onset of the intervention.

For outcome 2 (reduction in the number of bars with a majority of American customers), reduction in the number of bars with a majority of U.S. customers may be affected by other factors. The bar survey also depends on the quality of surveys conducted by local surveyors.

For outcome 3 (reduced nighttime alcohol-related crashes), records from California's Statewide Integrated Traffic Reporting System (SWITRS) are a rough indicator of the reduction in nighttime alcohol-related crashes caused by heavy drinking in Tijuana. Many other factors could contribute to traffic accidents. It is not clear how officers determine that drivers "had been drinking," and no data was presented as to whether this determination is reliable. No evidence was presented that the decline in "had been drinking" crashes is attributable to less drinking while in Mexico. There was no reference to missing data on this variable.

For outcome 4 (reduced number of youth crossing into Mexico to drink), INS turnstile counter records are a good measure but may not necessarily reflect all activities (i.e., nondrinking activities) of those crossing into Tijuana.

For outcome 5 (reduced number of youths returning from Tijuana with high BACs), the most likely confound is a broader downward trend in BACs among people returning from Mexico, and baseline BACs were not recorded.

Overall, the intervention is unique and innovative, but it is also difficult to control quality of all intervention measures for consistency and potential replication. Random selection might have been compromised due to uncontrollable factors (selection bias, response bias, and other social, economic, and legal factors). Some of the outcome measures may only be partially related to the intervention. With no experimental control, it is impossible to rule out other confounds.