Harwood, J.M., Weiss, R.E., Comulada, W.S. Prevention Science (April 2017), e-publication ahead of print, doi:10.1007/s11121-017-0788-y.
The authors proposed an alternative approach for comparing intervention versus control outcomes in complex interventions. Behavioral interventions are generally measured using multiple endpoints, including risk behavior, biological measurements, and health outcomes. Often, these outcomes are difficult to measure because of multiple correlations, making it difficult to ascribe specific outcomes to the intervention. The usual approach is to use the "sign test" to highlight differences, such as in pre- and post-intervention outcomes. The proposed binomial approach counted the number of significant treatment/control differences, and accounted for correlations among the outcomes. The authors used Monte Carlo simulation (which adjusts for correlation and provides updated critical values and p values) to examine the Philani Intervention Program (PIP) in South Africa, an intervention targeting mothers and children that measured 28 outcomes including maternal alcohol use, malnutrition, and HIV. This approach overcame the risk of false positive results and showed, for example, that PIP yielded significantly better outcomes in maternal and infant wellbeing over six months, compared to standard care. The authors advocated for further research on other solutions for identifying the outcomes of multi-outcome studies.