A problem that is frequently encountered in many areas of scientific research is that of estimating the effect of a non-randomized binary intervention on an outcome of interest by using time series data on units that received the intervention (‘treated’) and units that did not (‘controls’). One popular estimation method in this setting is based on the factor analysis (FA) model. The FA model is fitted to the preintervention outcome data on treated units and all the outcome data on control units, and the counterfactual treatment-free post-intervention outcomes of the former are predicted from the fitted model. Intervention effects are estimated as the observed outcomes minus these predicted counterfactual outcomes. We propose a model that extends the FA model for estimating intervention effects by jointly modelling the multiple outcomes to exploit shared variability, and assuming an auto-regressive structure on factors to account for temporal correlations in the outcome. Using simulation studies, we show that the method proposed can improve the precision of the intervention effect estimates and achieve better control of the type I error rate (compared with the FA model), especially when either the number of preintervention measurements or the number of control units is small. We apply our method to estimate the effect of stricter alcohol licensing policies on alcohol-related harms.
|Number of pages||23|
|Journal||Journal of the Royal Statistical Society. Series A: Statistics in Society|
|Early online date||8 May 2020|
|Publication status||Published - 5 Oct 2020|
Bibliographical noteFunding Information: The authors thank Frank de Vocht for useful discussions regarding the real data application. This work is funded by the National Institute for Health Research Health Protection Unit on Evaluation of Interventions (to PS and MH), Medical Research Council grants MC_UU_00002/10 (to SRS) and MC_UU_00002/11 (to DDeA), and by Public Health England (to DDeA). Dr Montagna was partially supported by grant MONS_RILO_18_02 from the University of Turin. This study is further funded by the National Institute for Health Research programme grants for applied research programme (grant RP‐PG‐0616‐20008). The views expressed are those of the author(s) and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care.
Open Access: This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
Publisher Copyright: © 2020 The Authors Journal of the Royal Statistical Society: Series A (Statistics in Society) published by John Wiley & Sons Ltd on behalf of Royal Statistical Society.
Citation: Samartsidis, P., Seaman, S.R., Montagna, S., Charlett, A., Hickman, M. and Angelis, D.D. (2020), A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes. J. R. Stat. Soc. A, 183: 1437-1459.
- Causal inference
- Factor analysis
- Intervention evaluation
- Panel data