A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes

Pantelis Samartsidis; Shaun R. Seaman; Silvia Montagna; Andre Charlett; Matthew Hickman; Daniela De Angelis

doi:10.1111/rssa.12569

A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes

Pantelis Samartsidis^*, Shaun R. Seaman, Silvia Montagna, Andre Charlett, Matthew Hickman, Daniela De Angelis

^*Corresponding author for this work

Statistics Unit

Research output: Contribution to journal › Article › peer-review

11 Citations (Scopus)

12 Downloads (Pure)

Abstract

A problem that is frequently encountered in many areas of scientific research is that of estimating the effect of a non-randomized binary intervention on an outcome of interest by using time series data on units that received the intervention (‘treated’) and units that did not (‘controls’). One popular estimation method in this setting is based on the factor analysis (FA) model. The FA model is fitted to the preintervention outcome data on treated units and all the outcome data on control units, and the counterfactual treatment-free post-intervention outcomes of the former are predicted from the fitted model. Intervention effects are estimated as the observed outcomes minus these predicted counterfactual outcomes. We propose a model that extends the FA model for estimating intervention effects by jointly modelling the multiple outcomes to exploit shared variability, and assuming an auto-regressive structure on factors to account for temporal correlations in the outcome. Using simulation studies, we show that the method proposed can improve the precision of the intervention effect estimates and achieve better control of the type I error rate (compared with the FA model), especially when either the number of preintervention measurements or the number of control units is small. We apply our method to estimate the effect of stricter alcohol licensing policies on alcohol-related harms.

Original language	English
Pages (from-to)	1437-1459
Number of pages	23
Journal	Journal of the Royal Statistical Society. Series A: Statistics in Society
Volume	183
Issue number	4
Early online date	8 May 2020
DOIs	https://doi.org/10.1111/rssa.12569
Publication status	Published - 5 Oct 2020

Bibliographical note

Funding Information: The authors thank Frank de Vocht for useful discussions regarding the real data application. This work is funded by the National Institute for Health Research Health Protection Unit on Evaluation of Interventions (to PS and MH), Medical Research Council grants MC_UU_00002/10 (to SRS) and MC_UU_00002/11 (to DDeA), and by Public Health England (to DDeA). Dr Montagna was partially supported by grant MONS_RILO_18_02 from the University of Turin. This study is further funded by the National Institute for Health Research programme grants for applied research programme (grant RP‐PG‐0616‐20008). The views expressed are those of the author(s) and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care.

Open Access: This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Publisher Copyright: © 2020 The Authors Journal of the Royal Statistical Society: Series A (Statistics in Society) published by John Wiley & Sons Ltd on behalf of Royal Statistical Society.

Citation: Samartsidis, P., Seaman, S.R., Montagna, S., Charlett, A., Hickman, M. and Angelis, D.D. (2020), A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes. J. R. Stat. Soc. A, 183: 1437-1459.

DOI: https://doi.org/10.1111/rssa.12569

Keywords

Causal inference
Factor analysis
Intervention evaluation
Panel data

Access to Document

10.1111/rssa.12569Licence: CC BY

Samartsidis2020ABayesianMultivariateFactorAnalysisModelForEvaluatingJRStatSocSerAStatSocFinal published version, 2.84 MBLicence: CC BY

Cite this

@article{66ec7127a1e048fdbd6ab13d4e1bce93,

title = "A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes",

abstract = "A problem that is frequently encountered in many areas of scientific research is that of estimating the effect of a non-randomized binary intervention on an outcome of interest by using time series data on units that received the intervention ({\textquoteleft}treated{\textquoteright}) and units that did not ({\textquoteleft}controls{\textquoteright}). One popular estimation method in this setting is based on the factor analysis (FA) model. The FA model is fitted to the preintervention outcome data on treated units and all the outcome data on control units, and the counterfactual treatment-free post-intervention outcomes of the former are predicted from the fitted model. Intervention effects are estimated as the observed outcomes minus these predicted counterfactual outcomes. We propose a model that extends the FA model for estimating intervention effects by jointly modelling the multiple outcomes to exploit shared variability, and assuming an auto-regressive structure on factors to account for temporal correlations in the outcome. Using simulation studies, we show that the method proposed can improve the precision of the intervention effect estimates and achieve better control of the type I error rate (compared with the FA model), especially when either the number of preintervention measurements or the number of control units is small. We apply our method to estimate the effect of stricter alcohol licensing policies on alcohol-related harms.",

keywords = "Causal inference, Factor analysis, Intervention evaluation, Panel data",

author = "Pantelis Samartsidis and Seaman, {Shaun R.} and Silvia Montagna and Andre Charlett and Matthew Hickman and Angelis, {Daniela De}",

note = "Funding Information: The authors thank Frank de Vocht for useful discussions regarding the real data application. This work is funded by the National Institute for Health Research Health Protection Unit on Evaluation of Interventions (to PS and MH), Medical Research Council grants MC_UU_00002/10 (to SRS) and MC_UU_00002/11 (to DDeA), and by Public Health England (to DDeA). Dr Montagna was partially supported by grant MONS_RILO_18_02 from the University of Turin. This study is further funded by the National Institute for Health Research programme grants for applied research programme (grant RP‐PG‐0616‐20008). The views expressed are those of the author(s) and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care. Open Access: This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. Publisher Copyright: {\textcopyright} 2020 The Authors Journal of the Royal Statistical Society: Series A (Statistics in Society) published by John Wiley & Sons Ltd on behalf of Royal Statistical Society. Citation: Samartsidis, P., Seaman, S.R., Montagna, S., Charlett, A., Hickman, M. and Angelis, D.D. (2020), A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes. J. R. Stat. Soc. A, 183: 1437-1459. DOI: https://doi.org/10.1111/rssa.12569",

year = "2020",

month = oct,

day = "5",

doi = "10.1111/rssa.12569",

language = "English",

volume = "183",

pages = "1437--1459",

journal = "Journal of the Royal Statistical Society. Series A: Statistics in Society",

issn = "0964-1998",

publisher = "Oxford University Press",

number = "4",

}

A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes. / Samartsidis, Pantelis; Seaman, Shaun R.; Montagna, Silvia et al.
In: Journal of the Royal Statistical Society. Series A: Statistics in Society, Vol. 183, No. 4, 05.10.2020, p. 1437-1459.

Research output: Contribution to journal › Article › peer-review

TY - JOUR

T1 - A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes

AU - Samartsidis, Pantelis

AU - Seaman, Shaun R.

AU - Montagna, Silvia

AU - Charlett, Andre

AU - Hickman, Matthew

AU - Angelis, Daniela De

N1 - Funding Information: The authors thank Frank de Vocht for useful discussions regarding the real data application. This work is funded by the National Institute for Health Research Health Protection Unit on Evaluation of Interventions (to PS and MH), Medical Research Council grants MC_UU_00002/10 (to SRS) and MC_UU_00002/11 (to DDeA), and by Public Health England (to DDeA). Dr Montagna was partially supported by grant MONS_RILO_18_02 from the University of Turin. This study is further funded by the National Institute for Health Research programme grants for applied research programme (grant RP‐PG‐0616‐20008). The views expressed are those of the author(s) and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care. Open Access: This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. Publisher Copyright: © 2020 The Authors Journal of the Royal Statistical Society: Series A (Statistics in Society) published by John Wiley & Sons Ltd on behalf of Royal Statistical Society. Citation: Samartsidis, P., Seaman, S.R., Montagna, S., Charlett, A., Hickman, M. and Angelis, D.D. (2020), A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes. J. R. Stat. Soc. A, 183: 1437-1459. DOI: https://doi.org/10.1111/rssa.12569

PY - 2020/10/5

Y1 - 2020/10/5

N2 - A problem that is frequently encountered in many areas of scientific research is that of estimating the effect of a non-randomized binary intervention on an outcome of interest by using time series data on units that received the intervention (‘treated’) and units that did not (‘controls’). One popular estimation method in this setting is based on the factor analysis (FA) model. The FA model is fitted to the preintervention outcome data on treated units and all the outcome data on control units, and the counterfactual treatment-free post-intervention outcomes of the former are predicted from the fitted model. Intervention effects are estimated as the observed outcomes minus these predicted counterfactual outcomes. We propose a model that extends the FA model for estimating intervention effects by jointly modelling the multiple outcomes to exploit shared variability, and assuming an auto-regressive structure on factors to account for temporal correlations in the outcome. Using simulation studies, we show that the method proposed can improve the precision of the intervention effect estimates and achieve better control of the type I error rate (compared with the FA model), especially when either the number of preintervention measurements or the number of control units is small. We apply our method to estimate the effect of stricter alcohol licensing policies on alcohol-related harms.

AB - A problem that is frequently encountered in many areas of scientific research is that of estimating the effect of a non-randomized binary intervention on an outcome of interest by using time series data on units that received the intervention (‘treated’) and units that did not (‘controls’). One popular estimation method in this setting is based on the factor analysis (FA) model. The FA model is fitted to the preintervention outcome data on treated units and all the outcome data on control units, and the counterfactual treatment-free post-intervention outcomes of the former are predicted from the fitted model. Intervention effects are estimated as the observed outcomes minus these predicted counterfactual outcomes. We propose a model that extends the FA model for estimating intervention effects by jointly modelling the multiple outcomes to exploit shared variability, and assuming an auto-regressive structure on factors to account for temporal correlations in the outcome. Using simulation studies, we show that the method proposed can improve the precision of the intervention effect estimates and achieve better control of the type I error rate (compared with the FA model), especially when either the number of preintervention measurements or the number of control units is small. We apply our method to estimate the effect of stricter alcohol licensing policies on alcohol-related harms.

KW - Causal inference

KW - Factor analysis

KW - Intervention evaluation

KW - Panel data

UR - http://www.scopus.com/inward/record.url?scp=85085139727&partnerID=8YFLogxK

U2 - 10.1111/rssa.12569

DO - 10.1111/rssa.12569

M3 - Article

AN - SCOPUS:85085139727

SN - 0964-1998

VL - 183

SP - 1437

EP - 1459

JO - Journal of the Royal Statistical Society. Series A: Statistics in Society

JF - Journal of the Royal Statistical Society. Series A: Statistics in Society

IS - 4

ER -

A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this