Health impact assessment for air pollution in the presence of regional variation in effect sizes: The implications of using different meta-analytic approaches

The estimated health effects of air pollution vary between studies, and this variation is caused by factors associated with the study location, hereafter termed regional heterogeneity. This heterogeneity raises a methodological question as to which studies should be used to estimate risks in a specific region in a health impact assessment. Should one use all studies across the world, or only those in the region of interest? The current study provides novel insight into this question in two ways. Firstly, it presents an up-to-date analysis examining the magnitude of continent-level regional heterogeneity in the short-term health effects of air pollution, using a database of studies collected by Orellano et al. (2020). Secondly, it provides in-depth simulation analyses examining whether existing meta-analyses are likely to be underpowered to identify statistically significant regional heterogeneity, as well as evaluating which meta-analytic technique is best for estimating region-specific estimates. The techniques considered include global and continent-specific (sub-group) random effects meta-analysis and meta-regression, with omnibus statistical tests used to quantify regional heterogeneity. We find statistically significant regional heterogeneity for 4 of the 8 pollutant-outcome pairs considered, comprising NO₂, O₃ and PM_2.5 with all-cause mortality, and PM_2.5 with cardiovascular mortality. From the simulation analysis statistically significant regional heterogeneity is more likely to be identified as the number of studies increases (between 3 and 30 in each region were considered), between region heterogeneity increases and within region heterogeneity decreases. Finally, while a sub-group analysis using Cochran's Q test has a higher median power (0.71) than a test based on the moderators' coefficients from meta-regression (0.59) to identify regional heterogeneity, it also has an inflated type-1 error leading to more false positives (median errors of 0.15 compared to 0.09).