Validation of an ICD-10-based algorithm to identify stillbirth in the Sentinel System

Susan E. Andrade*, Mayura Shinde, Tiffany A. Moore Simas, Steven T. Bird, Justin Bohn, Kevin Haynes, Lockwood G. Taylor, Julianne R. Lauring, Erin Longley, Cheryl N. McMahill-Walraven, Connie M. Trinacty, Cassandra Saphirak, Christopher Delude, Sandra DeLuccia, Tancy Zhang, David V. Cole, Nina DiNunzio, Autumn Gertz, Elnara Fazio-Eynullayeva, Danijela Stojanovic

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Purpose: To develop and validate an International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM)-based algorithm to identify cases of stillbirth using electronic healthcare data. Methods: We conducted a retrospective study using claims data from three Data Partners (healthcare systems and insurers) in the Sentinel Distributed Database. Algorithms were developed using ICD-10-CM diagnosis codes to identify potential stillbirths among females aged 12–55 years between July 2016 and June 2018. A random sample of medical charts (N = 169) was identified for chart abstraction and adjudication. Two physician adjudicators reviewed potential cases to determine whether a stillbirth event was definite/probable, the date of the event, and the gestational age at delivery. Positive predictive values (PPVs) were calculated for the algorithms. Among confirmed cases, agreement between the claims data and medical charts was determined for the outcome date and gestational age at stillbirth. Results: Of the 110 potential cases identified, adjudicators determined that 54 were stillbirth events. Criteria for the algorithm with the highest PPV (82.5%; 95% CI, 70.9%–91.0%) included the presence of a diagnosis code indicating gestational age ≥20 weeks and occurrence of either >1 stillbirth-related code or no other pregnancy outcome code (i.e., livebirth, spontaneous abortion, induced abortion) recorded on the index date. We found ≥90% agreement within 7 days between the claims data and medical charts for both the outcome date and gestational age at stillbirth. Conclusions: Our results suggest that electronic healthcare data may be useful for signal detection of medical product exposures potentially associated with stillbirth.

Original languageEnglish
Pages (from-to)1175-1183
Number of pages9
JournalPharmacoepidemiology and Drug Safety
Issue number9
Publication statusPublished - Sep 2021
Externally publishedYes

Bibliographical note

Funding Information:
S.E.A. has received grant support from Pfizer Inc. and GlaxoSmithKline. S.T.B., L.G.T., and D.S. are employed by the U.S. Food and Drug Administration.

Funding Information:
We thank the Data Partners who provided data used in the analysis: CVS Health Clinical Trial Services, part of the CVS Health family of companies; Healthcore, Inc./Anthem, Inc.; and Kaiser Permanente Center for Integrated Health Care Research Hawaii. The authors would like to acknowledge the contributions of Inna Dashevsky (Harvard Pilgrim Health Care Institute); Mary Ellen Stansky and Timothy Konola (Meyers Primary Care Institute); Anne M. Kline and Smita Bhatia (CVS Health Clinical Trial Services, part of the CVS Health family of companies); Lauren Parlett, Mark Paullin, and Shia Kent (HealthCore, Inc.); and Valentyna Pishchalenko, Joanne Mor, and Samantha Wong (Kaiser Hawaii), and thank them for their assistance with this project.

Publisher Copyright:
© 2021 John Wiley & Sons Ltd.


  • claims data
  • Food and Drug Administration
  • pharmacoepidemiology
  • Sentinel
  • stillbirth
  • validation


Dive into the research topics of 'Validation of an ICD-10-based algorithm to identify stillbirth in the Sentinel System'. Together they form a unique fingerprint.

Cite this