The relationship between missed breast cancers on mammography in a test-set based assessment scheme and real-life performance in a National Breast Screening Programme

Yan Chen; Jonathan J. James; Eleni Michalopoulou; Iain T. Darker; Jacquie Jenkins

doi:10.1016/j.ejrad.2021.109881

The relationship between missed breast cancers on mammography in a test-set based assessment scheme and real-life performance in a National Breast Screening Programme

Yan Chen^*, Jonathan J. James, Eleni Michalopoulou, Iain T. Darker, Jacquie Jenkins

^*Corresponding author for this work

HI: YPA Screening Team

Research output: Contribution to journal › Article › peer-review

Abstract

Purpose: This retrospective study determined whether a test-set based assessment scheme (PERFORMS) used in a national breast screening programme could be used to predict real-life performance by investigating if the number of cancers missed by mammography readers in real-life related to the number of cancers missed in the PERFORMS test-set and whether real-life reading volumes affected performance.

Method: Data was obtained from consenting readers in the screening programme in England (NHSBSP) where double reading is standard. The rate of cancers missed by individual first readers but correctly identified by second readers was compared with the number of cancers missed in the PERFORMS test-set over a 3-year period. NHSBSP readers are required to interpret at least 1500 cases per year as a first reader, so results were compared between readers who exceeded this target and those that did not. Parametric and non-parametric correlations were calculated.

Results: Amongst the 536 readers, there was a highly significant positive correlation between the real-life and PERFORMS test-set missed cancer metrics (Pearson Correlation = 0.228, n = 536, p < .0001, Spearman's rho = 0.265, n = 536, p < .0001). There was no significant difference in rates of missed cancers between the 452 readers who exceeded the 1500 first read per year target and those who did not (t(94.2) = −1.87, p = .0643, r = 0.19).

Conclusions: The use of a test-set based assessment scheme accurately reflects real-life mammography reading performance, indicating that it can be a useful tool in identifying poor reader performance.

Original language	English
Article number	109881
Journal	European Journal of Radiology
Volume	142
Early online date	28 Jul 2021
DOIs	https://doi.org/10.1016/j.ejrad.2021.109881
Publication status	Published - Sept 2021

Bibliographical note

Funding Information: The PERFORMS test scheme was funded by Public Health England, Breast Screening Programme. The funding source had no level of involvement in this study.

Open Access: No Open Access licence.

Publisher Copyright: Crown Copyright © 2021 Published by Elsevier B.V. All rights reserved.

Citation: Yan Chen, Jonathan J. James, Eleni Michalopoulou, Iain T. Darker, Jacquie Jenkins, The relationship between missed breast cancers on mammography in a test-set based assessment scheme and real-life performance in a National Breast Screening Programme, European Journal of Radiology, Volume 142, 2021, 109881, ISSN 0720-048X.

DOI: https://doi.org/10.1016/j.ejrad.2021.109881.

Keywords

Breast cancer
Mammography
Performance
Screening

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1016/j.ejrad.2021.109881

Cite this

@article{0b4fc9a1c20a43f1a87d5b0bb7d50704,

title = "The relationship between missed breast cancers on mammography in a test-set based assessment scheme and real-life performance in a National Breast Screening Programme",

abstract = "Purpose: This retrospective study determined whether a test-set based assessment scheme (PERFORMS) used in a national breast screening programme could be used to predict real-life performance by investigating if the number of cancers missed by mammography readers in real-life related to the number of cancers missed in the PERFORMS test-set and whether real-life reading volumes affected performance. Method: Data was obtained from consenting readers in the screening programme in England (NHSBSP) where double reading is standard. The rate of cancers missed by individual first readers but correctly identified by second readers was compared with the number of cancers missed in the PERFORMS test-set over a 3-year period. NHSBSP readers are required to interpret at least 1500 cases per year as a first reader, so results were compared between readers who exceeded this target and those that did not. Parametric and non-parametric correlations were calculated. Results: Amongst the 536 readers, there was a highly significant positive correlation between the real-life and PERFORMS test-set missed cancer metrics (Pearson Correlation = 0.228, n = 536, p < .0001, Spearman's rho = 0.265, n = 536, p < .0001). There was no significant difference in rates of missed cancers between the 452 readers who exceeded the 1500 first read per year target and those who did not (t(94.2) = −1.87, p = .0643, r = 0.19). Conclusions: The use of a test-set based assessment scheme accurately reflects real-life mammography reading performance, indicating that it can be a useful tool in identifying poor reader performance.",

keywords = "Breast cancer, Mammography, Performance, Screening",

author = "Yan Chen and James, {Jonathan J.} and Eleni Michalopoulou and Darker, {Iain T.} and Jacquie Jenkins",

note = "Funding Information: The PERFORMS test scheme was funded by Public Health England, Breast Screening Programme. The funding source had no level of involvement in this study. Open Access: No Open Access licence. Publisher Copyright: Crown Copyright {\textcopyright} 2021 Published by Elsevier B.V. All rights reserved. Citation: Yan Chen, Jonathan J. James, Eleni Michalopoulou, Iain T. Darker, Jacquie Jenkins, The relationship between missed breast cancers on mammography in a test-set based assessment scheme and real-life performance in a National Breast Screening Programme, European Journal of Radiology, Volume 142, 2021, 109881, ISSN 0720-048X. DOI: https://doi.org/10.1016/j.ejrad.2021.109881.",

year = "2021",

month = sep,

doi = "10.1016/j.ejrad.2021.109881",

language = "English",

volume = "142",

journal = "European Journal of Radiology",

issn = "0720-048X",

publisher = "Elsevier",

}

TY - JOUR

T1 - The relationship between missed breast cancers on mammography in a test-set based assessment scheme and real-life performance in a National Breast Screening Programme

AU - Chen, Yan

AU - James, Jonathan J.

AU - Michalopoulou, Eleni

AU - Darker, Iain T.

AU - Jenkins, Jacquie

N1 - Funding Information: The PERFORMS test scheme was funded by Public Health England, Breast Screening Programme. The funding source had no level of involvement in this study. Open Access: No Open Access licence. Publisher Copyright: Crown Copyright © 2021 Published by Elsevier B.V. All rights reserved. Citation: Yan Chen, Jonathan J. James, Eleni Michalopoulou, Iain T. Darker, Jacquie Jenkins, The relationship between missed breast cancers on mammography in a test-set based assessment scheme and real-life performance in a National Breast Screening Programme, European Journal of Radiology, Volume 142, 2021, 109881, ISSN 0720-048X. DOI: https://doi.org/10.1016/j.ejrad.2021.109881.

PY - 2021/9

Y1 - 2021/9

N2 - Purpose: This retrospective study determined whether a test-set based assessment scheme (PERFORMS) used in a national breast screening programme could be used to predict real-life performance by investigating if the number of cancers missed by mammography readers in real-life related to the number of cancers missed in the PERFORMS test-set and whether real-life reading volumes affected performance. Method: Data was obtained from consenting readers in the screening programme in England (NHSBSP) where double reading is standard. The rate of cancers missed by individual first readers but correctly identified by second readers was compared with the number of cancers missed in the PERFORMS test-set over a 3-year period. NHSBSP readers are required to interpret at least 1500 cases per year as a first reader, so results were compared between readers who exceeded this target and those that did not. Parametric and non-parametric correlations were calculated. Results: Amongst the 536 readers, there was a highly significant positive correlation between the real-life and PERFORMS test-set missed cancer metrics (Pearson Correlation = 0.228, n = 536, p < .0001, Spearman's rho = 0.265, n = 536, p < .0001). There was no significant difference in rates of missed cancers between the 452 readers who exceeded the 1500 first read per year target and those who did not (t(94.2) = −1.87, p = .0643, r = 0.19). Conclusions: The use of a test-set based assessment scheme accurately reflects real-life mammography reading performance, indicating that it can be a useful tool in identifying poor reader performance.

AB - Purpose: This retrospective study determined whether a test-set based assessment scheme (PERFORMS) used in a national breast screening programme could be used to predict real-life performance by investigating if the number of cancers missed by mammography readers in real-life related to the number of cancers missed in the PERFORMS test-set and whether real-life reading volumes affected performance. Method: Data was obtained from consenting readers in the screening programme in England (NHSBSP) where double reading is standard. The rate of cancers missed by individual first readers but correctly identified by second readers was compared with the number of cancers missed in the PERFORMS test-set over a 3-year period. NHSBSP readers are required to interpret at least 1500 cases per year as a first reader, so results were compared between readers who exceeded this target and those that did not. Parametric and non-parametric correlations were calculated. Results: Amongst the 536 readers, there was a highly significant positive correlation between the real-life and PERFORMS test-set missed cancer metrics (Pearson Correlation = 0.228, n = 536, p < .0001, Spearman's rho = 0.265, n = 536, p < .0001). There was no significant difference in rates of missed cancers between the 452 readers who exceeded the 1500 first read per year target and those who did not (t(94.2) = −1.87, p = .0643, r = 0.19). Conclusions: The use of a test-set based assessment scheme accurately reflects real-life mammography reading performance, indicating that it can be a useful tool in identifying poor reader performance.

KW - Breast cancer

KW - Mammography

KW - Performance

KW - Screening

UR - http://www.scopus.com/inward/record.url?scp=85111591762&partnerID=8YFLogxK

U2 - 10.1016/j.ejrad.2021.109881

DO - 10.1016/j.ejrad.2021.109881

M3 - Article

AN - SCOPUS:85111591762

SN - 0720-048X

VL - 142

JO - European Journal of Radiology

JF - European Journal of Radiology

M1 - 109881

ER -

The relationship between missed breast cancers on mammography in a test-set based assessment scheme and real-life performance in a National Breast Screening Programme

Abstract

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this