Increasing test specificity without impairing sensitivity: Lessons learned from SARS-CoV-2 serology

Thomas Perkmann; Thomas Koller; Nicole Perkmann-Nagele; Maria Ozsvar-Kozma; David Eyre; Philippa Matthews; Abbie Bown; Nicole Stoesser; Marie Kathrin Breyer; Robab Breyer-Kohansal; Otto C. Burghuber; Slyvia Hartl; Daniel Aletaha; Daniela Sieghart; Peter Quehenberger; Rodrig Marculescu; Patrick Mucher; Astrid Radakovics; Miriam Klausberger; Mark Duerkop; Barba Holzer; Boris Hartmann; Robert Strassl; Gerda Leitner; Florian Grebien; Wilhelm Gerner; Reingard Grabherr; Oswald F. Wagner; Christoph J. Binder; Helmuth Haslacher

doi:10.1136/jcp-2022-208171

Increasing test specificity without impairing sensitivity: Lessons learned from SARS-CoV-2 serology

Thomas Perkmann, Thomas Koller, Nicole Perkmann-Nagele, Maria Ozsvar-Kozma, David Eyre, Philippa Matthews, Abbie Bown, Nicole Stoesser, Marie Kathrin Breyer, Robab Breyer-Kohansal, Otto C. Burghuber, Slyvia Hartl, Daniel Aletaha, Daniela Sieghart, Peter Quehenberger, Rodrig Marculescu, Patrick Mucher, Astrid Radakovics, Miriam Klausberger, Mark DuerkopBarba Holzer, Boris Hartmann, Robert Strassl, Gerda Leitner, Florian Grebien, Wilhelm Gerner, Reingard Grabherr, Oswald F. Wagner, Christoph J. Binder, Helmuth Haslacher^*

^*Corresponding author for this work

RIPL: Lyme

Research output: Contribution to journal › Article › peer-review

2 Citations (Scopus)

Abstract

Background Serological tests are widely used in various medical disciplines for diagnostic and monitoring purposes. Unfortunately, the sensitivity and specificity of test systems are often poor, leaving room for false-positive and false-negative results. However, conventional methods were used to increase specificity and decrease sensitivity and vice versa. Using SARS-CoV-2 serology as an example, we propose here a novel testing strategy: the 'sensitivity improved two-test' or 'SIT²' algorithm. Methods SIT² involves confirmatory retesting of samples with results falling in a predefined retesting zone of an initial screening test, with adjusted cut-offs to increase sensitivity. We verified and compared the performance of SIT² to single tests and orthogonal testing (OTA) in an Austrian cohort (1117 negative, 64 post-COVID-positive samples) and validated the algorithm in an independent British cohort (976 negatives and 536 positives). Results The specificity of SIT² was superior to single tests and non-inferior to OTA. The sensitivity was maintained or even improved using SIT² when compared with single tests or OTA. SIT² allowed correct identification of infected individuals even when a live virus neutralisation assay could not detect antibodies. Compared with single testing or OTA, SIT² significantly reduced total test errors to 0.46% (0.24-0.65) or 1.60% (0.94-2.38) at both 5% or 20% seroprevalence. Conclusion For SARS-CoV-2 serology, SIT² proved to be the best diagnostic choice at both 5% and 20% seroprevalence in all tested scenarios. It is an easy to apply algorithm and can potentially be helpful for the serology of other infectious diseases.

Original language	English
Pages (from-to)	770-777
Number of pages	8
Journal	Journal of Clinical Pathology
Volume	76
Issue number	11
Early online date	30 Aug 2022
DOIs	https://doi.org/10.1136/jcp-2022-208171
Publication status	E-pub ahead of print - 30 Aug 2022

Bibliographical note

Funding Information:
The MedUni Wien Biobank is funded to participate in the biobank consortium BBMRI.at (www.bbmri.at) by the Austrian Federal Ministry of Science, Research and Technology. There was no external funding received for the work presented. However, test kits for the Technoclone ELISAs were kindly provided by the manufacturer.

Funding Information:
NP-N received a travel grant from DiaSorin. DWE reports lecture fees from Gilead outside the submitted work. OCB reports grants from GSK, grants from Menarini, grants from Boehringer Ingelheim, grants from Astra, grants from MSD, grants from Pfizer, and grants from Chiesi, outside the submitted work. SH does receive unrestricted research grants (GSK, Boehringer, Menarini, Chiesi, Astra Zeneca, MSD, Novartis, Air Liquide, Vivisol, Pfizer, TEVA) for the Ludwig Boltzmann Institute of COPD and Respiratory Epidemiology, and is on advisory boards for G. SK, Boehringer Ingelheim, Novartis, Menarini, Chiesi, Astra Zeneca, MSD, Roche, Abbvie, Takeda and TEVA for respiratory oncology and COPD. PQ is an advisory board member for Roche Austria and reports personal fees from Takeda outside the submitted work. The Dept. of Laboratory Medicine (Head: OWF) received compensations for advertisement on scientific symposia from Roche, DiaSorin, and Abbott and holds a grant for evaluating an in-vitro diagnostic device from Roche. CJB is a Board Member of Technoclone. HH receives compensations for biobank services from Glock Health Science and Research and BlueSky immunotherapies.

Publisher Copyright:
© Author(s) (or their employer(s)) 2022. No commercial re-use. See rights and permissions. Published by BMJ.

Keywords

allergy and immunology
medical laboratory science
serology

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1136/jcp-2022-208171

Cite this

Perkmann, T., Koller, T., Perkmann-Nagele, N., Ozsvar-Kozma, M., Eyre, D., Matthews, P., Bown, A., Stoesser, N., Breyer, M. K., Breyer-Kohansal, R., Burghuber, O. C., Hartl, S., Aletaha, D., Sieghart, D., Quehenberger, P., Marculescu, R., Mucher, P., Radakovics, A., Klausberger, M., ... Haslacher, H. (2022). Increasing test specificity without impairing sensitivity: Lessons learned from SARS-CoV-2 serology. Journal of Clinical Pathology, 76(11), 770-777. Advance online publication. https://doi.org/10.1136/jcp-2022-208171

@article{24cd8bc317e64cbb8ef3de3615e4bd87,

title = "Increasing test specificity without impairing sensitivity: Lessons learned from SARS-CoV-2 serology",

abstract = "Background Serological tests are widely used in various medical disciplines for diagnostic and monitoring purposes. Unfortunately, the sensitivity and specificity of test systems are often poor, leaving room for false-positive and false-negative results. However, conventional methods were used to increase specificity and decrease sensitivity and vice versa. Using SARS-CoV-2 serology as an example, we propose here a novel testing strategy: the 'sensitivity improved two-test' or 'SIT²' algorithm. Methods SIT² involves confirmatory retesting of samples with results falling in a predefined retesting zone of an initial screening test, with adjusted cut-offs to increase sensitivity. We verified and compared the performance of SIT² to single tests and orthogonal testing (OTA) in an Austrian cohort (1117 negative, 64 post-COVID-positive samples) and validated the algorithm in an independent British cohort (976 negatives and 536 positives). Results The specificity of SIT² was superior to single tests and non-inferior to OTA. The sensitivity was maintained or even improved using SIT² when compared with single tests or OTA. SIT² allowed correct identification of infected individuals even when a live virus neutralisation assay could not detect antibodies. Compared with single testing or OTA, SIT² significantly reduced total test errors to 0.46% (0.24-0.65) or 1.60% (0.94-2.38) at both 5% or 20% seroprevalence. Conclusion For SARS-CoV-2 serology, SIT² proved to be the best diagnostic choice at both 5% and 20% seroprevalence in all tested scenarios. It is an easy to apply algorithm and can potentially be helpful for the serology of other infectious diseases.",

keywords = "allergy and immunology, medical laboratory science, serology",

author = "Thomas Perkmann and Thomas Koller and Nicole Perkmann-Nagele and Maria Ozsvar-Kozma and David Eyre and Philippa Matthews and Abbie Bown and Nicole Stoesser and Breyer, {Marie Kathrin} and Robab Breyer-Kohansal and Burghuber, {Otto C.} and Slyvia Hartl and Daniel Aletaha and Daniela Sieghart and Peter Quehenberger and Rodrig Marculescu and Patrick Mucher and Astrid Radakovics and Miriam Klausberger and Mark Duerkop and Barba Holzer and Boris Hartmann and Robert Strassl and Gerda Leitner and Florian Grebien and Wilhelm Gerner and Reingard Grabherr and Wagner, {Oswald F.} and Binder, {Christoph J.} and Helmuth Haslacher",

note = "Funding Information: The MedUni Wien Biobank is funded to participate in the biobank consortium BBMRI.at (www.bbmri.at) by the Austrian Federal Ministry of Science, Research and Technology. There was no external funding received for the work presented. However, test kits for the Technoclone ELISAs were kindly provided by the manufacturer. Funding Information: NP-N received a travel grant from DiaSorin. DWE reports lecture fees from Gilead outside the submitted work. OCB reports grants from GSK, grants from Menarini, grants from Boehringer Ingelheim, grants from Astra, grants from MSD, grants from Pfizer, and grants from Chiesi, outside the submitted work. SH does receive unrestricted research grants (GSK, Boehringer, Menarini, Chiesi, Astra Zeneca, MSD, Novartis, Air Liquide, Vivisol, Pfizer, TEVA) for the Ludwig Boltzmann Institute of COPD and Respiratory Epidemiology, and is on advisory boards for G. SK, Boehringer Ingelheim, Novartis, Menarini, Chiesi, Astra Zeneca, MSD, Roche, Abbvie, Takeda and TEVA for respiratory oncology and COPD. PQ is an advisory board member for Roche Austria and reports personal fees from Takeda outside the submitted work. The Dept. of Laboratory Medicine (Head: OWF) received compensations for advertisement on scientific symposia from Roche, DiaSorin, and Abbott and holds a grant for evaluating an in-vitro diagnostic device from Roche. CJB is a Board Member of Technoclone. HH receives compensations for biobank services from Glock Health Science and Research and BlueSky immunotherapies. Publisher Copyright: {\textcopyright} Author(s) (or their employer(s)) 2022. No commercial re-use. See rights and permissions. Published by BMJ.",

year = "2022",

month = aug,

day = "30",

doi = "10.1136/jcp-2022-208171",

language = "English",

volume = "76",

pages = "770--777",

journal = "Journal of Clinical Pathology",

issn = "0021-9746",

publisher = "BMJ Publishing Group",

number = "11",

}

Perkmann, T, Koller, T, Perkmann-Nagele, N, Ozsvar-Kozma, M, Eyre, D, Matthews, P, Bown, A, Stoesser, N, Breyer, MK, Breyer-Kohansal, R, Burghuber, OC, Hartl, S, Aletaha, D, Sieghart, D, Quehenberger, P, Marculescu, R, Mucher, P, Radakovics, A, Klausberger, M, Duerkop, M, Holzer, B, Hartmann, B, Strassl, R, Leitner, G, Grebien, F, Gerner, W, Grabherr, R, Wagner, OF, Binder, CJ & Haslacher, H 2022, 'Increasing test specificity without impairing sensitivity: Lessons learned from SARS-CoV-2 serology', Journal of Clinical Pathology, vol. 76, no. 11, pp. 770-777. https://doi.org/10.1136/jcp-2022-208171

TY - JOUR

T1 - Increasing test specificity without impairing sensitivity

T2 - Lessons learned from SARS-CoV-2 serology

AU - Perkmann, Thomas

AU - Koller, Thomas

AU - Perkmann-Nagele, Nicole

AU - Ozsvar-Kozma, Maria

AU - Eyre, David

AU - Matthews, Philippa

AU - Bown, Abbie

AU - Stoesser, Nicole

AU - Breyer, Marie Kathrin

AU - Breyer-Kohansal, Robab

AU - Burghuber, Otto C.

AU - Hartl, Slyvia

AU - Aletaha, Daniel

AU - Sieghart, Daniela

AU - Quehenberger, Peter

AU - Marculescu, Rodrig

AU - Mucher, Patrick

AU - Radakovics, Astrid

AU - Klausberger, Miriam

AU - Duerkop, Mark

AU - Holzer, Barba

AU - Hartmann, Boris

AU - Strassl, Robert

AU - Leitner, Gerda

AU - Grebien, Florian

AU - Gerner, Wilhelm

AU - Grabherr, Reingard

AU - Wagner, Oswald F.

AU - Binder, Christoph J.

AU - Haslacher, Helmuth

N1 - Funding Information: The MedUni Wien Biobank is funded to participate in the biobank consortium BBMRI.at (www.bbmri.at) by the Austrian Federal Ministry of Science, Research and Technology. There was no external funding received for the work presented. However, test kits for the Technoclone ELISAs were kindly provided by the manufacturer. Funding Information: NP-N received a travel grant from DiaSorin. DWE reports lecture fees from Gilead outside the submitted work. OCB reports grants from GSK, grants from Menarini, grants from Boehringer Ingelheim, grants from Astra, grants from MSD, grants from Pfizer, and grants from Chiesi, outside the submitted work. SH does receive unrestricted research grants (GSK, Boehringer, Menarini, Chiesi, Astra Zeneca, MSD, Novartis, Air Liquide, Vivisol, Pfizer, TEVA) for the Ludwig Boltzmann Institute of COPD and Respiratory Epidemiology, and is on advisory boards for G. SK, Boehringer Ingelheim, Novartis, Menarini, Chiesi, Astra Zeneca, MSD, Roche, Abbvie, Takeda and TEVA for respiratory oncology and COPD. PQ is an advisory board member for Roche Austria and reports personal fees from Takeda outside the submitted work. The Dept. of Laboratory Medicine (Head: OWF) received compensations for advertisement on scientific symposia from Roche, DiaSorin, and Abbott and holds a grant for evaluating an in-vitro diagnostic device from Roche. CJB is a Board Member of Technoclone. HH receives compensations for biobank services from Glock Health Science and Research and BlueSky immunotherapies. Publisher Copyright: © Author(s) (or their employer(s)) 2022. No commercial re-use. See rights and permissions. Published by BMJ.

PY - 2022/8/30

Y1 - 2022/8/30

N2 - Background Serological tests are widely used in various medical disciplines for diagnostic and monitoring purposes. Unfortunately, the sensitivity and specificity of test systems are often poor, leaving room for false-positive and false-negative results. However, conventional methods were used to increase specificity and decrease sensitivity and vice versa. Using SARS-CoV-2 serology as an example, we propose here a novel testing strategy: the 'sensitivity improved two-test' or 'SIT²' algorithm. Methods SIT² involves confirmatory retesting of samples with results falling in a predefined retesting zone of an initial screening test, with adjusted cut-offs to increase sensitivity. We verified and compared the performance of SIT² to single tests and orthogonal testing (OTA) in an Austrian cohort (1117 negative, 64 post-COVID-positive samples) and validated the algorithm in an independent British cohort (976 negatives and 536 positives). Results The specificity of SIT² was superior to single tests and non-inferior to OTA. The sensitivity was maintained or even improved using SIT² when compared with single tests or OTA. SIT² allowed correct identification of infected individuals even when a live virus neutralisation assay could not detect antibodies. Compared with single testing or OTA, SIT² significantly reduced total test errors to 0.46% (0.24-0.65) or 1.60% (0.94-2.38) at both 5% or 20% seroprevalence. Conclusion For SARS-CoV-2 serology, SIT² proved to be the best diagnostic choice at both 5% and 20% seroprevalence in all tested scenarios. It is an easy to apply algorithm and can potentially be helpful for the serology of other infectious diseases.

AB - Background Serological tests are widely used in various medical disciplines for diagnostic and monitoring purposes. Unfortunately, the sensitivity and specificity of test systems are often poor, leaving room for false-positive and false-negative results. However, conventional methods were used to increase specificity and decrease sensitivity and vice versa. Using SARS-CoV-2 serology as an example, we propose here a novel testing strategy: the 'sensitivity improved two-test' or 'SIT²' algorithm. Methods SIT² involves confirmatory retesting of samples with results falling in a predefined retesting zone of an initial screening test, with adjusted cut-offs to increase sensitivity. We verified and compared the performance of SIT² to single tests and orthogonal testing (OTA) in an Austrian cohort (1117 negative, 64 post-COVID-positive samples) and validated the algorithm in an independent British cohort (976 negatives and 536 positives). Results The specificity of SIT² was superior to single tests and non-inferior to OTA. The sensitivity was maintained or even improved using SIT² when compared with single tests or OTA. SIT² allowed correct identification of infected individuals even when a live virus neutralisation assay could not detect antibodies. Compared with single testing or OTA, SIT² significantly reduced total test errors to 0.46% (0.24-0.65) or 1.60% (0.94-2.38) at both 5% or 20% seroprevalence. Conclusion For SARS-CoV-2 serology, SIT² proved to be the best diagnostic choice at both 5% and 20% seroprevalence in all tested scenarios. It is an easy to apply algorithm and can potentially be helpful for the serology of other infectious diseases.

KW - allergy and immunology

KW - medical laboratory science

KW - serology

UR - http://www.scopus.com/inward/record.url?scp=85137691679&partnerID=8YFLogxK

U2 - 10.1136/jcp-2022-208171

DO - 10.1136/jcp-2022-208171

M3 - Article

C2 - 36041815

AN - SCOPUS:85137691679

SN - 0021-9746

VL - 76

SP - 770

EP - 777

JO - Journal of Clinical Pathology

JF - Journal of Clinical Pathology

IS - 11

ER -

Increasing test specificity without impairing sensitivity: Lessons learned from SARS-CoV-2 serology

Abstract

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this