Evaluation of emergency preparedness exercises: The design of a questionnaire to measure staff perceptions

Elena A. Skryabina; Naomi Betts; Gabriel Reedy; Paul Riley; Richard Amlôt

doi:10.1504/IJEM.2020.117209

Evaluation of emergency preparedness exercises: The design of a questionnaire to measure staff perceptions

Elena A. Skryabina^*, Naomi Betts, Gabriel Reedy, Paul Riley, Richard Amlôt

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

1 Citation (Scopus)

54 Downloads (Pure)

Abstract

Evaluation is an essential part of health emergency preparedness exercises (HEPE) that allows identification of limitations in performance. Addressing limitations enhances preparedness. However, there is a lack of reliable and validated tools to assist with exercise evaluation. This study reports the design and validation of a questionnaire to collect data from participants to study their experiences with HEPE and perceptions of their own and their organisation's emergency preparedness. Questionnaire test-retest reliability using ICC was checked (N = 27). Internal reliability using Cronbach's alpha is reported separately for discussion-based (N = 97) and operation-based exercises (N = 238). Analysis checked for discriminant validity and sensitivity to change. The questionnaire consists of four scales (parts): pre-exercise assessment, participant's perceptions, exercise feedback, and satisfaction with the exercise. All scales demonstrated good internal consistency for both exercise types (Cronbach's alpha: 0.672-0.940), but mixed test-retest agreement for the pre-exercise and exercise feedback scales. The questionnaire offers a valid and reliable tool for assessing healthcare staff perceptions of emergency preparedness and exercise satisfaction.

Original language	English
Pages (from-to)	317-346
Number of pages	30
Journal	International Journal of Emergency Management
Volume	16
Issue number	4
DOIs	https://doi.org/10.1504/IJEM.2020.117209
Publication status	Published - 23 Aug 2021

Bibliographical note

Funding Information: This study was funded by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response, a partnership between Public Health England, King’s College London and the University of East Anglia. The views expressed are those of the authors and not necessarily those of the NIHR, Public Health England or the Department of Health and Social Care. Professor Richard Amlôt is also supported by the NIHR HPRUs in Evaluation of Interventions at Bristol University and Modelling Methodology at Imperial College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health and Social Care or Public Health England.

Open Access: This is an Open Access Article distributed under the CC BY license. (http://creativecommons.org/licenses/by/4.0/)

Publisher Copyright: © The Author(s) 2020. Published by Inderscience Publishers Ltd.

Citation: Skryabina, Elena A., et al. "Evaluation of emergency preparedness exercises: the design of a questionnaire to measure staff perceptions." International Journal of Emergency Management 16.4 (2020): 317-346.

DOI: 10.1504/IJEM.2020.117209

Keywords

Disaster
Drill
Emergency preparedness
Evaluation
HEPE
Health emergency preparedness exercises
Healthcare emergency
Healthcare emergency responder
Major incident
Mass casualty
Operation exercise
Questionnaire
Simulation exercise
Survey
TTX
Tabletop

Access to Document

10.1504/IJEM.2020.117209Licence: CC BY

Skryabina2020UKHealthcareStaffExperiencesAndPerceptionsofAMassEmergMedJFinal published version, 334 KBLicence: CC BY

Cite this

@article{f4fb99bb0a9341c7a5958d0942390ef7,

title = "Evaluation of emergency preparedness exercises: The design of a questionnaire to measure staff perceptions",

abstract = "Evaluation is an essential part of health emergency preparedness exercises (HEPE) that allows identification of limitations in performance. Addressing limitations enhances preparedness. However, there is a lack of reliable and validated tools to assist with exercise evaluation. This study reports the design and validation of a questionnaire to collect data from participants to study their experiences with HEPE and perceptions of their own and their organisation's emergency preparedness. Questionnaire test-retest reliability using ICC was checked (N = 27). Internal reliability using Cronbach's alpha is reported separately for discussion-based (N = 97) and operation-based exercises (N = 238). Analysis checked for discriminant validity and sensitivity to change. The questionnaire consists of four scales (parts): pre-exercise assessment, participant's perceptions, exercise feedback, and satisfaction with the exercise. All scales demonstrated good internal consistency for both exercise types (Cronbach's alpha: 0.672-0.940), but mixed test-retest agreement for the pre-exercise and exercise feedback scales. The questionnaire offers a valid and reliable tool for assessing healthcare staff perceptions of emergency preparedness and exercise satisfaction.",

keywords = "Disaster, Drill, Emergency preparedness, Evaluation, HEPE, Health emergency preparedness exercises, Healthcare emergency, Healthcare emergency responder, Major incident, Mass casualty, Operation exercise, Questionnaire, Simulation exercise, Survey, TTX, Tabletop",

author = "Skryabina, {Elena A.} and Naomi Betts and Gabriel Reedy and Paul Riley and Richard Aml{\^o}t",

note = "Funding Information: This study was funded by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response, a partnership between Public Health England, King{\textquoteright}s College London and the University of East Anglia. The views expressed are those of the authors and not necessarily those of the NIHR, Public Health England or the Department of Health and Social Care. Professor Richard Aml{\^o}t is also supported by the NIHR HPRUs in Evaluation of Interventions at Bristol University and Modelling Methodology at Imperial College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health and Social Care or Public Health England. Open Access: This is an Open Access Article distributed under the CC BY license. (http://creativecommons.org/licenses/by/4.0/) Publisher Copyright: {\textcopyright} The Author(s) 2020. Published by Inderscience Publishers Ltd. Citation: Skryabina, Elena A., et al. {"}Evaluation of emergency preparedness exercises: the design of a questionnaire to measure staff perceptions.{"} International Journal of Emergency Management 16.4 (2020): 317-346. DOI: 10.1504/IJEM.2020.117209",

year = "2021",

month = aug,

day = "23",

doi = "10.1504/IJEM.2020.117209",

language = "English",

volume = "16",

pages = "317--346",

journal = "International Journal of Emergency Management",

issn = "1471-4825",

publisher = "Inderscience Enterprises Ltd",

number = "4",

}

TY - JOUR

T1 - Evaluation of emergency preparedness exercises

T2 - The design of a questionnaire to measure staff perceptions

AU - Skryabina, Elena A.

AU - Betts, Naomi

AU - Reedy, Gabriel

AU - Riley, Paul

AU - Amlôt, Richard

N1 - Funding Information: This study was funded by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response, a partnership between Public Health England, King’s College London and the University of East Anglia. The views expressed are those of the authors and not necessarily those of the NIHR, Public Health England or the Department of Health and Social Care. Professor Richard Amlôt is also supported by the NIHR HPRUs in Evaluation of Interventions at Bristol University and Modelling Methodology at Imperial College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health and Social Care or Public Health England. Open Access: This is an Open Access Article distributed under the CC BY license. (http://creativecommons.org/licenses/by/4.0/) Publisher Copyright: © The Author(s) 2020. Published by Inderscience Publishers Ltd. Citation: Skryabina, Elena A., et al. "Evaluation of emergency preparedness exercises: the design of a questionnaire to measure staff perceptions." International Journal of Emergency Management 16.4 (2020): 317-346. DOI: 10.1504/IJEM.2020.117209

PY - 2021/8/23

Y1 - 2021/8/23

N2 - Evaluation is an essential part of health emergency preparedness exercises (HEPE) that allows identification of limitations in performance. Addressing limitations enhances preparedness. However, there is a lack of reliable and validated tools to assist with exercise evaluation. This study reports the design and validation of a questionnaire to collect data from participants to study their experiences with HEPE and perceptions of their own and their organisation's emergency preparedness. Questionnaire test-retest reliability using ICC was checked (N = 27). Internal reliability using Cronbach's alpha is reported separately for discussion-based (N = 97) and operation-based exercises (N = 238). Analysis checked for discriminant validity and sensitivity to change. The questionnaire consists of four scales (parts): pre-exercise assessment, participant's perceptions, exercise feedback, and satisfaction with the exercise. All scales demonstrated good internal consistency for both exercise types (Cronbach's alpha: 0.672-0.940), but mixed test-retest agreement for the pre-exercise and exercise feedback scales. The questionnaire offers a valid and reliable tool for assessing healthcare staff perceptions of emergency preparedness and exercise satisfaction.

AB - Evaluation is an essential part of health emergency preparedness exercises (HEPE) that allows identification of limitations in performance. Addressing limitations enhances preparedness. However, there is a lack of reliable and validated tools to assist with exercise evaluation. This study reports the design and validation of a questionnaire to collect data from participants to study their experiences with HEPE and perceptions of their own and their organisation's emergency preparedness. Questionnaire test-retest reliability using ICC was checked (N = 27). Internal reliability using Cronbach's alpha is reported separately for discussion-based (N = 97) and operation-based exercises (N = 238). Analysis checked for discriminant validity and sensitivity to change. The questionnaire consists of four scales (parts): pre-exercise assessment, participant's perceptions, exercise feedback, and satisfaction with the exercise. All scales demonstrated good internal consistency for both exercise types (Cronbach's alpha: 0.672-0.940), but mixed test-retest agreement for the pre-exercise and exercise feedback scales. The questionnaire offers a valid and reliable tool for assessing healthcare staff perceptions of emergency preparedness and exercise satisfaction.

KW - Disaster

KW - Drill

KW - Emergency preparedness

KW - Evaluation

KW - HEPE

KW - Health emergency preparedness exercises

KW - Healthcare emergency

KW - Healthcare emergency responder

KW - Major incident

KW - Mass casualty

KW - Operation exercise

KW - Questionnaire

KW - Simulation exercise

KW - Survey

KW - TTX

KW - Tabletop

UR - http://www.scopus.com/inward/record.url?scp=85113718063&partnerID=8YFLogxK

U2 - 10.1504/IJEM.2020.117209

DO - 10.1504/IJEM.2020.117209

M3 - Article

AN - SCOPUS:85113718063

SN - 1471-4825

VL - 16

SP - 317

EP - 346

JO - International Journal of Emergency Management

JF - International Journal of Emergency Management

IS - 4

ER -

Evaluation of emergency preparedness exercises: The design of a questionnaire to measure staff perceptions

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this