Attention-Based Recurrent Neural Networks (RNNs) for Short Text Classification: An Application in Public Health Monitoring

Oduwa Edo-Osagie*, Iain Lake, Obaghe Edeghere, Beatriz De La Iglesia

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

11 Citations (Scopus)

Abstract

In this paper, we propose an attention-based approach to short text classification, which we have created for the practical application of Twitter mining for public health monitoring. Our goal is to automatically filter Tweets which are relevant to the syndrome of asthma/difficulty breathing. We describe a bi-directional Recurrent Neural Network architecture with an attention layer (termed ABRNN) which allows the network to weigh words in a Tweet differently based on their perceived importance. We further distinguish between two variants of the ABRNN based on the Long Short Term Memory and Gated Recurrent Unit architectures respectively, termed the ABLSTM and ABGRU. We apply the ABLSTM and ABGRU, along with popular deep learning text classification models, to a Tweet relevance classification problem and compare their performances. We find that the ABLSTM outperforms the other models, achieving an accuracy of 0.906 and an F1-score of 0.710. The attention vectors computed as a by-product of our models were also found to be meaningful representations of the input Tweets. As such, the described models have the added utility of computing document embeddings which could be used for other tasks besides classification. To further validate the approach, we demonstrate the ABLSTM’s performance in the real world application of public health surveillance and compare the results with real-world syndromic surveillance data provided by Public Health England (PHE). A strong positive correlation was observed between the ABLSTM surveillance signal and the real-world asthma/difficulty breathing syndromic surveillance data. The ABLSTM is a useful tool for the task of public health surveillance.

Original languageEnglish
Title of host publicationAdvances in Computational Intelligence - 15th International Work-Conference on Artificial Neural Networks, IWANN 2019, Proceedings
EditorsGonzalo Joya, Andreu Catala, Ignacio Rojas
PublisherSpringer Verlag
Pages895-911
Number of pages17
ISBN (Print)9783030205201
DOIs
Publication statusPublished - 2019
Event15th International Work-Conference on Artificial Neural Networks, IWANN 2019 - Gran Canaria, Spain
Duration: 12 Jun 201914 Jun 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11506 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference15th International Work-Conference on Artificial Neural Networks, IWANN 2019
Country/TerritorySpain
CityGran Canaria
Period12/06/1914/06/19

Bibliographical note

Publisher Copyright:
© 2019, Springer Nature Switzerland AG.

Keywords

  • Deep learning
  • Natural Language Processing
  • Sequence modelling
  • Syndromic surveillance

Fingerprint

Dive into the research topics of 'Attention-Based Recurrent Neural Networks (RNNs) for Short Text Classification: An Application in Public Health Monitoring'. Together they form a unique fingerprint.

Cite this