Feasibility of using attention mechanism in abstractive summarization

AlMazrouei, RZ; Nelci, J; Salloum, S; Shaalan, K

Abstract

The Prevalence of information and its magnitude mandates a short description of the core of a document, an article, or legal documents. Abstractive summarization helps to concur with this problem utilizing the evolutions in machine learning and deep neural network. Attention-mechanism has extensively applied in the challenging issue of abstraction a text, in shorter length yet informative. We noticed in [13] after removing the attention layer from their proposed model, the performance only experience soft drawback, even can be ignored. Thus, motivates us to survey the latest models using attention-mechanism and its achievements, and the second objective is to run an experiment to compare standard stacked 3- Long Short-Term Memory (LSTM) layers incorporated with attention layer only (without any other hand-crafted algorithm) to explore how efficient this technique can generate better summarization, then a stand-alone model. The standard proposed model incorporated with attention-mechanism suffered from drawback performance and scored less than a stand-alone model by at least 6 point scores on ROUGE-1&2.

• Presentation Conference Type: Conference Paper (published)
• Conference Name: International Conference on Emerging Technologies and Intelligent Systems (ICETIS)
• End Date: Jun 26, 2021
• Online Publication Date: Aug 8, 2021
• Publication Date: Jan 1, 2022
• Deposit Date: Nov 30, 2021
• Journal: Lecture Notes in Networks and Systems
• Print ISSN: 2367-3370
• Electronic ISSN: 2367-3389
• Volume: 299
• Pages: 13-20
• Series Title: Lecture Notes in Networks and Systems
• Series Number: 299
• Book Title: Proceedings of International Conference on Emerging Technologies and Intelligent Systems
• ISBN: 9783030826154-(paperback);-9783030826161-(ebook)
• DOI: https://doi.org/10.1007/978-3-030-82616-1_2
• Publisher URL: https://doi.org/10.1007/978-3-030-82616-1_2
• Related Public URLs: https://doi.org/10.1007/978-3-030-82616-1
• Additional Information: Event Type : Conference