|Title:||Informer, an information organization transformer architecture||Authors:||Estupiñán Ojeda, Cristian David
Guerra Artal, Cayetano
Hernández Tejera, Francisco Mario
|UNESCO Clasification:||120304 Inteligencia artificial||Keywords:||Convolution
Neural machine translation, et al
|Issue Date:||2021||Publisher:||SciTePress Digital Library||Journal:||ICAART (Setúbal)||Conference:||13th International Conference on Agents and Artificial Intelligence (ICAART 2021)||Abstract:||The use of architectures based on transformers presents a state of the art revolution in natural language processing (NLP). The employment of these architectures with high computational costs has increased in the last few months, despite the existing use of parallelization techniques. This is due to the high performance that is obtained by increasing the size of the learnable parameters for these kinds of architectures, while maintaining the models' predictability. This relates to the fact that it is difficult to do research with limited computational resources. A restrictive element is the memory usage, which seriously affects the replication of experiments. We are presenting a new architecture called Informer, which seeks to exploit the concept of information organization. For the sake of evaluation, we use a neural machine translation (NMT) dataset, the English-Vietnamese IWSLT15 dataset (Luong and Manning, 2015). In this paper, we also compare this proposal with architectures that reduce the computational cost to O(n · r), such as Linformer (Wang et al., 2020). In addition, we have managed to improve the SOTA of the BLEU score from 33.27 to 35.11.||URI:||http://hdl.handle.net/10553/106914||ISBN:||978-989-758-484-8||ISSN:||2184-433X||DOI:||10.5220/0010372703810389||Source:||ICAART 2021 - Proceedings of the 13th International Conference on Agents and Artificial Intelligence [ISSN 2184-433X] ,v. 2, p. 381-389, (Enero 2021)|
|Appears in Collections:||Actas de congresos|
Items in accedaCRIS are protected by copyright, with all rights reserved, unless otherwise indicated.