Identificador persistente para citar o vincular este elemento: http://hdl.handle.net/10553/106914
Título: Informer, an information organization transformer architecture
Autores/as: Estupiñán Ojeda, Cristian David 
Guerra Artal, Cayetano 
Hernández Tejera, Francisco Mario 
Clasificación UNESCO: 120304 Inteligencia artificial
Palabras clave: Convolution
Deep learning
Informer
Linear transformer
Neural machine translation, et al.
Fecha de publicación: 2021
Editor/a: SciTePress Digital Library 
Publicación seriada: ICAART (Setúbal) 
Conferencia: 13th International Conference on Agents and Artificial Intelligence (ICAART 2021) 
Resumen: The use of architectures based on transformers presents a state of the art revolution in natural language processing (NLP). The employment of these architectures with high computational costs has increased in the last few months, despite the existing use of parallelization techniques. This is due to the high performance that is obtained by increasing the size of the learnable parameters for these kinds of architectures, while maintaining the models' predictability. This relates to the fact that it is difficult to do research with limited computational resources. A restrictive element is the memory usage, which seriously affects the replication of experiments. We are presenting a new architecture called Informer, which seeks to exploit the concept of information organization. For the sake of evaluation, we use a neural machine translation (NMT) dataset, the English-Vietnamese IWSLT15 dataset (Luong and Manning, 2015). In this paper, we also compare this proposal with architectures that reduce the computational cost to O(n · r), such as Linformer (Wang et al., 2020). In addition, we have managed to improve the SOTA of the BLEU score from 33.27 to 35.11.
URI: http://hdl.handle.net/10553/106914
ISBN: 978-989-758-484-8
ISSN: 2184-433X
DOI: 10.5220/0010372703810389
Fuente: ICAART 2021 - Proceedings of the 13th International Conference on Agents and Artificial Intelligence [ISSN 2184-433X] ,v. 2, p. 381-389, (Enero 2021)
Colección:Actas de congresos
Vista completa

Google ScholarTM

Verifica

Altmetric


Comparte



Exporta metadatos



Los elementos en ULPGC accedaCRIS están protegidos por derechos de autor con todos los derechos reservados, a menos que se indique lo contrario.