Machine translation with transformers

Abstract

The Transformer translation model (Vaswani et al., 2017), which relies on selfattention mechanisms, has achieved state-of-the-art performance in recent neural machine translation (NMT) tasks. Although the Recurrent Neural Network (RNN) is one of the most powerful and useful architectures for transforming one sequence into another one, the Transformer model does not employ any RNN. This work aims to investigate the performance of the Transformer model compared to different kinds of RNN model in a variety of difficulty levels of NMT problems.