Browsing by Author "Breul, Gerhard Christian"

Filter results by typing the first few letters
Now showing 1 - 1 of 1
  • Thumbnail Image
    ItemOpen Access
    How well do language models understand grammar? : a case study on Japanese
    (2022) Breul, Gerhard Christian
    Modern attention-based language models such as BERT and GPT have been shown to outperform previous state-of-the-art models on many NLP tasks. This performance implies a level of understanding of grammatical structures. This work attempts to contribute to the growing body of research assessing this understanding, by exploring language models' ability to predict the transitivity of verbs in Japanese, which seems to be somewhat underrepresented in research compared to English. I consider a variety of language models with different architectures, tokenization approaches, training data, and training regimes. In doing so, I find that bidirectional models outperform unidirectional ones, that different types of perplexity calculation can be advantageous in certain situations and should be considered on a case-by-case basis, and that the tested models only gain a somewhat limited understanding of the grammar required for the Transitivity Prediction task.