Browsing by Author "Tilli, Pascal"

Filter results by typing the first few letters
Now showing 1 - 1 of 1
  • Thumbnail Image
    ItemOpen Access
    Generation-based continual learning approach for visual question answering
    (2020) Tilli, Pascal
    Humans have the ability to continually acquire knowledge throughout their lifespan. In contrast, neural networks suffer from catastrophic forgetting when trained on new tasks. Continual learning studies the methods to achieve similar memory effects in artificial neural networks and enable them to learn tasks sequentially. In this thesis, we investigate generation-based continual learning methods. Generation-based models have been used to replay previously learned data distributions and retain knowledge of solving previous tasks. Generative replay has been shown to work on uni-modal datasets with relatively low complexity. Our experiments focus on Visual Question Answering (VQA), which is known to be a more complex, multi-modal domain. We provide approaches and results for three datasets of the domain VQA and one uni-modal toy dataset. As a proof of concept, we start by training the handwritten digits of the MNIST dataset continually. For the VQA domain, we study the VQAv2 dataset, CLEVR, and Shapeworld. We found that generative models do not perform well in VQA. Our models could not overcome catastrophic forgetting except for the Shapeworld dataset. Within the Shapeworld setting, our approach with generative replay did enable continual learning.