Unsupervised approach to estimate the optical flow using RAFT's architecture

Abstract

Optic flow estimation is a significant area of interest within the field of computer vision. Recently, end-to-end neural network-based approaches have received significant interest in the estimation of optic flow. One of the best methods in this field is RAFT, which outperforms previously published methods like PWC-Net. Although these approaches perform best on competitive benchmarks, they have their drawbacks. Most of the state-of-the-art techniques are supervised learning-based methods and require a large amount of annotated data. The creation of such data sets is an expensive tedious task. Due to this fact, unsupervised approaches that do not need annotated data for being trained to become more popular. One of the most successful unsupervised methods is ARFlow, which gains competitive results to the successful supervised approach of PWC-Net. The success of ARFlow comes from its unique pipeline that performs two forward passes of original and augmented image pairs and forces the consistency of the transformed estimate of the original image-pair with the computed flow of the augmented imagepair. ARFlow uses the architecture PWC-Net as baseline architecture. However, there have been no studies that incorporate RAFT’s architecture as a baseline architecture in ARFlow. This thesis aims to integrate the 2-view unsupervised approach of ARFlow with RAFT’s architecture. The final developed approach is trained in an unsupervised way with the benefit of using augmentation as a regularization while it has the advantage of a more robust architecture introduced by RAFT. The final architecture is evaluated using recent optic flow benchmarks. The final results of the unsupervised RAFT model has good cross-data generalization and achieve 3.7 % better results on Sintel clean test dataset compared to ARFlow (PWC-Net as a base network).