Browsing by Author "Jahedi, Azin"

Now showing 1 - 2 of 2

Open Access
Improved descriptor Learning for correspondence problems
(2018) Jahedi, Azin
Solving correspondence problems is a fundamental task in computer vision. In the past decades, many approaches tried to find the matches between images. One way to solve this task is to use feature-based methods, such as SIFT, SURF and DAISY. The mentioned methods are based on engineered features. Learned features are another type of descriptors that are basically learned and computed via a convolutional neural network (CNN). This type of descriptors has gained a lot of attention in the past years. In this thesis, we design and train many CNN to find an architecture and a set of parameters, so that it computes suitable descriptors for the optical flow estimation task. We implement a fast way of computing the descriptors from images, based on a strategy suggested by Bailer et al. After computing the non-dense set of matches by the Coarse-to-Fine PatchMatch (CPM) algorithm, we use the interpolation technique which is introduced and used in Edge-Preserving Interpolation of Correspondences for Optical Flow (Epicflow), to compute the dense flow field. We use the approach of CPM, which is a popular method to estimate optical flow for large displacements. We embed our trained CNN model into CPM in such a way that the algorithm uses the learned descriptors to find the matches in a coarse-to-fine manner. We use the recent benchmarks of KITTI 2015 and MPI-Sintel to train and evaluate our CNN. To this end, we compare the estimated optical flow to the ground truth optical flow provided in the mentioned benchmarks. By computing the Average End-Point Error (AEE) of the obtained optical flow, we thus have a measure to assess the learned descriptors and we can use it as a feedback to change the network so that it leads to more suitable descriptors. In this way, we were able to design and train a network that computes descriptors which perform better than DAISY and SIFT for the MPI-Sintel dataset.
Open Access
MS-RAFT+ : high resolution multi-scale RAFT
(2023) Jahedi, Azin; Luz, Maximilian; Rivinius, Marc; Mehl, Lukas; Bruhn, Andrés
Hierarchical concepts have proven useful in many classical and learning-based optical flow methods regarding both accuracy and robustness. In this paper we show that such concepts are still useful in the context of recent neural networks that follow RAFT’s paradigm refraining from hierarchical strategies by relying on recurrent updates based on a single-scale all-pairs transform. To this end, we introduce MS-RAFT+: a novel recurrent multi-scale architecture based on RAFT that unifies several successful hierarchical concepts. It employs a coarse-to-fine estimation to enable the use of finer resolutions by useful initializations from coarser scales. Moreover, it relies on RAFT’s correlation pyramid that allows to consider non-local cost information during the matching process. Furthermore, it makes use of advanced multi-scale features that incorporate high-level information from coarser scales. And finally, our method is trained subject to a sample-wise robust multi-scale multi-iteration loss that closely supervises each iteration on each scale, while allowing to discard particularly difficult samples. In combination with an appropriate mixed-dataset training strategy, our method performs favorably. It not only yields highly accurate results on the four major benchmarks (KITTI 2015, MPI Sintel, Middlebury and VIPER), it also allows to achieve these results with a single model and a single parameter setting. Our trained model and code are available at https://github.com/cv-stuttgart/MS_RAFT_plus .