On determining the Fundamental matrix: analysis of different methods and experimental results

Abstract

The fundamental matrix is a key concept when working with uncalibrated images and multiple viewpoints. It contains all the available geometric information and enables to recover the epipolar geometry from uncalibrated perspective views. This paper addresses the important problem of its robust determination given a number of image point correspondences. We first define precisely this matrix and show clearly how it is related to the epipolar geometry and to the essential matrix introduced earlier by Longuet-Higgins. In particular, we show that this matrix, defined up to a scale factor, must be of rank two. Different parametrizations for this matrix are then proposed to take into account these important constraints and linear and non-linear criteria for its estimation are also considered. We then clearly show that the linear criterion is unable to express the rank and normalization constraints. Using the linear criterion leads definitely to the worst result in the determination of the fundamental matrix. Several examples on real images clearly illustrate and validate this important negative result. To overcome the major weaknesses of the linear criterion, different non-linear criteria are proposed and analyzed in great detail. Extensive experimental work has been performed in order to compare the different methods using a large number of noisy synthetic data and real images. In particular, a statistical method based on variation of camera displacements is used to evaluate the stability and convergence properties of each method.