Keypoint descriptor fusion with Dempster-Shafer Theory

V.M. Mondéjar-Guerra, R. Muñoz-Salinas, M.J. Marín-Jiménez, A. Carmona-Poyato, R. Medina-Carnicer
Keypoint descriptor fusion with Dempster–Shafer theory
International Journal of Approximate Reasoning

Abstract

Keypoint matching is the  task  of accurately finding  the location of a scene point in two images. Many keypoint  descriptors have been proposed in the literature aiming at providing robustness against scale, translation and rotation transformations, each  having advantages and disadvantages. This paper proposes a novel approach  to fuse the information from multiple keypoint descriptors using the \acl{DST} of evidence \cite{ShaferBook:1976}, which has proven particularly efficient in combining  sources of information providing incomplete, imprecise, biased, and conflictive knowledge. The matching results of each descriptor are transformed into an evidence distribution on which  a confidence factor is computed making use of its entropy. Then, the evidence distributions are fused using the  \ac{DST}, considering its  confidence. As result of the fusion, a new evidence distribution that improves the result of the best descriptor is obtained. Our method has been tested with the SIFT, SURF, ORB, BRISK and FREAK descriptors using all possible combinations of them. Results on the  Oxford keypoint dataset \cite{schmid2003performance} shows that the proposed approach obtains an improvement  of up to $10\%$ compared to the best one (FREAK).
Keypoint matching is the  task  of accurately finding  the location of a scene point in two images. Many keypoint  descriptors have been proposed in the literature aiming at providing robustness against scale, translation and rotation transformations, each  having advantages and disadvantages. This paper proposes a novel approach  to fuse the information from multiple keypoint descriptors using the Depmster-Shafer theory of evidence which has proven particularly efficient in combining  sources of information providing incomplete, imprecise, biased, and conflictive knowledge. The matching results of each descriptor are transformed into an evidence distribution on which  a confidence factor is computed making use of its entropy. Then, the evidence distributions are fused using the  Dempster-Shafer Theory (DST), considering its  confidence. As result of the fusion, a new evidence distribution that improves the result of the best descriptor is obtained. Our method has been tested with the SIFT, SURF, ORB, BRISK and FREAK descriptors using all possible combinations of them. Results on the  Oxford keypoint dataset shows that the proposed approach obtains an improvement  of up to 10% compared to the best one (FREAK).
 

Supplementary material