Empirical Study of Audio-Visual Features Fusion for Gait Recognition

Francisco M. Castro, Manuel J. Marín-Jiménez, Nicolás Guil

Overview

The goal of this paper is to evaluate how the fusion of audio and visual features can help in the challenging task of people identification based on their gait (i.e. the way they walk), or gait recognition. Most of previous research on gait recognition has focused on designing visual descriptors, mainly on binary silhouettes, or building sophisticated machine learning frameworks. However, little attention has been paid to audio patterns associated to the action of walking. So, we propose and evaluate here a multimodal system for gait recognition. The proposed approach is evaluated on the challenging "TUM GAID" dataset, which contains audio recordings in addition to image sequences. The experimental results show that using late fusion to combine two kinds of tracklet-based visual features with audio features improves the state-of-the-art results on the standard experiments defined on the dataset.

Who are they?

In this project, we aim at identifying people by using their gait. In contrast to classical approaches, we study in [1] the contribution of the audio signal for gait-based identification by exploring several fusion information techniques.

Our approach

Given a video sequence of people walking, audio and visual information are combined in order to assign an identity from a set of predefined ones.

Downloads

Related Publications

[1] F.M. Castro, M. Marin-Jimenez, N. Guil
Empirical Study of Audio-Visual Features Fusion for Gait Recognition
Computer Analysis of Images and Patterns (CAIP), 2015
Document: PDF

Acknowledgements

This work has been partially funded by project TIC-1692 (Junta de Andalucía), and the Research Projects TIN2012-32952 and BROCA, both financed by FEDER and the Spanish Ministry of Science and Technology.