People Detection and Tracking using Depth Sensors

People Detection and Tracking using Depth Sensors 

 

 

People detection and tracking are essential capabilities in several fields such as: ambient intelligent systems, 
visual servoing applications , augmented reality and  human-computer interaction, video compression or robotics.
 
Detection and tracking in monocular images are topics widely explored in the related literature. However, the use of stereo vision for these purposes is 
an emerging research area. Stereo vision brings several advantages over monocular images. First, all the methods designed for tracking in monocular images can be applied, but with much richer per-pixel information (colour or luminance plus depth). Depth information can be employed to achieve a better tracking of people as well as a better understanding of their gestures. Besides, depth is an important piece of information for the development of robust background estimation techniques. Second, disparity information (from which depth is obtained) is relatively invariable to illumination changes.  Therefore, systems that employ stereo vision are expected to be more robust in real scenarios where sudden illumination changes might occur.
People detection and tracking are essential capabilities in several fields such as: ambient intelligent systems,  visual servoing applications , augmented reality and  human-computer interaction, video compression or robotics.
 
Detection and tracking in monocular images are topics widely explored in the related literature. However, the use of stereo vision for these purposes is an emerging research area. Stereo vision brings several advantages over monocular images. First, all the methods designed for tracking in monocular images can be applied, but with much richer per-pixel information (colour or luminance plus depth). Depth information can be employed to achieve a better tracking of people as well as a better understanding of their gestures. Besides, depth is an important piece of information for the development of robust background estimation techniques. Second, disparity information (from which depth is obtained) is relatively invariable to illumination changes.  Therefore, systems that employ stereo vision are expected to be more robust in real scenarios where sudden illumination changes might occur.
 
 

Use Cases

An example of this technology is shown in the following video. The project aimed at tracking and analyzing the activity of people in ambiente assisted environments. In the video, the activity of the person is automatically analyzed and potentially dangerous situations detected (such as a fall). An stereo camera mounted in the ceiling was used as sensor.  

 
 
 
 
The following video shows a more changelling situations in which a single stereo camera is employed to detect and track multiple people.
 
 
This other video shows how to takcle the problem of people detection and tracking using multiple stereo cameras
 

 

Related Publications

Rafael Muñoz-Salinas,"A Bayesian plan-view map based approach for multiple-person detection and tracking", Pattern Recognition 12/2008; 41(12-41):3665-3676. DOI:10.1016/j.patcog.2008.06.013

Rui Paúl, Eugenio Aguirre, Miguel García-Silvente, Rafael Muñoz-Salinas,"A new fuzzy based algorithm for solving stereo vagueness in detecting and tracking people", International Journal of Approximate Reasoning 06/2012; 53(4):693–708. DOI:10.1016/j.ijar.2011.11.003

 

Rafael Muñoz-Salinas, E. Yeguas-Bolivar, L. Díaz-Más, Rafael Medina-Carnicer "Shape from pairwise silhouettes for plan-view map generation",Image and Vision Computing 02/2012; 30:122-133. DOI:10.1016/j.imavis.2012.02.005

Rafael Muñoz-Salinas · R. Medina-Carnicer · F.J. Madrid-Cuevas · A. Carmona-Poyato ,"Particle filtering with multiple and heterogeneous cameras",Pattern Recognition 07/2010; 43(7-43):2390-2405. DOI:10.1016/j.patcog.2010.01.015