In this paper, we have presented a novel tracking method aiming at detecting objects and maintaining their la-bel/identification over the time. The key factors of this method are to use depth information and different strategies to track objects under various occlusion scenarios. The foreground objects are detected and refined by background subtraction and shadow cancellation. The occlusion detection is based on information of foreground blobs in successive frames. The occlusion regions are projected to the projection plane XZ to analysis occlusion situation. According to the occlusion analysis results, different objects’ corresponding strategies are introduced to track objects under various occlusion scenarios including tracking occluded objects in similar depth layer and in different depth layers. The experimental results show that our proposed method can track the moving objects under the most typical and challenging occlusion scenarios.
 D. Koller, K. Danilidis and H.-H. Nagel, “Model-Based Object Tracking in Monocular Image Sequences of Road Traffic Scenes,” International Journal of Computer Vision, Vol. 10, No. 3, 1993, pp. 257-281. doi:10.1007/BF01539538
 T. Hai, H. S. Sawhney and R. Kumar, “Object Tracking with Bayesian Estimation of Dynamic Layer Representations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 1, 2002, pp. 75-89. doi:10.1109/34.982885
 A. D. Jepson, D. J. Fleet and T. F. El-Maraghi, “Robust Online Appearance Models for Visual Tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 10, 2003, pp. 1296-1311. doi:10.1109/TPAMI.2003.1233903
 D. Comaniciu, V. Ramesh and P. Meer, “Kernel-Based Object Tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 5, 2003, pp. 564-577. doi:10.1109/TPAMI.2003.1195991
 D. Beymer, P. McLauchlan, B. Coifman and J. Malik, “A Real-Time Computer Vision System for Measuring Traffic Parameters,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, 17-19 June 1997, pp. 495-501.
 N. K. Paragios and R. Deriche, “A PDE-Based Level-Set Approach for Detection and Tracking of Moving Objects,” The Sixth International Conference on Computer Vision, Bombay, 4-7 January 1998, pp. 1139-1145.
 Z. Tao, R. Nevatia and W. Bo, “Segmentation and Tracking of Multiple Humans in Crowded Environments,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30No. 7, , 2008, pp. 1198-1211. doi:10.1109/TPAMI.2007.70770
 V. Papadourakis and A. Argyros, “Multiple Objects Tracking in the Presence of Long-Term Occlusions,” Computer Vision and Image Understanding, Vol. 114, No. 7, 2010, pp. 835-846. doi:10.1016/j.cviu.2010.02.003
 M. Wu, X. Peng, Q. Zhang and R. Zhao, “Segmenting and Tracking Multiple Objects under Occlusion Using Multi-Label Graph Cut,” Computers & Electrical Engineering, Vol. 36, No. 5, 2010, pp. 927-934. doi:10.1016/j.compeleceng.2009.12.013
 S. J. Krotosky and M. M. Trivedi, “On Color-, Infrared-, and Multimodal-Stereo Approaches to Pedestrian Detection,” IEEE Transactions on Intelligent Transportation Systems, Vol. 8, No. 4, 2007, pp. 619-629. doi:10.1109/TITS.2007.908722
 R. Okada, Y. Shirai and J. Miura, “Object Tracking Based on Optical Flow and Depth,” International Conference on Multisensor Fusion and Integration for Intelligent Systems (IEEE/SICE/RSJ), Washington DC, 8-11 December 1996, pp. 565-571.
 D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” International Journal of Computer Vision, Vol. 47, No. 1-3, 2002, pp. 7-42. doi:10.1023/A:1014573219977
 P. KaewTraKulPong and R. Bowden, “An Improved Adaptive Background Mixture Model for Real-Time Tracking with Shadow Detection,” In: P. Remagnino, G. Jones, N. Paragios and C. Regazzoni, Eds., Video-Based Surveillance Systems, Springer, Berlin, 2002, pp. 135-144.
 A. Prati, I. Mikic, M. M. Trivedi and R. Cucchiara, “Detecting Moving Shadows: Algorithms and Evaluation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 7, 2003, pp. 918-923. doi:10.1109/TPAMI.2003.1206520
 “OpenCV Wiki, cvBlobLib,” 2011. http://opencv.willowgarage.com/wiki/cvBlobsLib
 F. Chang, C.-J. Chen and C.-J. Lu, “A Linear-Time Component-Labeling Algorithm Using Contour Tracing Technique,” Computer Vision and Image Understanding, Vol. 93, No. 2, 2004, pp. 206-220. doi:10.1016/j.cviu.2003.09.002
 A. Bhattacharyya, “On a Measure of Divergence between Two Statistical Populations Defined by Their Probability Distributions,” Bulletin of the Calcutta Mathematical Society, Vol. 35, 1943, pp. 99-109.
 G. R. Bradski, “Real Time Face and Object Tracking as a Component of a Perceptual User Interface,” The Fourth IEEE Workshop on Applications of Computer Vision, Princeton, 19-21 October 1998, pp. 214-219.