ABSTRACT Public benchmark datasets have been widely used to evaluate multi-target tracking algorithms. Ideally, the benchmark datasets should include the video scenes of all scenarios that need to be tested. However, a limited amount of the currently available benchmark datasets does not comprehensively cover all necessary test scenarios. This limits the evaluation of multitarget tracking algorithms with various test scenarios. This paper introduced a computer simulation model that generates benchmark datasets for evaluating multi-target tracking algorithms with the complexity of multitarget tracking scenarios directly controlled by simulation inputs such as target birth and death rates, target movement, the rates of target merges and splits, target appearances, and image noise types and levels. The simulation model generated a simulated video and also provides the ground-truth target tracking for the simulated video, so the evaluation of multitarget tracking algorithms can be easily performed without any manual video annotation process. We demonstrated the use of the proposed simulation model for evaluating tracking-by-detection algorithms and filtering-based tracking algorithms.
Cite this paper
Vo, G. and Park, C. (2019) Computer Model for Evaluating Multi-Target Tracking Algorithms. Open Journal of Modelling and Simulation, 7, 1-18. doi: 10.4236/ojmsi.2019.71001.
 Popoli, R. and Blackman, S.S. (1999) Design and Analysis of Modern Tracking Systems. Artech House, Norwood, MA.
 Yilmaz, A., Javed, O. and Shah, M. (2006) Object Tracking: A Survey. ACM Computing Survey, 38, 1-45. https://doi.org/10.1145/1177352.1177355
 Skolnik, M.I. (1990) Radar Handbook. McGraw-Hill, New York City, NY.
 Stone, L.D., Streit, R.L., Corwin, T.L. and Bell, K.L. (2013) Bayesian Multiple Target Tracking. Artech House, Norwood, MA.
 Bar-Shalom, Y. (1987) Tracking and Data Association. Academic Press Professional, Inc.
 Blackman, S.S. (1986) Multiple-Target Tracking with Radar Applications. Artech House, Inc., Dedham, MA.
 Mahler, R.P.S. (2007) Statistical Multisource-Multitarget Information Fusion. Artech House, Norwood, MA.
 Lane, D.M., Chantler, M.J. and Dai, D.Y. (1998) Robust Tracking of Multiple Objects in Sector-Scan Sonar Image Sequences Using Optical Flow Motion Estimation. IEEE Journal Oceanic Engineering, 23, 31-46.
 Kocak, D.M., da Vitoria Lobo, N. and Widder, E. (1999) A Computer Vision Techniques for Quantifying, Tracking, and Identifying Bioluminescent Plankton. IEEE Journal Oceanic Engineering, 24, 81-95. https://doi.org/10.1109/48.740157
 Durrant-Whyte, H. and Bailey, T. (2006) Simultaneous Localization and Mapping: Part I. IEEE Robotics and Automation Magazine, 13, 99-110.
 Bailey, T. and Durrant-Whyte, H. (2006) Simultaneous Localization and Mapping (SLAM): Part II. IEEE Robotics and Automation Magazine, 13, 108-117.
 Spagnolini, U. and Rampa, V. (1999) Multitarget Detection/Tracking for Monostatic Ground Penetrating Radar: Application to Pavement Profiling. IEEE Transactions on Geoscience and Remote Sensing, 37, 383-394.
 Sethi, I.K. and Jain, R. (1987) Finding Trajectories of Feature Points in a Monocular Image Sequence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9, 56-73.
 Veenman, C.J., Reinders, M.J.T. and Backer, E. (2001) Resolving Motion Correspondence for Densely Moving Points. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 54-72. https://doi.org/10.1109/34.899946
 Pirsiavash, H., Ramanan, D. and Fowlkes, C.C. (2011) Globally-Optimal Greedy Algorithms for Tracking a Variable Number of Objects. IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, 20-25 June 2011, 1201-1208.
 Jiang, H., Fels, S. and Little, J.J. (2007) A Linear Programming Approach for Multiple Object Tracking. IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, 17-22 June 2007, 1-8.
 Zhang, L., Li, Y. and Nevatia, R. (2008) Global Data Association for Multi-Object Tracking Using Network Flows. IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, 23-28 June 2008, 1-8.
 Jaqaman, K., Loerke, D., Mettlen, M., Kuwata, H., Grinstein, S., Schmid, S.L. and Danuser, G.R. (2008) Single-Particle Tracking in Live-Cell Time-Lapse Sequences. Nature Methods, 5, 695-702. https://doi.org/10.1038/nmeth.1237
 Serge, A., Bertaux, N., Rigneault, H. and Marguet, D. (2008) Dynamic Multiple-Target Tracing to Probe Spatiotemporal Cartography of Cell Membranes. Nature Methods, 5, 687-694. https://doi.org/10.1038/nmeth.1233
 Choi, W. and Savarese, S. (2012) A Unified Framework for Multi-Target Tracking and Collective Activity Recognition. European Conference on Computer Vision, Florence, 7-13 October 2012, 215-230.
 Henriques, J.F., Caseiro, R. and Batista, J. (2011) Globally Optimal Solution to Multi-Object Tracking with Merged Measurements. IEEE International Conference on Computer Vision, Barcelona, 6-13 November 2011, 2470-2477.
 Park, C., Woehl, T.J., Evans, J.E. and Browning, N.D. (2014) Minimum Cost Multi-Way Data Association for Optimizing Multitarget Tracking of Interacting Objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37, 611-624.
 Broida, T.J. and Chellappa, R. (1986) Estimation of Object Motion Parameters from Noisy Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, 90-99.
 Beymer, D. and Konolige, K. (1999) Real-Time Tracking of Multiple People Using Continuous Detection.
 Rosales, R. and Sclaroff, S. (1999) 3D Trajectory Recovery for Tracking Multiple Objects and Trajectory Guided Recognition of Actions. IEEE Conference on Computer Vision and Pattern Recognition, Colorado, 23 June 1999, 117-123.
 Tanizaki, H. (1996) Nonlinear Filters: Estimation and Applications. Springer, New York City, 400.
 Khan, Z., Balch, T. and Dellaert, F. (2004) An MCMC-Based Particle Filter for Tracking Multiple Interacting Targets. European Conference on Computer Vision, Prague, 11-14 May 2004, 279-290.
 Hue, C., Le Cadre, J.-P. and Perez, P. (2002) Tracking Multiple Objects with Particle Filtering. IEEE Transactions on Aerospace Electronics System, 38, 791-812.
 Genovesio, A. and Olivo-Marin, J.-C. (2004) Split and Merge Data Association Filter for Dense Multi-Target Tracking. International Conference on Pattern Recognition, 4, 677-680.
 Khan, Z., Balch, T. and Dellaert, F. (2005) Multi-Target Tracking with Split and Merged Measurements. IEEE Conference on Computer Vision and Pattern Recognition, San Diego, 20-26 June 2005, Vol. 1, 605-610.
 Khan, Z., Balch, T. and Dellaert, F. (2006) MCMC Data Association and Sparse Factorization Updating for Real Time Multi-Target Tracking with Merged and Multiple Measurements. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 1960-1972. https://doi.org/10.1109/TPAMI.2006.247
 Storlie, C.B., Lee, T.C.M., Hannig, J. and Nychka, D. (2009) Tracking of Multiple Merging and Splitting Targets: A Statistical Perspective. Statistica Sinica, 19, 1-52.
 Yang, B. and Nevatia, R. (2012) Multi-Target Tracking by Online Learning of Non-Linear Motion Patterns and Robust Appearance Models. IEEE Conference on Computer Vision and Pattern Recognition, Providence, 16-21 June 2012, 1918-1925.
 Alahi, A., Jacques, L., Boursier, Y. and Vandergheynst, P. (2011) Sparsity Driven People Localization with a Heterogeneous Network of Cameras. Journal of Mathematical Imaging and Vision, 41, 39-58. https://doi.org/10.1007/s10851-010-0258-7
 BIWI Walking Pedestrians Dataset Computer Vision Laboratory-ETH, 2009.
 Pellegrini, S., Ess, A., Schindler, K. and Van Gool, L. (2009) You’ll Never Walk Alone: Modeling Social Behavior for Multi-Target Tracking. International Conference on Computer Vision, Kyoto, 27 September-4 October 2009, 261-268.
 Andriluka, M., Roth, S. and Schiele, B. (2010) Monocular 3D Pose Estimation and Tracking by Detection. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, 13-18 June 2010, 623-630.
 Yang, B. and Nevatia, R. (2012) An Online Learned CRF Model for Multi-Target Tracking. IEEE Conference on Computer Vision and Pattern Recognition, Providence, 16-21 June 2012, 2034-2041.
 Milan, A., Roth, S. and Schindler, K. (2014) Continuous Energy Minimization for Multitarget Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36, 58-72. https://doi.org/10.1109/TPAMI.2013.103
 Milan, A., Schindler, K. and Roth, S. (2013) Challenges of Ground Truth Evaluation of Multi-Target Tracking. IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, 23-28 June 2013, 735-742.
 Pedestrian Mobile Scene Analysis Computer Vision Laboratory-ETH, 2009.
 Ess, A., Leibe, B. and Van Gool, L. (2007) Depth and Appearance for Mobile Scene Analysis. International Conference on Computer Vision, Rio de Janeiro, 14-20 October 2007, 1-8.
 Benfold, B. and Reid, I. (2011) Stable Multi-Target Tracking in Real-Time Surveillance Video. IEEE Conference on Computer Vision and Pattern Recognition, Colorado, 20-25 June 2011, 3457-3464.
 Nannuru, S., Coates, M. and Mahler, R. (2013) Computationally-Tractable Approximate Probability Hypothesis Density and Cardinalized Probability Hypothesis Density Filters for Superpositional Sensors. IEEE Journal Selective Topics in Signal Processing, 7, 410-420.
 Hoseinnezhad, R., Vo, B.-N. and Vo, B.-T. (2013) Visual Tracking in Background Subtracted Image Sequences via Multi-Bernoulli Filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 61, 392-397.
 Pinsky, M. and Karlin, S. (2010) An Introduction to Stochastic Modeling. Academic Press, Waltham.
 Hida, T. (1980) Brownian Motion. Springer, New York.
 Karatzas, I. (1991) Brownian Motion and Stochastic Calculus. Springer, New York.
 Revuz, D. and Yor, M. (1999) Continuous Martingales and Brownian Motion. Springer, New York.
 Bibbona, E., Panfilo, G. and Tavella, P. (2008) The Ornstein-Uhlenbeck Process as a Model of a Low Pass Filtered White Noise. Metrologia, 45, 117.
 Matsushita, Y., Nishino, K., Ikeuchi, K. and Sakauchi, M. (2004) Illumination Normalization with Time-Dependent Intrinsic Images for Video Surveillance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 1336-1347.
 Sheskin, D.J. (2003) Handbook of Parametric and Nonparametric Statistical Procedures. CRC Press, Boca Raton.
 Keni, B. and Rainer, S. (2008) Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. EURASIP Journal in Image and Video Processing, 2008, Article ID: 246309.
 Brown, R.G., Hwang, P.Y.C., et al. (1992) Introduction to Random Signals and Applied Kalman Filtering. Wiley, Hoboken.
 Kim, P. and Huh, L. (2011) Kalman Filter for Beginners: With MATLAB Examples. CreateSpace, Seattle.