The use of Live, Virtual and Constructive (LVC) simulations are increasingly being examined for potential analytical use particularly in test and evaluation. In addition to system-focused tests, LVC simulations provide a mechanism for conducting joint mission testing and system of systems testing when fiscal and resource limitations prevent the accumulation of the necessary density and diversity of assets required for these complex and comprehensive tests. LVC simulations consist of a set of entities that interact with each other within a situated environment (i.e., world) each of which is represented by a mixture of computer-based models, real people and real physical assets. The physical assets often consist of geographically dispersed test assets which are interconnected by persistent networks and augmented by virtual and constructive entities to create the joint test environment under evaluation. LVC experiments are generally not statistically designed, but really should be. Experimental design methods are discussed followed by additional design considerations when planning experiments for LVC. Some useful experimental designs are proposed and a case study is presented to illustrate the benefits of using statistical experimental design methods for LVC experiments. The case study only covers the planning portion of experimental design. The results will be presented in a subsequent paper.
Haase, C. , Hill, R. and Hodson, D. (2014) Planning for LVC Simulation Experiments. Applied Mathematics, 5, 2153-2167. doi: 10.4236/am.2014.514209.
[1] Bjorkman, E.A. and Gray, F. B. (2009) Testing in a Joint Environment 2004-2008: Findings, Conclusions, and Recommendations from the Joint Test and Evaluation Methodology Project. ITEA Journal, 30, 39-44. [2] Department of Defense (2003) Joint Capabilities Integration and Development System. Chairman of the Joint Chiefs of Staff Instruction 3170.01C. [3] Department of Defense (2007) Capability Test Methodology v3.0.
https://www.jte.osd.mil/jtemctm/handbooks/PM\_Online/PM_Handbook\_17Apr09.htm [4] Hodson, D.D. and Hill, R.R. (2014) The Art and Science of Live, Virtual, and Constructive Simulation for Test and Analysis. Journal of Defense Modeling and Simulation, 11, 77-89.
http://dx.doi.org/10.1177/1548512913506620 [5] Hodson, D.D. (2009) Performance Analysis of Live-Virtual-Constructive and Distributed Virtual Simulations: Defining Requirements in Terms of Temporal Consistency. Ph.D. Thesis, Air Force Institute of Technology, Wright-Patterson AFB, Ohio. [6] Thorp, H.W. and Knapp, G.F. (2003) The Joint National Training Capability the Centerpiece of Training Transformation. In: Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, FL. [7] Montgomery, D.C. (2009) Design and Analysis of Experiments. 7th Edition, John Wiley & Sons, New York. [8] Coleman, D.E. and Montgomery, D.C. (1993) A Systematic Approach to Planning for a Designed Industrial Experiment. Technometrics, 35, 1-12.
http://dx.doi.org/10.1080/00401706.1993.10484984 [9] Madni, A.M. (2010) Integrating Humans with Software and Systems: Technical Challenges and a Research Agenda. Systems Engineering, 13, 232-243. [10] Wang, J. and Wu, C. (1992) Nearly Orthogonal Arrays with Mixed Levels and Small Runs. Technometrics, 34, 409-422.
http://dx.doi.org/10.1080/00401706.1992.10484952 [11] Heydat, A.S., Sloane, N. and Stufken, J. (1999) Orthogonal Arrays: Theory and Applications. Springer-Verlag, New York.
http://dx.doi.org/10.1007/978-1-4612-1478-6 [12] Nguyen, N.-K. (1996) A Note on the Construction of Near-Orthogonal Arrays with Mixed Levels and Economic Run Size. Technometrics, 38, 279-283.
http://dx.doi.org/10.1080/00401706.1996.10484508 [13] Xu, H. (2002) An Algorithm for Constructing Orthogonal and Nearly-Orthogonal Arrays with Mixed Levels and Small Runs. Technometrics, 44, 356-368.
http://dx.doi.org/10.1198/004017002188618554 [14] Lu, X., Li, W. and Xie, M. (2006) A Class of Nearly Orthogonal Arrays. Journal of Quality Technology, 38, 148-161. [15] Myers, R.H., Montgomery, D.C. and Anderson-Cook, C.M. (2009) Response Surface Methodology. 3rd Edition, John Wiley & Sons, New York. [16] Jones, B. and Nachtsheim, C.J. (2009) Split-Plot Designs: What, Why, and How. Journal of Quality Technology, 41, 340-361. [17] Horton, M.J.D. (2012) Conflict: Operational Realism versus Analytical Rigor in Defense Modeling and Simulation. Graduate Research Project, Air Force Institute of Technology, Wright-Patterson AFB, Ohio. [18] Bjorkman, E.A. (2010) USAF War Fighting Integration: Powered by Simulation. In WinterSim’10: Proceedings of the 2010 Winter Simulation Multiconference, San Diego, CA, Society for Computer Simulation International. [19] Cohen, A.N. (2009) Examining Split-Plot Designs for Developmental and Operational Testing. Master’s Thesis, Air Force Institute of Technology, Wright-Patterson AFB, Ohio.