Recently, according to the development of science and technology and changes in the concept of war performance, the value of strategic use of unmanned systems has been increasing because the idea of life emphasis has increased. As the battlefield expands to the ground, sea, air, and even cyber space, the importance of operations to secure the superiority of battlefield is increasing. Especially, there are a lot of cases in which unmanned systems are used in combat situations where there is a high risk or a human is difficult to carry out. One of the marine unmanned systems, USV is extended to various missions such as marine surveillance and reconnaissance, target search, and mine search    . In order for the USV to be operated autonomously, an artificial intelligence system methodology that recognizes, judges, and plans the situation like a human being is needed. This means a complex process that can make decisions by self-determination of the state of operation and surrounding conditions of various sensors and armed devices   . The related research includes the concept of Hierarchical Encapsulation & Abstraction Principle (HEAP), a Multi-Agent modeling technique of Zeigler and Chi for performing complex processing, and Norman’s 7-step model for human behavior processing   . However, it is very difficult to analyze the effects of various tactical environments and mission conditions on USVs at actual sea. Therefore, it is necessary to provide a consistent environment in which the user can set desired mission scenarios and parameters and can simulate and analyze them. There are various virtual environment systems for USV mission analysis such as 3D simulation using OpenGL and Reactive Layer Virtual World Considerate Layer (RVC) system model based on artificial intelligence    .
In addition, Modeling & Simulation (M&S) techniques, which are generally used in the defense field, are used for system design, development, analysis and acquisition, and especially for mission analysis   . Therefore, in this study, we defined an Experimental Framework (EF) for analyzing the task of USV using the M&S technique and designed the interface. In addition, the Integrated Mission Planning System designed in Part I was installed in this experimental framework. The experimental framework was classified into a module for generating and delivering input events to the verification system and a module for collecting and analyzing the result data. In Section 2, we introduce the structure of the experimental framework for analyzing the USV mission and the interfaces implemented on it. In Section 3, we define the mission effectiveness and suggest the direction of future research.
2. Design of Experimental Frame for USV Mission
In this section, we define 1) M&S-based frameworks for mission analysis and introduce 2) user interface for setting and executing simulation environment and parameters. The experimental frame structure was modeled using C# and Unity3D programs, and the Integrated Mission Planning System was modeled through previous studies and was applied on this framework.
2.1. Experimental Frame Structure
The experimental framework based on M&S is composed of a Generator Model that delivers mission scenario and mission parameter information to the target system and a Transducer Model that collects and analyzes the result data after simulation. In the Generator Model, you set up mission objectives such as reconnaissance surveillance or mine search in the Mission Scenario Setup Module. In the Mission Parameter and Event Modules, you will create scenarios for mission planning, equipment configuration, and contingency scenarios such as speed, fuel, detection range, and unmanned ship encounters. The generated scenario is transferred to the Integrated MPS Simulation Model for simulation and performs its mission according to the set conditions.
The Integrated MPS Simulation Model consists of a Logical Model, Multi-Agent Process that serves as the brain of the USV, a Physical Model, Platform Model and a simulation environment, Space Model. Since the logical and physical models mentioned above share a single space model, information on the USV location and surrounding conditions can be received in real time from the mission planning process model and can be transferred for a plan/re-plan command. The Multi-Agent Process is most important part of the integrated mission planning process. It performs Situation Awareness, Decision Making, and Task Re-planning to make decisions itself like a human being according to the surrounded situations. The behavior of Sensor, Engagement, and Movement in the Platform model is controlled through this model and it is covered in detail in the previous research, Remote Operation S/W for USV (I). Figure 1 shows the
Figure 1. Experimental framework for USV mission analysis based on MPS.
structure of the experimental framework for mission analysis based on the Integrated Mission Planning Process designed in this study.
2.2. Design of Interface
The interface designed in this study was developed using Unity3D. The interface for performing the EF Model function is shown in Figure 2(a). This interface can perform various tasks by designing various parameters such as mission type, detailed scenarios, and performance of the mounted equipment. Figure 2(b) is the simulation part performing the mission based on the information set up above. In addition, information on detection information and events, mission status, navigation information, equipment operation information, and mission planning/re-planning that occur in real time is visualized to the user. Figure 3 and Figure 4 show the detailed interface configuration information.
As shown in Figure 3, the mission parameter and scenario setup display are divided into three parts. 1) The Mission Planning Process Connection and Mission Setup part establishes the integrated mission planning process and the USV mission type. 2) In the Mission Parameter & Event Setup part, you set up a contingency plan such as encountering obstacles, enemy encounters, and fuel
Figure 2. (a) Mission parameter and scenario setup display, (b) USV simulation display.
Figure 3. Mission parameter and scenario setup display configuration.
Figure 4. USV simulation display configuration.
shortages. Finally, 3) Mission Scenario Display part shows a brief scenario outline based on the set information.
Figure 4 shows the USV simulation display where the simulation is performed based on the information set by user. At the top display, there are 1) Monitoring display, 2) USV current path, 3) Radar identification information, 4) Mission execution information and operation mode setting window. At the bottom, there are 5) MPS display, 6) USV status and navigation information, 7) Sensor equipment information, and 8) Real-time mission planning/re-planning information that an operator can plan a route or mission.
3. Define Mission of Effectiveness for USV
For USV mission analysis, the mission effectiveness according to various operations must be defined first. Therefore, the effectiveness of the USV mission was defined as 1) mission time, 2) mission accomplishment, and 3) mission regulation through naval expert advices. Here, the mission time is a measure of the mission area divided by the mission operation time, indicating how quickly the mission is accomplished. Mission accomplishment is an aspect of accuracy. It is an element that measures how accurately a given task is performed. Finally, mission regulation is measured in terms of doctrinal mission performance.
Since the MOE depends on the mission of USV mission, MOE was established in this study focusing on the main mission of USV, objective search and mine search. Figure 5 shows the MOE modification for mine search and objective search defined in this study.
In this study, MOE is defined as “coverageRate” for search mission. In the case of a search mission, “coverageRate” is a measure of how efficiently a given search area is searched. Here, Area is the area of the mission, Width is the length of the search area, Height is the length of the search area, “trackLength” is the total path length, and “searchSpeed” is the average reconnaissance speed of USV, “trackNum” is number of tracks. Therefore, the ideal number of tracks to search for the mission area and the speed for the effect on the mission processing time is defined can be expressed as Equation (1) below.
Figure 5. MOE modification for mine search and objective search.
Mission accomplishment means the number of detections, N, when Nall is all numbers of unidentified objects existing in the mission area. The mission regulation represents the number of missions performed for missions according to tactical doctrine. Figure 6 shows the effectiveness of the mine search and target search missions.
Key indicators include sonar performance, search speed, search width, and track width. In addition, various MOE can be defined, and specific studies are needed to apply them to this software developed in this study. The MOE analysis module will be modeled through further studies in the future and it is expected that the USV optimal operating condition can be derived by analyzing the MOE results according to the key indicators as shown in Figure 6.
In this study, we developed remote operation S/W for USV, which can analyze missions and modeled Integrated MPS by dividing Remote Operation S/W for USV into Part I and Part II. This software enables USV mission based on Integrated Mission Planning Process in a virtual marine environment, and allows users to set various mission and accident conditions. In addition, the simulation execution display can be confirmed through the interface, and since it is a Multi-Agent module type, it can be easily extended and reusable. However, since the effect analysis module is not installed, it is necessary to study the mission effectiveness module. It is important to select the evaluation items based on the opinions of the military experts since the USV effect varies depending on the purpose of mission operations. Therefore, in the future, we will carry out a study to further expand and systematize the MOE defined in this study, and develop an interface that the main result can be visualized after the end of the simulation. It is expected that it will be easy to check parameters affecting the mission effectiveness through the visualized data and it will enhance the utility of this software. Therefore, in the future, we plan to enhance the effectiveness of this study
Figure 6. MOE result example according to searchspeed and trackwidth in minesearch.
by applying the mission effectiveness analysis module to the S/W developed in this study to derive the USV optimal operating conditions.
This work was supported by the Agency for Defense Development (ADD) under the Contract No. UD160008DD.
 Weon, I.S., Ryu, J.K. and Lee, S.G. (2017) Recognition of Object Tracking with Radar for USV. Proceedings of the Fall Conference of the Korean Society of Precision Engineering, Seoul, 14-17 November 2017, 299-300.
 Lee, H.W., Roh, M., Ham, S.H., Zhao, L., Kim, N.W., Ha, S., Woo, J.H., Jung, W.H. and Yu, C.W. (2017) A Study on Integrated Simulation Method for Mine Detection Mission of USV. Korean Journal of Computational Design and Engineering, 9, 306-316.
 Hwang, H.-G., Kim, H.-W., Kim, B.-S., Woo, Y.-T., Shin, I.-S., Shin, J.-H., Lee, Y.-J. and Choi. B.-W. (2017) A Development of Integrated Control System for Platform Equipments of Unmanned Surface Vehicle (USV). Journal of the Korea Institute of Information and Communication Engineering, 21, 1611-1618.
 Park, J., Han, J., Kim, J., Son, N.-S. and Kim, S.Y. (2017) Automatic Detection of Nearby Ships Using Monocular Vision for Autonomous Navigation of USVs. Journal of Institute of Control, Robotics and Systems, 23, 416-423.
 Dong Z.P., Wan, L., Li, Y.M., Liu, T. and Zhang, G.C. (2015) Trajectory Tracking Control of Underactuated USV Based on Modified Backstepping Approach. International Journal of Naval Architecture and Ocean Engineering, 7, 817-832.
 Duan, L., Luo, B., Li, Q.Y. and Yu, G.H. (2016) Research on Intelligence, Surveillance and Reconnaissance Mission Planning Model and Method for Naval Fleet. IEEE Conference on Control and Decision (CCDC), Yinchuan, 28 May 2016, 2419-2424.
 Thomas, G., Schneider, M. and Otto, P. (2008) Multi System Mission Control for Teams of Unmanned Marine Vehicles-Software Structure for Online Replanning of Mission Plans. 7th International Conference on Computer Applications and Information Technology in the Maritime Industries (COMPIT), Liège, April 2008, 185-199.
 Kim, J.H., Lyu, Y.J., Yu, C.W. and Lee, D.H. (2016) Interface Form Based Experimentation Framework for Mission Planning Analysis of Multi-Mission USV. Journal of the Korean Operations Research and Management Society, 4, 6306-6311.