1. General Introduction
Electric power flows through several transmission and/or distribution substations from the generation source to the end consumer. An electrical substation is a node in a power system network where voltage is transformed from high to low or vice versa using transformers. It comprises of primary or field equipment, such as bus bars, switches, switch yard instrument transformers electrical cable lines etc. The control, protection and constant monitoring of these primary devices are conducted using an integrated mechanism known as the Substation Automation (SA) System    . The SA System consists of micro-processor based secondary device known as an IED (server) which monitors, measures and collects data from the primary equipment via a process bus interface. The collected data is then transmitted to a higher central or distributed control system (client) which guarantees an informed and intelligent administration of all SA system devices  .
The SA system serves as an important method for the maintenance and control of various substation equipments. Historically, in order to transmit equipment data between SAS devices, the implemented IEDs employ different communication protocol which used to be propriety-based  . This hindered the smooth inter-IED communication as each IED uses a different protocol for communication  . Modern substations require multi-vendor interoperability of IEDs using a single easy-to-configure communication protocol to enable simplicity of design. Thus, IEC 61850 “communication networks and systems in substation” has been proposed and internationally accepted as a standard to enable manufacturer independent communication in a digital substation system    . The IEC 61850 System architecture is illustrated in Figure 1. It consists of hierarchical three networked levels, namely the station-level, the bay-level and the process level. The station-level consists of devices like the Human Machine Interface (HMI), and the gateway to the Network Control Center (NCC). The bay-level network includes protection and control bay units while the process
Figure 1. Basic architecture of IEC 61850 substation.
level network interconnects the switch control, sensors for voltage and current measurements and circuit breaker or disconnector control. Recently, the IEC 61850 based digital substation has expanded, and the demand for IEC 61850 based IEDs has increased  . The availability of multiple manufacturers IEDs in a single network within a digital substation increases the need for IEC 61850 communications specification diagnostics to ensure interoperability between products.
IEC 61850 Part 10 presents the test items for testing the conformance of communication specifications and provides the following details. Basic Exchange, Data Sets, Data Set Definition, Substitution, Setting Group Selection, Setting Group Definition, Logging, GOOSE publish, GOOSE subscribe, GOOSE management, control general, direct control, Select Before Operate (SBO) control, enhanced SBO control, etc.  . The International Users Group (UCAIUG) develops and manages the detailed procedures for conducting internationally authorized certification tests based on IEC 61850-10  . For reliable substation operations, the IEDs must pass accredited testing to be installed to digital substations. A test system is developed and widely used at test institutions and sites of power utilities so that the test items can be tested using program-based tools rather than manually performed by the tester during the communication specification conformance test. However, these test systems are conventionally scripted-based systems that can only be used if the tester has a basic knowledge of the computer-based language for setting up a test environment   . This paper presents model-based testing system which, due to the fact that it is not scripted, is easy use by test engineers to use. Our proposed model uses the drag-and-drop method for the design of the test procedure and would thereby make it easy to use. The paper is divided into the following sections; Section 2 introduces the conformance testing for IEC 61850. In Section 3, we discuss the practical implementation of the proposed model-based conformance tool. We also discuss the step-by-step process of using the tool including the all configurations for both the DUT and the software. The IEC 61850 reporting service under test together with the implementation of the model-based test process is also discussed into details. Results and analysis are presented in this section too. Section 4 presents the conclusion.
Review of Related Work
Research in   discussed the importance of IEC 61850 service conformance testing. A more detailed discussion on the purpose and value of service conformance testing was presented by  and  . The authors in  studied the standardized service conformance testing procedure as stipulated by the IEC 61850-10 in comparison with other functional testing techniques. The testing and certification system for specific substation equipment including client/server devices, IEDs, gateways servers etc. are also described. A similar approach is adopted by  where the purpose and certification of the IEC 61850 service conformance testing is discussed. A complete tutorial on the general procedure for service conformance testing has been explained by authors in  . The authors provided a detailed account of the procedure to implement service conformance testing for an IED. This procedure includes the verification of the syntax, data models and how to ensure the conformity of extended logical nodes and data. Authors of   focused on the client conformance testing and presented an evaluation of the method for testing the data communication services. By focusing on the reporting services of IEC 61850 including buffered and unbuffered, the authors in  developed the client testing system which can request and analyze Manufacturing Message Specification (MMS) reports Packet Data Unit (PDU). This research emphasized the importance of client conformance testing using IEC 61850 reporting service as an example. Finally, a close-loop testing system was developed by researchers in  . The researchers designed an automated system to enable the complete IEC 61850 service conformance testing therefore eliminating the possibility of errors caused by human intervention. The limitations of existing tools in literature can be summed up in the fact that some are designed for the implementation of client conformance testing only while other tools are also based on the computer programmable script language. The computer programmable script is quite difficult for most substation engineers or system integrators to use due to the fact that it requires an understanding of programming language like C#. These limitations are addressed by our paper by using object-based modelling approach to the implementation of the service conformance test cases.
2. Introduction to Conformance Testing
The Utility Communication Association International Users Group (UCAIUG) created the testing committee to develop a set of standardized test cases to allow the conformance inspection of multi-vendor devices based on the IEC 61850 standard. This ensures that all device manufacturers comply with the application requirements making the “IEC 61850” a global standard. The serve conformance testing consists of techniques used to check the conformity of the IED to the IEC 61850 standard. As can be observed in Figure 2, the server conformance test consists of two parts which are the static conformance and the dynamic conformance.
The static conformance consists of document version management and the Substation Configuration Description Language (SCL) validation while the dynamic conformance consists of the data and communication model testing. Document version management includes checking the PICS, PIXIT, MICS and TICS. These are explained in Table 1.
These documents, although not standardized by IEC 61850 are required to assist the test engineer when selecting test parameters. The SCL file also has to be checked for semantics and syntax errors. Finally, the dynamic test case is where the actual testing is implemented. Each test case from the conformance test suite is selected, implemented and analyzed for conformance. A basic equipment setup is required to enable the proposer analysis of the server test results. The architecture as can be observed in Figure 3 consists of the DUT, the device running the conformance tool and the test protocol analyzer all connected to an Ethernet switch.
Components of the conformance testing setup are explained in Table 2.
All related test cases are categorized into tables as can be seen in Table 3. Each test group consists of multiple positive and negative test cases. The positive test case verifies device conditions which results in a positive response while a negative test is the opposite.
Table 1. Various documentation for conformance testing.
Table 2. Uses of each Conformance testing device.
Table 3. Uses of each Conformance testing device.
Figure 2. Conformance assessment process  .
Figure 3. Schematic and practical test set for conformance testing. (a) Schematic test; (b) Practical test.
2.1. Techniques for Implementing the Conformance Testing
Conventional IED conformance tool are script-based. This implies that the test engineer has to explicitly write the sequence of conformance test instructions to be interpreted by another program. For most tools, the preferred scripting language is Visual Basic due to its support for Rapid Application Development (RAD). This is quite difficult for utility test engineers who have very limited expertise of computer programming. That is the why we propose the model-based tool for conformance testing. Thus instead of writing down several lines of code in order to implement a test case, the test engineer can select the objects which encapsulates the standard IEC 61850 communication services and foundational programming concepts. The model-based tool is designed such that after a test case is created, the script is automatically generated. The test engineer creates the test case by “dragging and dropping” the test models (objects) which simplifies the testing process for the ordinary substation engineer. In order to demonstrate the implementation of the object model based IEC 61850 conformance test tool, we select sAss1 as an example. Table 4 shows the details of the test procedure as recommended by UCAIUG.
2.2. Script-Based and Model-Based Testing
The script-based conformance tool would require the test engineer to translate the test description in Table 5 into an algorithm before converting it into a script-language as shown in Figure 4. To write this script the test engineer needs to understand syntax, semantics, statements and variables, which is usually difficult for them. This also means that it cannot easily be edited. Thus we propose the use of the model-based tool for conformance testing. Figure 5 shows the implementation of test sAss1 using the proposed model-based tool. After selecting the desired service objects, the test engineer would then link them in accordance with the specification of the IEC 61850-10 test under consideration. This is a much easier to use version as compared with the script-based model. The general list consists of the commands like start, end, log, repeat, select etc. required to create the test case. The service list contains IEC 61850 communication service for each test suite. After selecting the services and creating the test case, each service object can be edited to take on specifications required for the test case.
Table 4. Conformance test procedure for sAss1.
Table 5. IEC 61850 ACSI-MMS Mapping.
Figure 4. Schematic of script-based conformance testing.
Figure 5. Schematic of object-based conformance testing.
2.3. IEC 61850 Application Layer Diagnostics
To achieve successful real-time monitoring and operation of the substation system, communication is essential. As we progress towards the digital age, the fact that communication bandwidth is no longer a limiting factor plays a major role in providing thousands of data points to be processed by a single IED server device.
The basis for service conformance testing is the client-server communication model which is based on the Two-Party Application Association (TPAA). It uses both the Abstract Communication Service Interface (ACSI) and the MMS protocols to achieve TPAA. Figure 6 shows the TPAA client-server communication profile applied in service conformance testing. The IEC 61850 established the ACSI in order to create a vendor-neutral method of transmitting and receiving data stored in the IED. The abstract data objects and services defined by IEC 61850 ACSI provides a standardized means by which power system devices model and present their data. This therefore enables multiple vendors to design devices which operate in a similar manner from a network perspective achieving interoperability. It is used for the definition of the various abstract communication objects and services of utility field devices. “Abstract definition” implies that the standard provides the description of both input/output parameters but does not provide information on how to achieve real-world implementation. The standard focuses on abstraction in order to make it convenient to map the communication services to any existing communication stack as seen in Figure 7. The Open Systems Interconnection 7-layer model is currently being used by many vendors and is supported by the IEC 61850 standard. The ACSI services are mapped to the MMS-based OSI model using the Specific Communication Service Mapping (SCSM) which is well defined in IEC 61850 part 8. The MMS protocol operates as the application layer and uses the TCP-IP which implies dedicated IP address for dedicates communication with each server. The MMS is selected as the “real” protocol due to the fact that it enables direct mapping of ACSI objects to an array of complex named object and services.
The mapping of the ACSI services to the MMS enables the transformation of the abstract model (object) into a unique and unambiguous reference for device implementation and easy comprehension by substation engineer. Table 5 below provides a partial list of the mapping of ACSI to MMS according to IEC 61850-8-1.
Performing an IEC 61850 service conformance test requires the diagnostics of the communication packets or PDU’s to ensure correct implementation in accordance to the standard. To successfully conduct this diagnostics, most engineers focus only on the application layer (IEC 61850 service layer) since the lower layers of the communication stack is independent of the upper layers i.e. the communication service layer is independent of the TCP/IP layer. The separation of the application layer from the lower layer communication stack enables the incorporation of different types of network technologies into IEC 61850 design. Thus diagnostics are performed by focusing on the ACSI and the MMS. Typically, many utility engineer appreciate the simplicity of the MMS protocol as compared to the “abstract” nature of the ACSI, although the standard makes provisions for both service protocols.
Figure 8 shows an example of the complete OSI packets for the release communication service which shows the details of all parameters necessary for the transmission of the PDU across the TCP/IP layer.
Figure 6. TPAA client-server communication profile.
Figure 7. OSI 7 Layer Communication Protocol for client/server communication.
Figure 8. A description of MMS PDU for ACSI Release service.
3. Practical Test Implementation
Figure 9 represents the Graphical User Interface (GUI) of the proposed conformance test tool. It consists of the list of conformance test cases, the modelling view where the test cases are modelled and the device view which enables a full self-description of the IED under test. There is also the log view which provides a summary of the test results as implemented. A pass or fail can be seen at the log view. The last major part is the system view which shows the client and server connection and communication. Figure 10 shows the flowchart for implementing the proposed model-based conformance testing tool. After starting the program there is the need to create a new session or use already available ones. A session contains a set of already pre-configured test cases based on the recommendations of UCAIUG. The device setting shown in Figure 11 involves the configuration of both the server information.
And also the local computer IP addresses to enable device-to-device communication. The time server is also configured in case there is the need to perform time synchronization test. It also helps with accurate time stamping of transmitted signal between client and server. The next step is to extract the IED self-description including all information on the logical devices, logical nodes, data objects and data attributes. The global variables consist of values for each conformance/communication service group as can be seen in Figure 12. By running the initializations, all default values and IED specific data reference required for the test are generated and stored. An example can be observed in Figure 13 where initialization for the association test case generates correct values for the MMS Access Point (AP), Presentation Selector value (PSEL), Session selector (SSEL) and Transport selector (TSEL). The test case under consideration is then selected from the list as can be seen in Figure 13 and then modelled using object models provided. For this purposes, as can be seen in Figure 14 the test engineer can use the model from the service list and the general list.
The service list contains conformance specific objects like associate, release, setDataValues etc. while the general list contains general objects like the start, end, if, wait, invoke etc. After all the configuration and settings are completed, the next step is to run the selected case in order to test the DUT. The results are then analyzed using the log results and/or the communication packets. The pass or fail is automatically detected by also configuring the logic decision maker of the test tool as illustrated in Figure 15 and then the program comes to an end.
Figure 9. Full GUI description of the proposed object-based conformance test software.
Figure 10. Flowchart on the how to use the proposed conformance test software.
Figure 11. Device Setting and configuration for testing.
Figure 12. Global variables and values used for setting various conformance test groups.
Figure 13. Running initializations in the proposed Conformance test tool.
Figure 14. Modelling the test procedure of the selected test case.
Figure 15. Configuring the results to enable test tool independently check for pass or fail cases.
3.1. Understanding the Test Case (Reporting Service)
The IED 61850 IED provides different services for the client (HMI) including the reporting service. Details of the reporting service are provided in IEC 61850 Part 7-2 which specifies the Abstract Communication Service Interface (ACSI). The events in the SA system are known as data objects and are grouped into datasets as can be seen in Figure 16 for reporting. The reporting model enables the transfer of events (data values) from a logical Node (LN) to a client either instantly by way of the Unbuffered reporting or after some time using the buffered reporting. The Report Control Block (RCB) enables the attribute values to be set or read by the controlling the operation of both the report handler and the event monitor. The main condition for generating a report is in case there is an update or a change in the original data attribute in the dataset. There are three (3) types of these changes known as Trigger options (TrigOps). Table 6 summarizes the TrigOps.
The standard defines two (2) classes of the report control block namely Buffered Report Control Block (BRCB) and Unbuffered Report Control Block (URCB). The BRCB class permits the immediate release of the dataset after being issued or for the data changes to be buffered for a specific time period (using bufTm) property. By using the sequence-of-events functionality, it is possible to buffer data in case of disconnection and later submit the data when connection is established. The URCB allows transmission of reports based on the time specified in the BufTm. In case connection is lost, no buffering occurs and the data is lost. The generated reports are sent to the recipient application via TPAA or MPAA mechanism.
Figure 16. Reporting service in IEC 61850.
Table 6. List of the Report Service Trigger options in IEC 61850.
3.2. Reporting Service Test Case Description
As explained previously the report service enables the transmission of data from the IED (server) to the client. In order to present the model based conformance test approach of our proposed tool, we implement a test case involving the report service. The report test case six (6) which involves testing the configuration revision of unbuffered RCB is selected from the list of test cases. The configuration revision is a counter which represents the number of times a referenced dataset has been changed. Figure 17 shows the details of the test procedure according to UCAIUG. As can be observed the main communication service used in the test procedure is the GetURCBValues which enables the client to read attributes from the IED.
While implementing this test case, the main references to ensure successful conformance would be the IEC 61850-8-1 subclause 17.2 and Tissue number 453 subclause 220.127.116.11.
Figure 17. UCAIUG procedure for configuration revision (ConfRev) test.
3.3. Model-Based Implementation of Conformance Test
The following illustrates how to implement the Rp6 test case using the proposed model-based approach. In Figures 18(a)-18(c) each object is described for easy comprehension of the test procedure. Step 2 creates a TCP association between the client testing software or device and the server (DUT). To obtain the initial values of the DUT, the client uses the GetURCBValues to inquire data from the server. The invoke object in step 4 of Figure 18(b) is used to create a temporary dataset for testing purposes. By using the SetURCBvalues, the client testing device causes default values in the dataset to be changed. The ConfRev value is then compared to previous data and it is expected that the value increases.
The process is then repeated from step 8 to step 10 for different datasets associated with the same URCB. The dataset that was created temporarily is then deleted using the invoke in step 11. The test client device releases the DUT at step 12 and then the test ends at step 13 as illustrated in Figure 18(c).
In order to generate this flowchart, each model is selected from the list of communication and service objects as discussed earlier. Also a unique feature of the proposed tool is that after creating the model based flowchart as shown above, the corresponding script file is generated as shown in Figure 19.
3.4. Results and Analysis
To analyze the results, the log reports and the packet analyzer are typically used. As can be seen in Figure 20, the log result shows how each step in the algorithm shown in Figures 18(a)-18(c) is implemented. Example at step  , the client requests the URCB values from the server (DUT) using the GetURCBValues. A positive response is classified as positive if the response shows a successful implementation of the client request. Thus the GetURCBValues results in a positive response which includes the Report ID, Report enabled, reference dataset, optional fields, buffer time, sequence number, trigger options, integrity period and general interrogation as seen in Figure 20 The initial value of the ConfRev is 10,001. The invoke is used to create a new dataset in the server with IP (192.168.1.210). The name of the dataset is known as Temp_Dataset. Values are written to newly created dataset by using the SetURCBValues. It can be observed in Figure 21 that when GetURCBValue is requested, the ConfRev value increases to 1002.
The similar procedure is repeated with the same URCB and a new dataset. It can be observed again that the ConfRev values increases from 1002 to 1003. Thus, as can be observed in Figure 22, the test case passes due to the fact that the ConfRev value increases with every new dataset configuration.
Figure 18. (a) Model-based implementation of configuration revision (ConfRev) test. (b) Main model-based implementation of configuration revision (ConfRev) test. (c) End of the model-based implementation of configuration revision (ConfRev) test.
Figure 19. Script file for the configuration revision (ConfRev) test.
Figure 20. Log result 1: Association and GetURCBValues.
Figure 21. Log result 1: Create temporary dataset, set the URCB values and request the URCB values from DUT.
Figure 22. Log result 3: Compare the ConfRev values and repeat and test.
3.5. Advantages of Object-Based Implementation
There are several advantages of the object model based conformance testing tool as compared to the script-based conformance tool. The most obvious advantage is the ease in generating new test cases or modifying existing cases without the need to write multiple lines of script. Another dimension of this benefit is the fact that by using our proposed object model based conformance tool, the script language is automatically generated. The proposed tool can support all the required IEC 61850 conformance test cases defined by UCAIUG and can also support special cases as defined by the test engineer. The tool has been integrated with a detailed packet analysis software which can help check the network packet traffic including ACSI, MMS and Goose packets. With the object model based tool, it is relatively easier to change the device parameters while testing without having to look through multiple lines of script. For example, for sAss1 conformance testing, the associate service parameters including the AP title, AE qualifier, Transport selector, presentation selector and session selector can easily be selected and edited.
This paper presents the structure, algorithms, and test cases of a model-based diagnostic system to verify the compliance of the IEDs with IEC 61850 communications specifications. Traditional script-based testing methods were inconvenient because the tester had an understanding of the computer language and edited the program directly for setting up the test environment. This test method suggested that the tester may not know the language of the computer, but may test it. Because test objects based on GUI model are developed and the tester edits only the visual model, the test configuration is easier to configure and perform. This is important as it helps test engineers to focus on the test results instead of the test implementation process which can sometimes be very complicated especially in cases where the test engineer has limited understanding of the IEC 61850 standard.
 Kezunovic, M. (2009) Substation Automation Research Frontiers. Proceedings of the IEEE/PES Power Systems Conference and Exposition, Seattle, WA, 15-18 March 2009, 1-2.
 Kirkman, R. (2007) Development in Substation Automation Systems. Proceedings of the 14th International Conference on Intelligent System Applications to Power Systems, Kaohsiung, 5-8 November 2007, 1-6.
 Leonardi, A., Mathioudakis, K., Wiesmaier, A. and Zeiger, F. (2014) Towards the Smart Grid: Substation Automation Architecture and Technologies. Advances in Electrical Engineering, 2014, Article ID: 896296.
 Ozansoy, C.R., Zayegh, A. and Kalam, A. (2002) Communications for Substation Automation and Integration, School of Communications and Informatics. Victoria University of Technology, Melbourne.
 Pruthvi, P., Bhuvaneswari, H.B. and Sudheendran, L. (2013) Analysis of Utility Communication Protocol IEC 61850 for Substation Automation Systems. Proceedings of the National Conference on Challenges in Research & Technology in the Coming Decades (CRT 2013), Ujire, 27-28 September 2013, 1-8.
 Matsuda, S., Watabe, Y., Asrizal, I.I., Katayama, S., Okuno, K. and Kasuga, K. (2011) Issues Overcome in the Design and Application of IEC 61850-Compliant Substation Automation Systems. Proceedings of the International Conference on Advanced Power System Automation and Protection, Beijing, 16-20 October 2011, 198-202. https://doi.org/10.1109/APAP.2011.6180408
 Roostaee, S., Hooshmand, R. and Ataei, M. (2011) Substation Automation System using IEC 61850. Proceedings of the 5th International Power Engineering and Optimization Conference, Shah Alam, 6-7 June 2011, 393-397.
 Gao, C., Zhou, H., Deng, Q.J. and Zhang, Z.Y. (2010) The Research and Implementation of IEC 61850 Communication Protocol for Smart Grid. Proceedings of the 2nd International Conference on Information Science and Engineering, China, 4-6 December 2010, 2463-2466.
 Janssen, M.C., Crossley, P.A. and Yang, L. (2011) Bringing IEC 61850 and Smart Grid Together. Proceedings of the 2nd IEEE PES International Conference and Exhibition on Innovative Smart Grid Technologies, Manchester, 5-7 December 2011, 1-5.
 Apostolov, A. (2017) Impact of IEC 61850 Edition 2 on the Object Modelling of Distribution Protection IEDs. CIRED-Open Access Proceedings Journal, 2017, 1184-1187.
 Richard Schimmel (2011) Conformance Test Procedures for Server Devices with IEC 61850-8-1 Interface. Revision 2.3, on request of the UCA International Users Group.
 Choi, C., Park, J., Han, J., Shin, Y., Kim, Y. and Shim, E. (2011) The Development of a Client Conformance Tool based on IEC 61850. Proceedings International Conference on Electrical Machines and Systems, Beijing, 20-23 August 2011, 1-5.
 Tan, J.C., Zhang, C. and Bo, Z.Q. (2008) The Importance of IEC 61850 Interoperability Testing. Proceedings of 43rd International Universities Power Engineering Conference, Padova, 1-4 September 2008, 1-5.
 Udren, E.A., Willem, S. and Dolezilek, D. (2007) IEC 61850: Role of Conformance Testing in Successful Integration. Proceedings of the 19th International Conference on Electricity Distribution, Vienna, 21-24 May 2007, 1-4.
 Rim, S., Lee, Y. and Lee, W. (2009) Test and Evaluation for Power IT Apparatus. Proceedings of the Transmission & Distribution Conference & Exposition: Asia and Pacific, Seoul, 26-30 October 2009, 1-5.
 Castaño, A., Zapata, G. and García R. (2013) Monitoring and Checking Standard IEC 61850, Respect to the Manufacturer. Proceedings of Simposio Internacional sobre Calidad de la Energia Electrica, Medellín, 27-29 November 2014.
 Lee, N.H., Jang, B.T. and Song, I.J. (2009) The Testing Method of Data Transmission Services of IEC 61850 Based Client. Proceedings of the Transmission & Distribution Conference & Exposition: Asia and Pacific, Seoul, 26-30 October 2009, 1-4.
 Lee, N.H., Jang, B.T., An, Y.H. and Shim, E.B. (2012) A Study on the Development of Model Based SA Client Conformance Testing System. Journal of International Council on Electrical Engineering, 2, 1-5.