Home > IEC 61131-3 (english) > IEC 61131-3: Unit-Tests

IEC 61131-3: Unit-Tests

Unit tests are an essential tool for every programmer to ensure functioning of his software. Software bugs cost time and money, so you need an automated solution to find these bugs and preferably before the software is used. Unit tests should be used wherever software is professionally developed. This article is intended to provide a quick introduction and allow an understanding of the benefits of unit tests.

Motivation

Separate test programs are often written to test function blocks. In such a test program, an instance of the desired function block is created and called. The output variables are observed and manually checked for correctness. If these do not match the expected values, the function block is adjusted until it works as intended.

But testing software once is not enough. Amendments and extensions of a program are often responsible that functions or function blocks, that were previously tested and worked without errors, suddenly do not work correctly any longer. It is also possible that the correction of program errors can also affect other parts of the program and can lead to malfunctions in other parts of the code. Previously executed and completed tests must therefore be repeated manually.

One possible way to improve this is to automate the tests. For this purpose, a test program is developed which calls up the functionality of the program to be tested and checks the return values. A test program written once offers a number of advantages:

– The tests are automated and can therefore be repeated at any time with the same framework conditions (timings, …).

– Once written tests are retained for other team members.

Unit Tests

A unit test checks a very small and self-sufficient part (unit) of a software. In IEC 61131-3, this is a single function block or a function. Each test calls the unit to be tested (function block, method or function) with test data (parameters) and checks its reaction to this test data. If the delivered result matches the expected result, the test is considered passed. A test generally consists of a whole series of test cases that not only check one target/actual pair, but several of them.

A developer decides himself which test scenarios he implements. However, it makes sense to test with values that typically occur in practice when they are called. Consideration of limit values (extremely large or small values) or special values (zero pointer, empty string) is also useful. If all these test scenarios deliver correct values as expected, developer can assume that his implementation is correct.

A positive side-effect is that it gives developers less headaches to make complex changes to their code. After all, they can check the system at any time after making such changes. If no errors occur after such a change, it is most likely to have been successful.

However, the risk of poor test implementation must not be ignored. If these are inadequate or even false, but produce a positive result, this deceptive certainty will sooner or later lead to major problems.

The Unit Test Framework TcUnit

Unit test frameworks offer necessary functionalities to create unit tests quickly and effectively. They offer further advantages:

– Everyone in the team can extend the tests quickly and easily.

– Everyone is able to start the tests and check the results of the tests for correctness.

The unit test framework TcUnit was developed as part of a project. In fact, it is a PLC library that provides methods for verifying variables (assert methods). If a check was not successful, a status message is displayed in the output window. The assert methods are contained in the function block FB_Assert.

There is a method for each data type, whereby the structure is always similar. There is always a parameter that contains the actual value and a parameter for the setpoint. If both match, the method returns TRUE, otherwise FALSE. The parameter sMessage specifies the output text to be displayed in the event of an error. This allows you to assign the messages to the individual test cases. The names of the assert methods always begin with AreEqual.

Here, for example, the method checks a variable of type integer for validity.

Pic01

Some methods contain additional parameters.

Pic02

All standard data types (BOOL, BYTE, INT, WORD, STRING, TIME, …) have corresponding assert methods. Some special data types, such as AreEqualMEM for checking a memory area or AreEqualGIUD, are also supported.

The first Example

Unit tests are used to check individual function blocks independently of other components. These function blocks can be located in a PLC library or in a PLC project.

For the first example, the FB to be tested should be located in a PLC project. This is the function block FB_Foo.

Pic03

Defines the time that the output bOut remains set if no further positive edges are applied to bSwitch.

bSwitch A positive edge sets the output bOut to TRUE. This remains active for the time tDuration. If the output is already set, the time tDuration is restarted.
bOff The output bOut is immediately reset by a positive edge.
tDuration Defines the time that the output bOut remains set if no further positive edges are applied to bSwitch.

Unit tests are intended to prove that the FB_Foo function block behaves as expected. The code for testing is implemented directly in the TwinCAT project.

Project Setup

To separate the test code from the application, the folder TcUnit_Tests is created. The POU P_Unit_Tests is stored in this folder from which the respective test cases are called.

A corresponding test FB is created for each FB. This has the same name plus the postfix _Tests. In our example, the name is FB_Foo_Tests.

Pic04

In P_Unit_Tests, an instance of FB_Foo_Tests is created and called.

PROGRAM P_Unit_Tests
VAR
   fbFoo_Tests : FB_Foo_Tests;
END_VAR

fbFoo_Tests();

FB_Foo_Tests contains the entire test code for checking FB_Foo. In FB_Foo_Tests, an instance of FB_Foo is created for each test case. These are called with different parameters and the return values are validated using the assert methods.

The execution of the individual test cases takes place in a state machine, which is also managed by the PLC library TcUnit. This means, for example, that the test is automatically terminated as soon as an error has been detected.

Definition of Test Cases

The individual test cases must first be defined. Each test case occupies a certain area in the state machine.

For the naming of the individual test cases, several naming rules have proven useful, which help to make the test setup more transparent.

The name of the test cases that are to check an inbox of FB_Foo is composed of [input name]_[test condition]_[expected behaviour]. Test cases that test methods of FB_Foo are named similarly, i. e. [method name]_[test condition]_[expected behaviour].

The following test cases are defined according to this scheme:

Switch_RisingEdgeAndDuration1s_OutIsTrueFor1s

Tests whether the output bOut is set to 1 s by a positive edge at bSwitch, if tDuration is set to t#1s.

Switch_RisingEdgeAndDuration1s_OutIsFalseAfter1100ms

Tests whether a positive edge at bSwitch causes the output bOut to become FALSE again after 1100 ms, if tDuration has been set to t#1s.

Switch_RetriggerSwitch_OutKeepsTrue

Tests whether the time tDuration is restarted by a new positive edge at bSwitch.

Off_RisingEdgeAndOutIsTrue_OutIsFalse

Tests whether the output bOut is set to FALSE by a positive edge at bOff.

Test Case Implementation

Each test case occupies at least one step in the state machine. In this example, the increment between the individual test cases is 16#0100. The first test case starts at 16#0100, the second at 16#0200, etc. In step 16#0000, initializations will be performed, whereas step 16#FFFF must be available, since this is started by the state machine as soon as an assert method has detected an error. If the test runs without errors, a message is displayed in 16#FF00 and the unit test for FB_Foo is finished.

The pragma region is very helpful to simplify navigation in the source code.

FUNCTION_BLOCK FB_Foo_Tests
VAR_INPUT
END_VAR
VAR_OUTPUT
   bError : BOOL;
   bDone : BOOL;
END_VAR
VAR
   Assert : FB_ASSERT('FB_Foo');
   fbFoo_0100 : FB_Foo;
   fbFoo_0200 : FB_Foo;
   fbFoo_0300 : FB_Foo;
   fbFoo_0400 : FB_Foo;
END_VAR

CASE Assert.State OF
{region 'start'}
16#0000:
   bError := FALSE;
   bDone := FALSE;
   Assert.State := 16#0100;
{endregion}

{region 'Switch_RisingEdgeAndDuration1s_OutIsTrueFor1s'}
16#0100:
   fbFoo_0100(...
   ...
   Assert.State := 16#0200;
{endregion}

{region 'Switch_RisingEdgeAndDuration1s_OutIsFalseAfter1100ms'}
16#0200:
   fbFoo_0200(...
   ...
   Assert.State := 16#0300;
{endregion}

{region 'Switch_RetriggerSwitch_OutKeepsTrue'}
16#0300:
   fbFoo_0300(...
   ...
   Assert.State := 16#0400;
{endregion}

{region 'Off_RisingEdgeAndOutIsTrue_OutIsFalse'}
16#0400:
   fbFoo_0400(...
   ...
   Assert.State := 16#FF00;
{endregion}

{region 'done'}
16#FF00:
   Assert.PrintPassed('Done');
   Assert.State := 16#FF10;

16#FF10:
   bDone := TRUE;
{endregion}

{region 'error'}
16#FFFF:
   bError := TRUE;
{endregion}

ELSE
   Assert.StateMachineError();
END_CASE

There is a separate instance of FB_Foo for each test case. This ensures that each test case works with a newly initialized instance of FB_Foo. This avoids mutual influence of the test cases.

16#0100:
   fbFoo_0100(bSwitch := TRUE, tDuration := T#1S);
   Assert.AreEqualBOOL(TRUE, fbFoo_0100.bOut, 'Switch_RisingEdgeAndDuration1s_OutIsTrueFor1s');
   tonDelay(IN := TRUE, PT := T#900MS);
   IF (tonDelay.Q) THEN
      tonDelay(IN := FALSE);
      Assert.State := 16#0200;
   END_IF

The block to be tested is called for 900 ms. During this time, bOut must be TRUE, because bSwitch has been set to TRUE and tDuration is 1 s. The assert method AreEqualBOOL checks the output bOut. If it does not have the expected status, an error message is output. After 900 ms, you switch to the next test case by setting the property State of FB_Assert.

A test case can also consist of several steps:

16#0300:
   fbFoo_0300(bSwitch := TRUE, tDuration := T#500MS);
   Assert.AreEqualBOOL(TRUE, fbFoo_0300.bOut, 'Switch_RetriggerSwitch_OutKeepsTrue');
   tonDelay(IN := TRUE, PT := T#400MS);
   IF (tonDelay.Q) THEN
      tonDelay(IN := FALSE);
      fbFoo_0300(bSwitch := FALSE);
      Assert.State := 16#0310;
   END_IF

16#0310:
   fbFoo_0300(bSwitch := TRUE, tDuration := T#500MS);
   Assert.AreEqualBOOL(TRUE, fbFoo_0300.bOut, 'Switch_RetriggerSwitch_OutKeepsTrue');
   tonDelay(IN := TRUE, PT := T#400MS);
   IF (tonDelay.Q) THEN
      tonDelay(IN := FALSE);
      Assert.State := 16#0400;
   END_IF

Triggering of bSwitch is performed in line 7 and line 12. Lines 3 and 13 check whether the output remains set.

Output of Messages

After executing all test cases for FB_Foo, a message is output (step 16#FF00).

Pic05

If an assert method detects an error, it is also displayed as a message.

Pic06

If the AbortAfterFail property of FB_Assert is set to TRUE, the step 16#FFFF is called in case of an error and the test is terminated.

The assert method prevents the same message from being issued more than once in a single step. Multiple output of the same message, e. g. in a loop, is thus suppressed. By setting the MultipleLog property to TRUE, this filter is deactivated and every message is output.

Due to the structure shown above, the unit tests are clearly separated from the actual application. FB_Foo remains completely unchanged.

This TwinCAT solution is stored in the source code management (such as TFS or Git) together with the TwinCAT solution for the PLC library. Thus, the tests are available to all team members of a project. With the unit test framework, tests can also be extended by anyone, and existing tests can be started and easily evaluated.

Even though the term unit test framework is somewhat sophisticated for the PLC library TcUnit, it is evident that automated tests with only a few tools are also possible with the IEC 61131-3. Commercial unit test frameworks go far beyond what a PLC library can do. Thus, they contain dialogs to start the tests and display the results. Also, the areas in the source code that have been run through by the individual test cases are often marked.

Library TcUnit (TwinCAT 3.1.4022) on GitHub

Sample (TwinCAT 3.1.4022) on GitHub

Tips

The biggest obstacle in unit tests is often one’s weaker self. Once this has been overcome, the unit tests write themselves almost automatically. The second hurdle is the question which parts of the software have to be tested. It makes little sense to want to test everything. Instead, you should concentrate on essential areas of the software and test thoroughly the function blocks that form the basis of the application.

Basically, a unit test is considered to be fairly qualitative if possibly many branches are run through during execution. When writing unit tests, the test cases should be selected in such a way that as many branches of the function block as possible are run through.

If errors still occur in practice, it can be advantageous to write tests for this error case. This ensures that an error that has occurred once does not occur again.

The mere fact that two or more function blocks work correctly and this is proven by unit tests does not mean that an application also applies these function blocks correctly. Unit tests do not replace integration and acceptance tests in any way. Such test methods validate the overall system and evaluate it as a whole. Even though the unit test are applied, it is necessary to continue to test the entire work. However, a significant part of potential errors is eliminated in advance by unit tests, which saves time and money in the end.

Further Information

During the preparation for this post, Jakob Sagatowski published the first part of an article series on Test driven development in TwinCAT in his blog AllTwinCAT. For all those who want to go deeper into the topic, I can highly recommend the blog. It is encouraging that other PLC programmers also confront themselves with the testing of their software. The book The Art of Unit Testing by Roy Osherove is also a good introduction to the topic. Although the book was not written for IEC 61131-3, it contains some interesting approaches that can be implemented in the PLC without any problems.

Finally, I would like to thank my colleagues Birger Evenburg and Nils Johannsen. The basis for this post was a PLC library, kindly provided by them.

Advertisements
  1. Vince Bednarz
    January 31, 2018 at 7:34 pm

    Hi there, interesting article but I can’t see any of the “PIC” files. PIC01 thru PIC06 are just placeholders. Clicking on them doesn’t do anything useful. Could be a problem on my end but I’m not really sure.

    I’m really interested in this since I have to rewrite an application originally in C/C++ in CoDeSys.

    • February 6, 2018 at 2:27 pm

      Hi Vince, I have tested the site on different OS and with different browsers. In all cases the pictures showed correctly.
      Can you try to view the pages on a different browser? If that doesn’t work, I could send you the pictures via email.
      Stefan

  2. David Freiberger
    February 12, 2018 at 8:22 pm

    Hi Stefan,

    I am working on a new project and trying to incorporate unit testing. I am trying to figure out how to “mock” an FB which is called directly inside of another FB, in order to preserve the concept of a unit test.

    For instance, in a more conventional program I might have
    FUNCTION BLOCK FB_Gantry
    VAR_IN_OUT
    aAxis : ARRAY[*] OF FB_Axis;
    END_VAR

    aAxis[0].UpdatePosition();
    someVar := FB_Axis.rPosition;

    Where FB_Axis has an UpdatePosition method and rPosition output variable.

    However, for unit testing purposes we should isolate FB_Axis when testing FB_Gantry. In the software world this is done by mocking the referenced class so we can manipulate the data it produces to a known state, e.g. we need to mock FB_Axis. What do you think is the best way to accomplish this?

    I can think of a few options:
    1. Treat FB_Axis as an “Abstract Base Class”, which is a FB with variables but no logic, and then extend it for actual logic, such as FB_LinearAxis, and FB_MockAxis.

    2. Create an interface I_Axis, where the header now becomes
    FUNCTION BLOCK FB_Gantry
    VAR_IN_OUT
    aAxis : ARRAY[*] OF I_Axis;
    END_VAR

    and implement all data that needs to be exchanged as Methods and Properties.

    3. Leave definition as-is, and build a “mock” configuration into FB_Axis, which you enable externally when testing.

    4. Don’t do anything, and just build tests which either assume the internal FB works as expected or tests for everything.

    Where I am getting stuck is on what is the best approach. An interface seems like the “correct” approach. But using an interface can grow program complexity, since almost everything that was an input or output of the original FB will need to be accessed externally, and so will become a property, and then end up being duplicated by internal variables and possibly HMI variables (if you are using OPC-UA, I don’t believe it is possible to read the value of a property).

    Thoughts?

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: