Model based testing of state machines
Model based testing of state machines
You can see that the first test route starts from the initial state (line 0) and ends at line 9 where no untaken transitions are left to go (take a look in the state diagram below and follow the red arrows). The output moves to the right by every new transition on the route. Sometimes not all outgoing transitions of a state can be tested. In this case a branch in the test route is necessary. Take WaitChannelB as an example. As you can see not all transitions starting from WaitChannelB were tested in the red route. Two further branches are necessary. They are shown in line 21 and 23 (follow the yellow line) as shown in Figure 1. Branches are indicated by the same indentation level. Other branches on the suggested test routes are indicated with the red, green and blue lines. A probably better way to visualize the necessary test routes is to use a tree. But this is much more difficult on a text console so branches are shown with the same level of indentation.
It’s still your task to create a test-bed triggering the state machine in a way that all the transitions of the different routes are taken. For routes starting not in the initial state (lines 10, 12, 17, 19, 21, 23) it is necessary to send events to the machine to enter this starting state first. The algorithm built into the codegenerator is not always optimal and you might find possibilities to cover all transitions with fewer routes.
The equivalent example used here has three inputs (a,b,enable). The test-patterns to trigger the state machine in the way the coverage algorithm has suggested it looks the following:
// a, b, enable
IO_DATA testpatternCoverage1to8[] = {{0,0,1},{1,0,1},{0,0,1},{1,1,1},{0,0,1},{0,1,1},{0,1,1},
{0,1,1},{0,1,1},{0,0,1},{1,1,0}};
IO_DATA testpatternCoverage10[] = {{0,0,1},{0,1,1},{0,0,1}};
IO_DATA testpatternCoverage12_15[] = {{0,0,1},{0,1,1},{1,1,1},{1,0,1},{1,0,1},{1,0,1},{0,0,1}};
IO_DATA testpatternCoverage17[] = {{0,0,1},{0,1,1},{1,1,1},{1,0,1},{0,0,1}};
IO_DATA testpatternCoverage19[] = {{0,0,1},{0,1,1},{1,1,1},{0,1,1}};
IO_DATA testpatternCoverage21[] = {{0,0,1},{1,0,1},{1,1,1}};
IO_DATA testpatternCoverage23[] = {{0,0,1},{1,0,1},{1,0,1},{1,0,1},{1,0,1}};
A possible test driver is shown here:
// select test data
pTestdata = testpatternCoverage1to8;
size=sizeof(testpatternCoverage1to8)/sizeof(IO_DATA);
do {
// dump inputs here
…
equivalent(&instData); // execute generated state machine
// dump state and state machine outputs here
…
testpatternIdx++; // next test pattern
if (testpatternIdx == size)// end?
shouldStop=1;
}while(shouldStop==0);
As mentioned in [1] testing state machines might “require a fair amount of patience and coffee, because even a mid-size state machine can have 100 different transitions. However, the number of transitions is an excellent measure of the system's complexity. The complexity is driven by the user's requirements: the state machine makes it blindingly obvious how much you have to test. With a less-organized approach, the amount of testing required might be equally large-you just won't know it.”
Instead of simply dumping the data on the screen I suggest a more handy solution now. Every embedded engineer is used to read timing diagrams as shown here:
09.08.2008
![You can find several articles about testing state machines, but the one written by Martin Gomez [1] nicely summarizes the usually used approach.
“The beauty of coding even simple algorithms as state machines is that the test plan almost writes itself. All you have to do is to go through every state transition. I usually do it with a highlighter in hand, crossing off the arrows on the state transition diagram as they successfully pass their tests. This is a good reason to avoid "hidden states"-they're more likely to escape testing than explicit states. Until you can use the "real" hardware to induce state changes, either do it with a source-level debugger, or build an "input poker" utility that lets you write the values of the inputs into your application.”
Written in 2000 this still describes the common practice. In the following text I’ll show how the sinelabore codegenerator can support you in testing state machines. Furthermore a waveform viewer is suggested to display the test results.
The latest beta version of the codegenerator automatically finds and prints ways through a given state chart. The algorithm ensures that all transitions are taken at least once (100% coverage). So it is not anymore necessary to go through the diagram with a highlighter in hand. Depending on the complexity of the machine not a single way can cover all transitions but several ways are necessary starting from different states.
The following command creates the state machine code from the equivalent.xml specification (see example 1). In addition it prints out one or more routes to test the state machine. The new command line option ‘-c’ switches on the path coverage output.
java -jar ../../../testcases/codegen.jar -v -c -p md -o equivalent -t "Model:Equivalent" equivalent.xml
The output looks the following](9_Model_based_testing_of_state_machines_files/shapeimage_3.png)

Figure 3: Waveform view for the test pattern data testpatternCoverage1to8. The first seven signals show the inputs and outputs of the machine. The last two signals show the active top level state (RootLevel) and the active child state within the Active state.
It’s quite simple to create such a diagram. The wave viewer is based on styx which was developed from the university of Hamburg / Germany. A little piece of C-code (see end of the article) creates the required input file which is then displayed from the waveform viewer. This works without any further help if the test-bed is running on a PC.
If you run the state machine on the target hardware this is usually not that simple anymore. But there might be a serial interface left that allows you to send the required data (in a compact format) to the PC. On the PC side a small tool is then needed to write the received data to a file.
The updated test driver would then look this:
// select test data
pTestdata = testpatternCoverage1to8;
size=sizeof(testpatternCoverage1to8)/sizeof(IO_DATA);
char buffer[20]={0};
do {
// dump inputs here
updateTime(testpatternIdx);
startItem();// start a new data set
sprintf(buffer,"%d",getEnable());
addItemAttribute("Enable",buffer);
sprintf(buffer,"%d",getS_ChannelA());
addItemAttribute("A",buffer);
sprintf(buffer,"%d",getS_ChannelB());
addItemAttribute("B",buffer);
equivalent(&instData);// execute generated state machine
// dump outputs and state
sprintf(buffer,"%d",getReady());
addItemAttribute("Ready",buffer);
sprintf(buffer,"%d",getS_EquivalentOut());
addItemAttribute("Out",buffer);
sprintf(buffer,"%d",getError());
addItemAttribute("Error",buffer);
sprintf(buffer,"%d",getDiag());
addItemAttribute("Diag",buffer);
addItemAttribute("RootLevel",states[instData.stateVar]);
addItemAttribute("InnerActive",states[instData.stateVarActive]);
endItem();// end of data set
testpatternIdx++; // next test pattern
if (testpatternIdx == size)// end?
shouldStop=1;
}while(shouldStop==0);
To display the stored waveform just type java -jar scviewer.jar testpatternCoverage1to8.xml showing the waveform.
This article has discussed how to test state machines and how the sinelabore codegenerator and a waveform viewer can support you. I hope this article was useful for you. And as usual is your feedback very welcome!
Figure 2: State machine with test path from step 0 to 8 as suggested from the coverage algorithm.
Figure 1: Coverage output generated from the code generator for the equivalent state machine.