State Transition Testing

The present system state depends on the past

Alongside input data, preceding steps in the program flow also influence output and general system behavior. In other words, you also need to pay attention to the history of the system you test. State models and state diagrams are used to help design state transition tests.

Guard conditions

Based on its initial state, a system or test object can adopt various new states. Changes in state or transitions between states can be initiated by events such as function calls or system input. If an event leads to two or more potential transitions from a single state, you have to differentiate clearly between them using “guard conditions” that clearly define which transition (and therefore which successor state) follows the event.

Changes in state may trigger actions, and a completed event is said to be in its “end state”. State transitions are modeled using state transition diagrams and/or state transition tables.

Side Note: About state machines

This side note goes into some detail on state machines and assumes that you are already familiar with the basic concept.

A state machine changes its state according to its current state and the next input, which can also be an event. The machine has a finite number of states. The machine can perform an action (such as producing output) in every state or during every transition from one state to the next. It is assumed that:

• Actions are related to transitions, not states
• A state machine is deterministic—i.e., from a given initial state and following a given sequence of inputs (events), the machine will be in an unambiguously defined successor state
• The state machine is finite—i.e., there is a transition for every input in every state. This is not always the case in state transition models. However, this situation can be remedied by extending the machine so that for previously unknown inputs a transition either
  • leads to the same state without preforming an action, or
  • causes a fault

Case Study: Finite state machine for a vehicle software update

Here is an example of a (highly simplified) state transition test:

Fig. 5-4State machine for a vehicle software update

The vehicle software can be updated “over the air” (OTA), but only when the vehicle is not being driven (the “Engine OFF” state in the diagram).

The situation is modeled in a state transition diagram (see figure 5-4). There are three states (Update MODE, Engine OFF, Engine ON), and transitions between the states are indicated by the arrows. The machine has initial and end states.

Under the precondition that the battery supplies sufficient current, the machine transitions from its (pseudo-) initial state to the “Engine OFF” state. From here, there are three possible successor states. If the engine is started, the vehicle is in the “Engine ON” state. This can only change if the “Engine_stop” event takes place and the vehicle switches back to the “Engine OFF” state. If the driver checks whether an update is available by activating the button in the vehicle’s display (the “Update_check” event), the vehicle transitions to the “Update MODE” state. The vehicle remains in this state until the update is completed (the “Update_ready” event) and the vehicle transitions back to the “Engine OFF” state. If the battery is then depleted, the machine switches to its (pseudo-) end state.

The specifications have to define which event (Engine_start, Engine_stop, Update_check, do_Update, Update_ready) can occur in which states and is allowed to trigger an event.

The state transition table (see table 5-11) for this finite state machine (without its initial and end states) looks like this:

Table 5-11State transition table for a vehicle software update

The state machine is equivalent to the table, which shows the successor state for every state, depending on the input/events that occur. While the table illustrates all the possible permutations, the diagram doesn’t show transitions that the specifications don’t allow (indicated by a “-” in the table). These non-specified (i.e., invalid) transitions are nonetheless relevant for testing, as they enable us to identify potential system failures.

A Sample concrete test case

Here is a sample test case with its pre- and postconditions:

Precondition:	The engine is off (“Engine OFF” state)
Event:	Engine_start
Expected reaction:	State transition to “Engine ON” state
Postcondition:	State is “Engine ON”

A state transition test object can be a complete system with various states or, in an object-oriented system, a class with various states. A state transition test is required every time the historical system flow leads to differing behaviors.

More sample test cases

There are various possible levels of testing thoroughness. The minimum requirement should be that the test object reaches all its possible states. In our example, these are Engine OFF, Engine ON, and Update MODE. The following test case evokes all three ([State], Event²⁰):

Test case 1

[Engine OFF], Engine_start [Engine ON], Engine_Stop [Engine OFF], update_check [Update MODE]

However, this test case doesn’t cover all the possible events. If the minimum requirement also states that all events have to be triggered, the test case above needs to be extended to include “do_Update” (no transition), “Update_ready” (transition to “Engine OFF”), and “Car battery voltage low” (transition to the end state).

Test criteria

A useful test criterion to aim for is triggering every possible event for each state at least once. This way, you can be sure to verify that the state model matches the specified system behavior.

Test invalid transitions too

In critical systems, the transitions included in the table but not specified for the state diagram need to be tested too. For example, you could test whether the “Update_check” event can be triggered in the “Engine ON” state, thus triggering a non-specified change of state.

Side Note

Simply testing states and/or transitions separately doesn’t really do justice to the complexity of a finite state machine. A finite state machine describes cycles of virtually any desired depth. In our example, constant switching between the “Engine OFF” and “Engine ON” states is possible and mirrors daily, real-world usage. But how “deep” should testing really go? How can we handle cycles to “pry them open”?

The issue of endless cycles

Introduce test case creation using a transition tree, and testing transition sequences of differing lengths (N-switch coverage).

Transition trees

Test cases created using a transition tree concentrate on testing separate sequences of transitions that always begin with the initial state.

A transition tree illustrates all the state transition diagram’s possible event and input sequences while simultaneously eliminating repeat cycles. If the current sequence leads us to a previously reached state (a simple cycle), the tree is not expanded any further.

The state machine diagram is traversed and the process generates a transition tree as follows:

1. The initial state represents the tree root
2. Each possible transition is represented by a new branch, which in turn ends in a node that represents the successor state. Each pair of nodes is connected by the event that triggers the transition. This “branch creation” step is repeated until:
   1. previously visited state is reached (thus preventing repeat cycles), or
   2. state offers no outgoing transitions (i.e., the path/branch has reached its end state)

The transition tree shown in figure 5-5 represents all the possible paths²¹ that lead from the initial state to an end state (or a state that is already part of the path you are on). Each path from the root to a leaf represents a test case (i.e., a unique sequence of inputs and/or events). Each such sequence contains no repeat cycles.

Fig. 5-5The transition tree for our example

This transition tree describes four test cases that all begin with the same initial state. You can create an extended transition tree by including events that the current state is not designed to react to. If they can be triggered at all, testing such invalid transitions should lead to the event being rejected or ignored, or to an error message.

Triggering every event and visiting each state at least once is a relatively simple exit criterion for a state transition test. However, depending on the requirements, this base criterion won’t always be sufficient. A comprehensive test suite includes paths of varying lengths. For example, if individual transitions (or sequences of transitions) are to be tested from the initial state Za to the end state Zz, a sequence of events has to be executed that leads to the initial state (Za) of the transition sequence you wish to test. You then need to execute the inputs and/or events that trigger the actual sequence of transitions (from Za to Zz) that is due for testing.

n-switch coverage

The length of the transition sequences can vary according to the thoroughness of the test. The process of testing sequences of varying length is called n-switch coverage, where N is the number of states between the initial and the end state of the sequence being tested. The term 0-switch coverage is used if only one transition is required to go from the initial state to the end state. Correspondingly, if there are two transitions (i.e., one additional state), the aim is 1-switch coverage, while a transition with two successor states requires 2-switch coverage, and so on. Note that successive states can be the same if the previous input doesn’t trigger a change of state. This technique enables you to test for cycles in state machines and limit the number of potential cycles.

Transition state testing is a useful technique for system testing graphic user interfaces (GUIs). A GUI usually consists of a whole raft of masks and dialogs that the user navigates using menu commands and keyboard input, or by clicking buttons. If a mask or dialog box is treated as a state and inputs as transitions, you can model all the navigation options using a finite state machine. You can then derive suitable test cases and coverage criteria using the techniques described above.

Case Study: Testing the DreamCar GUI

The previous VSR version of the window-based DreamCar GUI was tested as follows:

The test begins in the main mask of the DreamCar module (State 1). The action²² Settings>Vehicles triggers the switch to the Edit Vehicle dialog (State 2). The Cancel action closes the dialog and transitions the system back to State 1. “Local” tests (in this case to test the functionality of the mask) can take place within a state. This technique can be used to navigate through sequences of menus/dialogs that can be as complex as you like. The transition state model of the GUI helps to ensure that all the available dialogs are included in the test.

Fig. 5-6The transition tree for our example

Test Cases

The complete definition of a transition state test case must include the following information:

• The initial state of the test object (precondition)
• Input data for the test object
• The expected result/behavior
• The expected state of the test object following the test case (postcondition)

Additionally, the following aspects of each transition also have to be defined:

• The state prior to the transition
• The event that triggers the transition
• The expected reaction to the transition
• The expected successor state

It is not always easy to determine the state(s) of a test object. The current state is often the result of changes in the values of multiple variables, making it tricky to verify and evaluate individual test cases.

Our Tip

• Assess the state transition diagram for its testability at the specification stage. If there are too many states and/or transitions, point out the additional testing effort that this would entail and request simplification of the specifications where possible.
• Insist at the specification stage that the various states are as simple as possible to determine, with as few variables and combinations of variables as possible.
• State variables should be simple to query. Additional functions that enable you to read, set, and reset states are a boon to testing.

Defining Exit Criteria

Coverage and exit criteria can also be defined for state transition tests:

• Each state has to be reached at least once
• Every transition is executed at least once
• All transitions that contravene the specifications have to be checked

As with the other testing techniques we have looked at, percentage exit criteria can be defined using the ratio of the available requirements to the requirements that are actually covered by test cases.

Further checks

Highly critical applications can be subjected to further-reaching state transition tests, such as:

• All possible combinations of transitions
• All transitions in every possible sequence using all the available states (if necessary, multiple consecutive times)

However, this depth of testing with its huge number of test cases makes it difficult to achieve an appropriate degree of coverage. It usually makes sense to restrict the number of combinations and/or transition sequences (see the side note on n-switch coverage above).

Benefits and Limitations

State transition tests are useful for any situation in which states influence the behavior of a system or where functionality is affected by the state of the test object. The other testing techniques we have discussed can’t do this because they don’t account for the effects of the system flow in time or the current state that past activities produce.

Our Tip Especially suitable for testing object-oriented systems

• Objects that are part of an object-oriented system can take on varying states, so the method used to manipulate an object therefore has to react to these differing states. State transition testing is therefore a widely used technique for testing object-oriented systems.