·Home ·Table of Contents ·Reliability and Validation 1

Testing real-Time Systems R. Castanet LaBRI, UMR 5800, Université Bordeaux I, 33405 Talence, France P. Laurençot LIMOS, FRE 2239, Université Clermont-Ferrand II, 63173 Aubière, France Contact

Abstract

During the development of a real-time application, a testing step is necessary to check the conformity of the implementation in accordance with its specication. The test sequences, also named temporised tests, have got temporal constraints because not only the events, but also the dates of these events need to be verify. In this article, we fistly present a method which permits to transcribe temporised test in TTCN for a local architecture. Then, as it's generally more convenient to use a distributed architecture, we introduce here a new architecture which permits to take into account the round trip delay of a message in the network and also an expansion of our method for this case. This architecture is based on a reliable communication between a "spy" and the tester.

1 Introduction

Many critical, real-time systems or multimedia applications have time constraints in their specication. Such systems must be tested in order to check their reliability before being used. This is done during the testing step.
But, in order to realize a test campaign, we need test sequences with temporal constraints, which allow the verication of the real-time requirements of the specication and a test architecture which permits to obtain the dierent verdicts. There exists different methods which give test sequences [6, 3] and the resulting tests have got the same structure: it's a sequence of data that must be sent or received during a time interval which is relative to the beginning of the test. It's the comparison between these intervals and the real date when the event occurs that permits to express the verdict of the test: Pass if the result is the one expected, Fail if the result is wrong, and inconclusive if the result is not the one expected, but is valid according to the specication.
The aim of this article is to present an architecture which can be used with temporised tests. The second chapter introduces TTCN (Tree and Tabular Combined Notation [4]). The third chapter presents the different problems to express temporised test in TTCN and the last chapter shows a distributed architecture for tests which have real-time constraints.

2 TTCN language

The Tree and Tabular Combined Notation [4] language is a notation standardized by the ISO for the specifcation of tests. The standard introduces the concept of abstract test suites. This is a description of a set of tests (test cases) that should be executed for testing a SUT (System Under Test) together with the relevant data declarations. The tests are described using a black box model (where the internal behaviour of the SUT is not relevant). Each test case is an event tree in which external behaviour is seen as transmissions or receptions of data or events.
The TTCN language supports two notations that are equivalent: TTCN-GR (Graphics) which is easy to understand for human and TTCN-MP (Machine processable) which is the compiled version of TTCN-GR and which is used for the computer to do the test. As there exist a lot of compilers between GR to MP, it's interesting nowadays to create our test suite in the graphic format.
TTCN-GR is composed of two principal parts:
1. the declaration part : it is used to identify the test and is made up by the declaration of :
   • one or two PCO (Point of Control and Observation) according to the architecture. They can be noted L (lower tester) or U (upper tester),
   • different timers with the associated values,
   • different parameters and variables.
2. the dynamic part : it describes the test suites and associates the verdict for each transaction.

The tests are built up by a behaviour tree (table 1). The events are defined in lines on different indentation levels. The rows on the same indentation are events that describe alternatives, and rows on the next indentation are the continuation of the previous line. A send statement is expressed by a "!" and a received statement by a "?".

Table 1: TTCN example for the test case sc5

All the leaves of the event tree are assigned a verdict which depends on the response of the SUT. TTCN language has also some commands that rules the timers. We explain here rapidly some of them which are going to be used afterwards in the translation of temporised tests in TTCN.

• ? TIMEOUT < timer > : The tester waits for the timer to reach its limit .
• START < timer > < number > : This command permits to set the limit of the timer to "number" and to reset it. Time for a timer always increases.
• READTIMER < timer > < name > : Copy the value of the timer in the variable "name".

Our aim consists in expressing temporised tests in this syntax of TTCN, because we can use afterwards different compilers to obtain tests in TTCN-MP. But, as the expression of time isn't well defined in TTCN, we need a methodology which is introduced in the next paragraph. An extension of TTCN with time has been proposed in [9], but as it isn't normalised yet, we prefer using the standardized version.

3 Temporised tests expression

When we have a real-time specification, if we check only the different events which travel through the network, it's not enough, because we don't verify when the event occurs. So, we need tests with temporal constraints which permit to detect when the event is too late or too soon in accordance to the specification. Different methods already exist, based on the region graph [1], or synchronised temporal products [7]. They give a temporised automaton representing a test sequence where the transitions are constrained by an event and two temporal intervals: the first one "Pass" is used to obtain the verdict Pass, the second one, "Au" is larger than "Pass" and permits to detect the verdict Inconclusive (which is the complementary of Pass in the interval Au). Outside the interval "Au", we have the verdict Fail. For exemple, the test below means that for the transition 0, the event A must be sent between 0 and 3 to continue
the test.Between 3 and 5,the verdict is inconclusive,and otherwise the verdict is Fail. For the transition 1, we receive B between 3 and 7, and for the transition 2, we need to know the time when the transition 1 occurs (h(t1)) to calculate the different verdicts.

We suppose now that we have such a sequence, and we want to transform it in TTCN. In fact, we distinguish two cases:

• the first one corresponds to the reception of the data. In this case, we simply verify that the data has occurred in the good interval and when we obtain a (Pass) verdict, we can continue the test.
• The second case is for the transmission of an event. This case is more dicult because we have to choose the moment when the event must be sent. For that, we use the uniformity hypothesis [2] which permits to say that \all instants in the interval are equivalent". So, we have only to select one moment in the interval. We could use a random function, but as TTCN doesn't have this capability, we are obliged to choose ourselves, before the creation of the test, the moment in the interval when the data must be sent. Generally, we take the middle of the interval, but we could also choose the earliest or latest moment,...

In the example shown above, we have seen that a transition can be constrained by an interval which depends from a previous transition. It's the reason why we are obliged to memorise the moment when a transition occurs. We note fire{i} this instant, where i is the number of the transition. (For example, the Pass interval of the transition 2 depends on the date of the previous transition (h(t1)).)

Before firing a transition, it's necessary to calculate the bound of the different interval. As it is done for all the transition, we have developped a macro in TTCN (Calcul), which permits to do that. The principle consists in doing comparisons between different values, and taking the higher or lesser value. Now, we introduce the two different translations: one for the reception, and one for the transmission

3.1 Reception of a data
When we have a reception like that :


We obtain the TTCN syntax expressed by table 2:

Table 2: Reception of a data in TTCN

3.2 Transmission of a data
For an automaton like that :

The translation in TTCN is less easier than for a reception because we must choose the time when the data is going to be sent. In fact, we are going to wait until this moment arrives and for that, we try to the amount of time to sleep. For example, if we decide to send the data in the middle of the interval, we will have the two next cases:
• if the interval is already started, then wait:= (max Pass -Tcourant) /2,
• if the interval is not started, then wait:= (max Pass +min Pass) /2 -Tcourant.

where Tcourant is the valuation of the original clock (obtained with READTIMER). TTCN syntax is then expressed in table 3 (an expression between [ ] is a condition and only the true one can be chosen):

Table 3: Transmission of a data in TTCN

Now, we want to use a distributed architecture[5, 8] for our test. This mean that the tester and the SUT are separated by a network, and so the the communications between these two entities are no more negligible. In the next paragraph, we introduce a distributed architecture which permits to evaluate the time that a data remains in the network, and an improvement of our method to take into account this case.

4 Architecture for the distributed test

The problem for this architecture is to find the latency for a data to go from the tester to the SUT ( latency out -LO- ) and to come back (latency back -LB-). We need to calculate these delays each time we sent or received a data. For this reason, we have developed the architecture which is explained on figure 1.

Fig 1: A distributed architecture

As every distributed architecture, the tester and the SUT can communicate via a network, which is called primary link. At this classical architecture, we add another link (called secondary link) which is reliable and quicker than the primary link and a \spy". This secondary link is used by the spy to transmit informations to the tester quicker than the SUT. The spy is a small program on the same computer than the SUT that watches all the data that receives or send the SUT ( by spying the socket for example ) and sends to the tester a packet for each data.
Because of this round trip delay which is different for each data, we need three di erent steps to translate a temporised test in TTCN. All these steps are explained below, and are represented in the figure 2.

Fig 2: The different steps for the test

• Step 1: Caracterisation of the secondary link. This step is used to measure the round trip delay for a message to go from the tester to the spy. All these delays are constant and the way back is egal to the way go, because nobody else uses this network. As we know the reacting time of the spy (RTS), we can easily deduce the delay between the tester and the spy. The duration of this latency is egal to : L2 = [(?A' - !A) -RTS]/2
  (Where ?A' and !A mean respectively the date of the reception of A' and the date of the transmission of A).
• Step 2: Begging of the test, resolution of the Lateny Out(LO).
  The tester sends a data on the primary network (!B) and waits for a response from the spy (?B'). As the reactive Time of the spy is known, and also the time that take a message to go from the spy to the tester on the secondary link, we can calculate easily the latency out with this formula : LO = (?B'-!B) - RTS - L2.For Each data which is sent, this latency is evaluated.
• Step 3:Resolution of the Latency Back(LB).
  After receiving a request from the tester, the SUT is going to send back data. At this moment , the spy will see that a message is going to be sent, and it informs the tester. So, we can then calculate the latency back : LB = (?C -?C') + RTS + L2

As all the latencies insert by the network can be evaluated we can modify our methodology to take them into account. The tester must check that the reactive time of the SUT is in accordance with its specification. For example, if T is the bound max for a transition in the specification, then, in this distributed architecture, it becomes (LO+T+LB), because of the latency of the primary network. So, when the tester receives a data, it must already calculate the latency out of the request that permits to receive this data.

For a reception, the conformity is obtained when the date of the reception is in the interval "Pass". The translation in TTCN is shown in the table 4.

Table 4: Reception of a data in a TTCN syntax

For a transmission, the data that the tester has to send must arrive in the (Pass) interval of the SUT. So, the tester has to anticipate the beginning of the interval with the time of the latency out (LO). But, as the delay in the primary network isn't always constant, the tester can only verify ( with the help of the spy) that the data which has been sent arrives at the good time.

We introduce here two new variables t_init and t where t_init is the first delay of the latency out, and t the last one. The difference between these two values is the jiger of the network. It permits to see if the delay in the network has increased or decreased, and to adjust a bit the transmission of data. The values of t and T act are each time valued during the previous transition and renewed in the transition. The problem of sending a data is that we don't know when it will arrive, and the tester can only verify afterwards the date of its arrival in the SUT. So, we obtain the TTCN syntax of the table 5.

Table 5: Transmission of data in a TTCN syntax

5 Conclusion

This article introduces a method to test real-time systems with a distributed architecture. For that, we need a secondary link which is faster than the network (primary link) and a spy which permits to inform the tester when the SUT receives or sends a data. We also show in this article how to translate a temporised test in TTCN which can be used after by all tester companies. Our next aim is to see if this architecture can also be used with distributed SUT.

References

1. R. Alur, C. Courcoubetis, and D.L. Dill. Model checking for real-time systems. In Proceedings of the 5th Annual Symposium on Logic in Computer Science, pages 414{425. IEEE Computer Society Press, 1990.
2. G. Bernot, M. Gaudel, and B. Marre. Software testing based on formal specifications: a theory and tool. Software Engineering Journal, 6(6), 1991.
3. A. Ennouaary, R. Dssouli, and A. Elqortobi. Génération des tests temporisés. In Proceedings of CFIP, Liege, Belgique, 1997.
4. Information Technology ISO/IEC 9646-3. Open Systems Interconnection-Conformance Testing methodology and framework- Part 3: Tree and Tabular Combined Notation (TTCN).
5. Information Technology ISO/IEC 9646-4. Open Systems Interconnection- Conformance Testing methodology and framework- Part 4: Test Realization.
6. P. Laurençot. Intégration du temps dans les tests de protocoles de communication. PhD thesis, Université de Bordeaux I, 1999.
7. P. Laurençot, O. Koné, and R. Castanet. On the Fly Test Generation for Real Time Protocols. In Proc of International Conference on Computer Communications and Networks, Lafayette, Louisiane, U.S.A, 1998.
8. O. Rafiq, R. Castanet, and C. Chraibi. Towards an environment for testing osi protocol. In 5th International Workshop on Protocol Specification, Testing and verification, Toulouse, France, 1985.
9. T. Walter and J. Grabowski. Real-time ttcn for testing real-time and multimedia systems. In 10th International Workshop on testing of Communication, Korea, 1997.

© AIPnD , created by NDT.net

|Home|    |Top|