Time-triggered Communication for Distributed Control Applications ina Timed Computation Model

Guido Menkhaus, Michael Holzmann and Sebastian Fischmeister, University of Salzburg, Austria

Abstract

Distributed real-time control applications consist ofsets of tasks that interact with the physical worldthrough sensors and actuators and are executed on adispersed set of locations that are interconnected bya communication subsystem. Timeliness and safetyrequirements of the application demanddeterministic execution of tasks and predictivecommunication. Deterministic and predictablesystems can be build, if upper bounds for processingand communication latencies are known and eventarrivals have deterministic distributions.In this paper we describe the timing definitionlanguage (TDL) system architecture implementingtime-triggered computation and time-triggeredcommunication. The TDL system implements thetimed computation model and its architectureconsists of two parts: TDL-Exe (for time and valuedeterministic execution of tasks) and TDL-Com (forpredictive communication of values). The paperpresents TDL-Exe and describes implementationdetails of TDL-Com.Keywords: real-time control systems, timedcomputation model, time-triggered communication,system architecture

1 Introduction

Most modern control applications are implementedin software, dedicated to perform specific tasks andinteracting with the physical world through sensorsand actuators. Physical and software processes differconceptually in their treatment of the role of time[1]. Physical processes evolve in real-time andsoftware processes evolve in so-called soft-time.Soft-time coincides with real-time only at theinstances of input and output activities. Timed

real-time programming deals with the activities ofmapping soft-time to real-time [2].Real-time control systems are often implemented asdistributed systems where a set of computationalnodes is interconnected by a communication system.The dispersal of application resources,organizational issues, and fault-tolerancemechanisms are design rationales for distributedsystems. In such a system, each node executes a setof tasks contributing a specific functionality to theoverall control system. Individual tasks cooperateand results of one task may require to becommunicated to tasks located at different nodes.The design of a real-time distributed control systemcan broadly be divided into global and local designissues [3]:

• Global design decisions.Global designdecisions deal with activities that are relevant tomore than one computational node (e.g.,communication concerns). These activitiesmust be coordinated among the computationalnodes and require to work consistently togethertowards the goal of the control system.

• Local design decisions.Local design decisionsare concerned with activities within a singlecomputational node (e.g., the set of tasksreleased on this node need to be invoked andexecuted in a consistent and synchronizedway).

The correctness of a real-time control systemdepends on the computed values and on the point intime, at which these values are available. A keyproblem of distributed real-time systems is thetimely interaction between the system and theenvironment while maintaining consistency andcorrectness of data. To achieve consistency in

In Proc. of the 23rd Intl. Digitial Avionics and System Control Conference (DASC'04), 2004.

1 of 12

distributed systems in the time and value domain,the system must be made deterministic. Beingdeterministic is mandatory for software systems thatcontrol physical systems that are ruled bydeterministic physical laws [4].

• Non-determinism.A non-deterministic systemdoes not have a unique output sequence to agiven input sequence. An external observercannot consistently predict the behavior of thesystem. In a deterministic (i.e. predictable)system the development of future states of thesystem can be predicted.

• Value determinism.A systems is said to bevalue deterministic, if the same sequence ofinputs produces the same sequence of outputs.

• Time determinism.If the system produces forthe same sequence of inputs the same sequenceof outputs at always the same time, it istime-deterministic.

Determinism in time-triggered systems isaccomplished by introducing a logical executiontime (LET) and a logical computation time (LCT)for each task as well as a logical transmission time(LTT) for communicating messages betweencomputational nodes.

• Logical execution time.Input and output portsdefine logical points of interaction betweentasks. The start of the LET marks the point intime when the values from input ports to a taskare read. The end of the LET marks the point intime when the results of the computation of atask become available at the output ports toother tasks or actuators. Even if the output of atask became available prior the end of the LET,the output values will not be released prior tothe expiration of the LET.

• Logical computation time.The LCT specifies afixed time interval in real-time in which thetask is active. The task is schedulable, if thereis enough soft-time available within the intervalto execute the task. According to a schedulingscheme, the task starts after it has beenreleased, may be preempted, but resumes andcompletes its execution before the LCT has

elapsed. The release and the termination of atask are time-triggered events emitted at thestart and the end of the LCT. The LCT of a taskis always greater or equal to its worst caseexecution time (WCET).

• Logical transmission time.The LTT isdetermined by a time-triggered communicationschedule. The schedule defines when thecommunication system transfers values fromone computational node to another and whenthe next transmission will take place. The LTTof transferring a value from one node to anotheris always greater or equal to the worst casecommunication time (WCCT) [5].

The LTT is important for the inter-nodecommunication and modeled in the global designdecisions. The LCT is determined during the localdesign decisions and deals with computationalresources on a single node.

• Centralized Application.For controlapplication executed on a single computationalnode, the length of the LET equals the length ofLCT (see Figure 1). Local intra-nodecommunication is instantaneous, which meansthat reading inputs and writing output happensconceptually in zero time.

• Decentralized Application.To sustain theprinciple of instantaneous communication in adistributed control application, the LETconsists of the LCT and the LTT (see Figure 2).

Release Terminate

Logical execution time = Logical computation time

Reading

input

ports

Writing

output

ports

t t +T

Task t

Suspend Start Resume Stop

Figure 1. Logical execution time in a centralizedapplication.

Information necessary to determine the LET of atask is the invocation frequency that the control lawof the controlled object requires. The length of the

2 of 12

Release Terminate

Logical execution time

Reading

input

ports

Writing

local and non-

local output ports

t t +T

Task t

Suspend Start Resume Stop

Communication

Logical computation time Logical transmission time

Time slot k Time slot l

Figure 2. Logical execution time in a decentral-ized application.

LCT must be larger than the WCET. Theschedulability test of the TDL compiler verifies thisproperty. The length of the LTT is determined by theWCCT, which represents the upper bound requiredto transmit a message from the sender to the receiverover the network.In this paper we present the TDL system architecturethat allows for the design of control applicationsusing time-triggered computation and time-triggeredcommunication. The TDL [6] bases on the conceptof a fixed LET of tasks. The communication systemof TDL provides the programmer with timingabstraction for implementing distributed controlsystems with hard real-time constraints. For anend-to-end time-triggered approach, it is necessaryto provide time and value determinism for the globaland local design.The remaining of the paper is structured as follows:Section 2 presents the motivation of the work.Section 3 discusses related work such as TTP andFlexRay. Section 4 presents the TDL systemarchitecture for centralized and decentralizedsystem. Section 5 provides details about theimplementation and characteristics of thedecentralized system are discussed in Section 6.Section 7 concludes the paper.

2 Motivation

The software implementation of distributed real-timecontrol applications must be predictable yet flexible.Predictable, because hard real-time applications aretime and safety critical. Flexible, because all tasksdo not have to be specified fully pre run-time.Non-deterministic system require over-sizing of

computing and communication resources to avoidtime delays in worst load case situations: forexample in dynamic control systems, in which theupper limit of processing and communicationlatencies is unknown and the event or task arrivalhave non-deterministic distributions [7].Event-triggered systems offer more choices forscheduling task computation and communication.However, it is difficult to build deterministic systemsutilizing event-triggered communication togetherwith jitter introduced by communication-errorcorrection.In time-triggered systems, communication andcomputation of tasks are predictable. Predictabilityimplies deterministic temporal system behaviorunder the imposed timing and functional constraints.For those systems the functional timingrequirements are always met. Static schedules areused to plan task executions on each computationalnode. But for this, the release time of tasks must beknown a priori. Time-triggered communicationprovides predictable message transmission. Staticschedules drive the communication system anddetermine the timely transmission of messages.

3 Related Work

A number of time-triggered systems have beendevised in the context of distributed controlapplications for safety critical hard real-timesystems.The time-triggered protocol (TTP) [8] providestime-triggered communication of messages andstatic cyclic scheduling of application tasks. Amember of the TTP family is the time-triggeredprotocol TTP/C, intended for safety critical hardreal-time applications, and TTP/A, intended forlow-cost field bus applications. TTP/C providesdistributed fault tolerant clock synchronization, errordetection, membership service, and redundancymanagement. Cyclic scheduling of application tasksis described with the task descriptor lists (TADL).The TADL specifies the time of starting andstopping a task and the WCET of a task. It describesthe temporal behavior of the system before thesystems starts. The finishing time of a task isdetermined by the number and length of

3 of 12

preemptions. Results are immediately available toother tasks on the same node after the finishing time.A consortium of companies supports and promotesthe FlexRay Communications System [9, 10], whichis a communication infrastructure for high-speedcontrol systems targeting the automotive domain.The communication cycle of FlexRay consists of astatic and a dynamic segment. Each communicationcycle starts with the static segment. Similar to thetime-triggered protocol, all communication isdivided into slots, which the developer assigns toindividual nodes. Following the static segment, thedynamic segment is intended for aperiodic messagessuch as burst transmissions or diagnosis information.The dynamic segment utilizes the flexibletime-division multiple access (FTMA) protocolByteFlight [11] that uses message identifiers asmeans for messages scheduling. Applications usingFlexRay can be implemented with OSEKtime [12].OSEKtime is responsible for starting tasks accordingto a periodic task execution scheme (similar to theTADL) and it monitors the task deadlines. Thetime-triggered tasks can preempt each other.TTP and Flexray provide time-triggeredcommunication that ensures time determinism on aglobal level. However, on a local level, i.e., on asingle computational node, time determinism andvalue determinism cannot be guaranteed. We presentthe TDL system architecture for time-triggeredcomputation and communication that aims at valueand time determinism on a global and local level.

4 TDL System Architecture

The design process for a TDL control applicationcan be split into a local design process, targeting acentralized single processor solution, and a local andglobal design processes for distributed controlapplications. The run-time environment for acentralized processor TDL control application is theTDL runtime environment (TDL-Exe), consisting ofthe E machine [13]. The TDL-Communicationenvironment (TDL-Com) complements it fordistributed control application.We first present the activities of the local designprocess and the local runtime environment beforediscussing the global design process and the

TDL-Com.

4.1 Centralized System

The following steps lead to a TDL application forthe centralized, single processor solution (see Figure3).

1. Task-Set Declaration.A TDL program can bemodeled using a visual task modeling tool [14].The essential idea of the timed computationmodel is time-triggered cyclic computation inwhich the LET, the LCT, and the WCETdescribe the timing behavior of a task.

The most important programming abstractionsof TDL are modules, modes, tasks, and ports:The highest-level programming construct is amodule. A module declares a set of modes,which specify sets of tasks and other activitiesthat are executed periodically and in parallel. ATDL module can only be in one mode at a time,but can change from one mode to another at theend of a period. A module may have a startmode. If a module has a start mode it is anexecutable TDL program and the applicationdescribed by the TDL program starts executingin this mode. A task has a set of input ports,output ports and a set of drivers that handles thedata for the ports. Ports are logical points ofinterconnection between tasks and modes. Taskdrivers copy values from an output task port toan input task port, and there are drivers forsensor readings and actuator updates. Modedrivers read sensors and update mode ports,which are a subset of the task output ports.

2. Task Timing Definition.A TDL mode specifiesthe invocation period, i.e., the length of onecomputation cycle. The LET of a task isdetermined with respect to the invocationperiod of the mode to which the task isassigned. It is calculated by dividing theinvocation period of the mode by the tasksfrequency.

Task drivers, sensor drivers, and actuatordrivers differ in the fact that task and sensordrivers are called at the beginning and the end

4 of 12

Environment

Driver code TDL−Exe (E machine) Driver code

Platform

Sensor Actuator

TDL model

Simulink

E code

Modeling tool

TDL program

Applicationobject code

Applicationsource code

Systemlibraries

runs on

calls

callscalls

executes

off−line

on−line

simulates

generatesgenerates

compiles and links intocompiles into

generates

Figure 3. Centralized TDL system architecture for a single processor application.

of the LET, whereas actuator drivers are treatedlike tasks having their own invocation period.

If the set of tasks is schedulable, the TDLprogram is time-safe. However, time safetydepends on the correct analysis and calculationof the WCET of each task for the givenplatform.

3. TDL Program.The visual modeling toolgenerates a TDL program.

4. TDL Model.On the basis of a TDL program, aTDL model generator produces a Simulinkmodel [15]. Simulink is designed for thesimulation and modeling of control laws. Formodeling, Simulink provides a graphical userinterface for building models as blockdiagrams. It includes a comprehensive blocklibrary of sinks, sources, components, andconnectors. The Simulink model that resultsfrom the TDL program (the TDL model) can besimulated to validate the timing and functionalbehavior. This is especially helpful, if theapplication code has been modeled in Simulink.

5. E code.The TDL compiler compiles a TDLprogram into E code for the E machine [16].The E code is a platform-independentassembler language that targets the E machine.The E machine executes E code that ensures thetiming consistency of the task and driverexecutions. The E code consists of a small setof instructions for basic control flow andprocessing, that allows for synchronous drivercalls, task scheduling and initializing theexecution of a set of E code instructions atsome point in time in the future.

6. TDL-Exe (E machine).The TDL runtime partof the architecture is represented by theTDL-Exe consisting of the E machine. TheE machine is a virtual machine that executesthe platform-independent E code and calls theplatform-dependent application code.

5 of 12

Com FT Nat

Com FT Nat

Com FT Nat

TDL-Com

TDL-Exe

N1

TDL-Com

TDL-Exe

N2

TDL-Com

TDL-Exe

Bus

N3

Figure 4. Distributed TDL System architecture.

4.2 Decentralized System

The TDL system architecture for a decentralized (orequivalently distributed) real-time control system ismodeled by a set of TDL nodes that areinterconnected by a real-time communicationsystem. Each node consists of a TDL-Exe and aTDL-Com part (see Figure 4).The TDL-Exe is not concerned with the globaldesign of the communication between nodes,because the synchronization is done implicitly bythe timing definitions of the tasks on each node.Each task provides the data items that need to betransmitted to a different node. However, the timingdefinitions of tasks on each node are designed inmutual agreement with tasks running on other nodes.TDL-Com supplies a data communication systemthat allows for transmitting values from task outputports of a TDL program to input ports of a taskrunning on a different node. TDL-Com uses atime-triggered communication subsystem totransmit data. It works autonomously: sending andreceiving of messages happens without anyinteraction from the application program.

4.3 TDL-Com Interfaces

TDL-Com exposes three interfaces to TDL-Exe (seeFigure 4):

1. Com Interface.The Com interface mediatesbetween TDL-Exe and the communicationsubsystem of TDL-Com. It allows theTDL-Exe to submit and retrieve data values viadrivers that are connected to input and outputports. Input and output ports define logical

points of interaction between tasks and theCom interface. The drivers copy values ofoutput ports of a task to the input ports of theCom interface, which forwards them to thecommunication network interface. The Cominterface has a set of output ports, whose valuesare destined for input ports of tasks, which arethen retrieved via tasks drivers.

2. FT Interface.The FT interface allows foraccess to information related to fault tolerance.Redundantly produced and communicatedvalues have status fields, which provideinformation, for example, on the number ofactive replicas, the status of fault-tolerantcommunication, or the confidence values offusion algorithms.

3. Native Interface.The native interface allowsfor access to platform specific services, such asthe error counter of the CAN controller.

4.4 TDL-Com Architecture

Figure 5. Distributed TDL Architecture

Figure 5 illustrates the TDL system architecture of anetwork node of a distributed TDL application.TDL-Exe provides the runtime environment which

6 of 12

executes TDL applications. TDL-Com builds on atime-triggered communication subsystem. Thecommunication subsystem of each network nodeprocesses autonomously all communicationactivities. The communication network interface isthe interface between the communication controllerand the TDL-Com layer. The communicationnetwork interface contains messages, which are sentand received by the communication subsystem.TDL-Com reads data from the communicationnetwork interface and passes it to the TDL-Exelayer. TDL-Com writes data to the communicationnetwork interface to send data submitted by theTDL-Exe layer. The interface between TDL-Exeand TDL-Com is the Com interface that allows fordata exchange via drivers and ports.The communication schedule list determines thetemporal behavior of the communication subsystem.Each node stores such a list. This list specifies atwhich point in time (time slot) a node is allowed tosend messages and at which point in time it willreceive messages.Figure 5 shows an example of sending and receivinga message. TaskT1 produces a value A and submitsit to TDL-Com in step 1. In Step 2, TDL-Comcopies valueA to locationL1 within thecommunication network interface (it will becommunicated in time slot 2 with messageM1).When sending messageM1in Step 3, thecommunication subsystem reads the data from theindicated locationL1 in the communication networkinterface, generates the messageM1and transmits it.In Step 4, the communication system receivesmessageM2during time slot 3. It stores its data atthe indicated locationL2 . TDL-Com copies the dataitemD from the communication network interface tothe valueD within the Com interface (Step 5).Finally, taskT2 consumes the value in Step 6.

Temporal Synchronization of TDL-Com. Asdescribed above, there are temporal dependenciesbetween the TDL-Exe, TDL-Com, and itssubsystems. For example, a value has to beproduced, copied, and marshaled before itstransmission. The dependencies result from the factthat the time-triggered communication subsystemruns autonomously and the progression of time

drives the timing of the whole node. The activitiesof the TDL-Com and the TDL-Exe layer need to besynchronized to the activities of the communicationsubsystem.From the point of view of TDL-Exe, communicatingvalues between tasks (reading input and writing tooutput ports) is transparent in centralized as well asin the decentralized applications. However, incentralized applications, the LCT determines theLET of a task (i.e., LCT = LET). In decentralizedapplications, if values of output ports of this tasksneed to be transmitted to a task on a different node,the LET consists of the LCT plus the LTT (i.e., LET= LCT + LTT). The LTT always succeeds the LCT.LET includes the transmission time of a message.Consequently, from the point of view of TDL-Exe,inter-node communication happens conceptually inzero time (i.e., transparent to the single-nodecomputation model).TDL-Com and communication subsystemsynchronize their timing via a communicationschedule list. TDL-Exe is implicitly coordinatedwith the communication subsystem by the timingdefinitions of the tasks on each node. However, fromthe communication schedule list and the local designof timing definitions of the tasks may raise globaldesign restrictions (scheduling restrictions), thatneed to be resolved.

4.5 TDL-Com Toolchain

The construction of the communication schedule listof the communication subsystem is an off-lineactivity. The schedule is then used on-line (duringruntime) to ensure predictive communication.Figure 6 illustrates the off-line activities of theTDL-Com toolchain.

• Off-line. The off-line part of the toolchaindetermines the communication requirementsfor the distributed application and generates aglobal communication schedule list and theTDL-Com schedules for each network node.

To generate the global communication schedulelist, the TDL-Com Compiler determines thecommunication requirements of the wholedecentralized application. It scans TDLprograms and modules and detects accesses to

7 of 12

non-local data ports. Access to remote portsresults in communication requirements.Optimization of communication can byachieved, for example, by packaging severaldata items within the same invocation periodinto a single message.

A TDL-Com Compiler plug-in maps thecommunication requirements into the format ofa vendor-specific bus scheduling tool, whichgenerates the network schedule for a specificcommunication platform (e.g., TTP, FlexRay,TTCAN). The plug-in reads and analyzes thegenerated network schedule and provides theTDL-Com Compiler with information togenerate the TDL-Com schedule lists for eachnode.

• On-line.The TDL-Com layer providesinterfaces to TDL-Exe and to thecommunication subsystem (see Figure 5). Itcopies values to be transmitted between theCom interface and the communicationcontroller interface and vice versa, marshalsvalues into communication frames and supportsvoting for fault tolerance. The behavior of theTDL-Com layer is statically pre-defined beforeruntime. The activities are cyclically repeated.

Figure 6. Overview TDL-Com tool chain

5 TDL-Com PrototypeImplementation on Top of CAN

The prototype implementation of the TDL-Com usestime-triggered CAN (TTCAN) [17] ascommunication protocol. TTCAN extends the CANprotocol by providing time-triggered communicationvia the standard physical CAN link. CAN [18] is amature standard and widespread in the field ofautomation as well as in the automotive field.

We implemented a proprietary version of theTTCAN protocol in software which we call softwareTTCAN (sTTCAN). The reasons were as follows:(1) It allows for adaptation to the needs of theTDL-Com prototype implementation, (2) there wereno embedded boards equipped with TTCANcontrollers available, (3) the TTCAN chip was stillin evaluation status, (4) lack of a supporting toolchain. A benefit of the TTCAN softwareimplementation is that it can be run on standardembedded boards and ECUs without the need tochange or modify the hardware to profit fromreliable, time-triggered communication. Like theTTCAN implementation by Bosch [19], we supporttransmitting sporadic messages within dedicatedtime slots. The objective of our implementation is toprovide an implementation for the OSEK/VDXoperating system.

The TDL-Com implementation bases on theMotorola MPC 555 Power PC derivate and theOSEK/VDX operating system. The processor chipalready integrates a variety of common I/O (e.g., twoCAN controllers). KANIS OAK EMUF boards,hosting the Motorola MPC 555 processor, are thetarget hardware for our implementation. Eachnetwork node consists of one KANIS OAK EMUFboard, the boards are interconnected via a CAN buslink. Each board features two physical CAN linksincluding bus drivers and on-board connectors.

The prototype is implemented under OSEK/VDXusing OSEKWorks, an OSEK implementation byWindRiver and the development environmentsupplied by WindRiver [20].

8 of 12

5.1 Clock Synchronization

Time-triggered communication requires asynchronized time base among the participatingnodes to provide a time division multiple access(TDMA) bus arbitration scheme. TDMA allows fora number of nodes to access a single transmissionchannel without interference by allocating uniquetime slots to each node within each channel.The sTTCAN implementation of clocksynchronization is inspired by TTCAN [17].TTCAN uses a master-slave clock synchronizationscheme based on the idea of the TTP/A fireworksprotocol [21]. A dedicated station, the master,periodically sends a synchronization frames, whichother nodes use to synchronize their local clocks tothe clock of the master.We describe the clock synchronization algorithmusing the notation of event-recording automata [22].Timed automata have been introduced to model thebehavior of real-time systems. They augment finitestate automata with a set of clocks. Event-clockautomata are timed automata that correlate the valueof clocks and the occurrence of events and maintaintheir correspondence.An automaton consists of a set of locationsV and afinite set of event-recording clocksX. We writexl

to denote that the clockx ∈ X is assigned to alocationl ∈ V . The infinite wordw = (s, t0)(s, t1) . . . is emitted by the master nodeand is the input to the automaton that describes theclock synchronization system.1pi

, with anequidistant periodpi = ti − ti−1 for all i is thefrequency of the occurrence ofs. The content ofmessages is the value of the clock of the masternode at timest0, t1, . . .. We writexl

s to denote thetime of the clockx at locationl at reception ofsynchronization messages of the master node. Wewrite xm

s to denote the value of the clock of themaster node at the time when sending the messages.At reception of messages at locationl, we computethe deviationdl

s between the value of the masterclockxm

s and the value of the clock at locationl (weassume an instantaneous transmission of messages):

dls = v(xl

s)− v(xms ).

Clock values are measured in ticks. They are splitinto a macro tick and a micro tick part. We write

〈v(x)〉 to denote the micro tick part of a clock andbv(x)c to denote the macro tick part ofv(x), suchthatv(x) = bv(x)c+ 〈v(x)〉. Macro tickstM arecounted in our implementation in OSEK systemtimer ticks. A macro tick is composed of a specificnumberm of micro tickstm (CPU timer ticks), suchthattM = m0 · tm at timet0.To synchronize a clockxl

s, the nominal number ofmicro ticks for every macro tick is increased to slowdown the clock and it is decreased to speed it up. Tocompensate for a clock deviation ofdl

s in periodpi

in the next synchronization period, the number ofmicro tickstmi,corr need to be added to the nominalnumber of micro ticks that make up a macro tickduring the next periodpi+1. tmi,corr is computed bydividing the deviation by the length of periodpi.Thesign(dl

s) indicates a positive or a negative clockdeviation.

tmi,corr = sign(dls)

dls

bpic .

The number of micro ticks that make up a macrotick for the next periodpi+1 is computed as

tM = (mi−1 + tmi,corr) tm.

with mi = mi−1 + tmi,corr. tmi,corr is usually afraction number and the value is split into an integraland a fractional part. The integer part is immediatelycorrected as shown above. The fraction part isaccumulated and corrected within the current periodby adding or subtracting one moretm as soon as theabsolute sum exceeds 1.Valuation of clocks uses the time stampingmechanism of the MPC555 CAN controller. Itautomatically generates a time stamp at the time ofstart of frame [18] of an incoming message. Theusual method of generating time stamps withinterrupts is imprecise because of interrupt latencies.Furthermore, using interrupt service routines togenerate the time stamps causes delay of runningtasks and increases CPU load, which mightnegatively influence the temporal predictability ofthe system.With the current implementation of the clocksynchronization algorithm we achieve an accuracyof 20µs, i.e., a maximum deviation of +/- 10µs ofthe slave clocks from the master clock. The CAN

9 of 12

bus is currently run at 100kBit. Increasing thetransmission speed of the CAN bus from 100kBit to1MBit and thus decreasing the length of the bit cellon the bus and improving the synchronizationalgorithm by adding rate correction of the localclocks we expect to improve the overall accuracydown to 5µs.

5.2 Time-Triggered Communication

The clock synchronization algorithm of sTTCANcoordinates the OSEK system timer of theOSEK/VDX operating system of the involved nodes,which allows for the implementation oftime-triggered communication based on the CANbus using solely OSEK API functions.Time-triggered communication is done in aperiodically, recurring pattern, the so calledcommunication round [21]. The synchronizationmessage that is periodically sent by the master node,indicates the start of a new communication round. Acommunication round is subdivided into time slots,in which only one dedicated node is allowed toaccess the network and to transmit messages. Thelength of time slots is fixed and is measured inOSEK system timer ticks (macro ticks,tM ) anddenotes the LTT for the message sent in the timeslot. After reception of the synchronization message(at the beginning of a communication round), thenodes start transmitting messages according tostatically predefined communication schedules thatassigns communication time slots to nodes (seeFigure 7).

Figure 7. A communication round.

The structure of the static communication schedulelist is inspired by the message descriptor list(MEDL) introduced by Kopetz [8]. The schedule islocally stored at each node and it defines for thelocal node the time slots that it may use to access thenetwork to transmit messages.Sending messages is implemented as follows:OSEK alarms are successively set up for each entry

in the communication schedule list indicating thetime slots for the local node. Whenever an alarmfires, the alarm-callback routine triggers the CANcontroller to send the message buffer denoted in thecurrent schedule entry. The next alarm is set-upaccording to the next entry in the communicationschedule list until the list is completely processed.With the beginning of the next communicationround the processing of the communication schedulelist restarts from the beginning.Table 1 shows the structure of the communicationschedule for one node with two entries: The nodesends a four byte message contained in buffer one inthe time slot which starts attM = 2 (time slots arecounted always relative to the start of thecommunication round). It then sends a two bytemessage contained in buffer two in the time slotwhich begins attM = 15.

Pos. Alarm #Buffer Length

1 2 1 42 15 2 2

Table 1. Structure of the communication sched-ule.

Receiving messages is done automatically by theCAN controller, no software interaction is required.The received data is stored within one of the 16buffers of the CAN controller. The TDL-Com stackfetches the data from the message buffers and copiesit to the Com interface where TDL tasks can accessit.Verification of the reception time of messages, doneby protocols for safety-critical applications such asTTP or Flexray, has been omitted in the currentimplementation for performance reason.

5.3 TDL-Com Com-layer

The prototype implementation of TDL-Com consistsof a set of specific drivers and supports only theCom interface yet. The message buffers of the CANcontroller are directly used as communicationnetwork interface (see Figure 5). Forcommunicating data to a receiving node, TDL-Exeexecutes a set of specific drivers that are declared inthe E code of the TDL program that is executed on

10 of 12

the sending node. These drivers copy the values ofoutput ports of a tasks directly to CAN controllermessage buffers. Drivers declared in the E-code ofthe TDL program on the receiving nodes get thesemessages, extract the values, and copy the value tothe correct task input ports.The timing of the communication subsystem and theE machine rely both on the local OSEK systemtimer and on the fact that the OSEK system timer ofall involved nodes are synchronized. If an input portof a task receives a value from a different nodetimely arrival of the value at TDL-Com is ensured.

6 Discussion

The TDL system architecture allows for the designof deterministic control applications usingtime-triggered computation and time-triggeredcommunication. It provides the programmer withtiming abstraction for implementing distributedcontrol systems with hard real-time constraints.However, the current approach of the TDL-Comimplementation has a drawback: It leads to aruntime behavior of the application which exhibitsunbalanced CPU and network load. The TDLsemantic defines, that the first period of all taskswithin a mode period (hyper period) startssimultaneously at the beginning of the modeperiod [5]. In a centralized TDL application, theperiod of all tasks equals the LET and computationof the tasks can be equally distributed over the hyperperiod of the application. In a decentralized system,the LET model defines for each task that the intervalof computation (defined by the LCT) coincides at thebeginning of the task period whereas communicationactivities (defined by the LTT) succeedscomputation and thus occurs towards the end of theperiod. Because the periods of all tasks of a TDLapplication start simultaneously at the beginning ofthe hyper period, there is a peak in computation(higher CPU load) at the beginning of the hyperperiod, whereas communication (network traffic)dominates at the end of the period. To balance theCPU and the network load and to distribute itequally over the hyper period, we currently work onnew approaches, which overcome this drawback ofthe current model on which TDL-Com is based on.

7 Conclusion

The timing definition language (TDL) providesprogramming abstraction for the implementation ofcontrol systems. The TDL system architectureconsists of a runtime environment for deterministicexecution of tasks and predictive communication ofmessages for distributed control applications.Determinism and predictability are requirements forsafety-critical and fault-tolerant systems. The timedcomputation and communication model bases onthese principles.In this paper, we introduced TDL-Com, atime-triggered communication system. TheTDL-Com toolchain produces static scheduling lists,which describe the communication behavior of thewhole distributed application. A globallysynchronized time base is the basis forinterconnected tasks. The tasks that run on differentcomputational nodes need to be coordinated in thetime domain to operate in a consistent and highlysynchronized manner.

8 Acknowledgments

We thank the team of the Software Research Lab ofthe University of Salzburg for their help indeveloping and implementing the ideas presented inthis paper.

References

[1] E.A. Lee, S. Neuendorfer, and M.J. Wirthlin,2003, Actor-Oriented Design of EmbeddedHardware and Software Systems.Journal ofCircuits, Systems, and Computers, 12(3):231 –260.

[2] C.M. Kirsch, 2002, Principles of Real-TimeProgramming.LNCS, 2491.

[3] E. Fuchs and D. Millinger, 1998, Task setdesign tools for an embedded distributedcontrol system. InProceedings of the 8th ACMSIGOPS European workshop on Support forcomposing distributed applications, pages182–188. ACM Press.

11 of 12

[4] N. Halbwachs, 1997,SynchronousProgramming of Reactive Systems. Kluwer.

[5] B. Horowitz T. A. Henzinger, C. Kirsch, 2001,Giotto: A Time-triggered Language forEmbedded Programming. InProceedings ofEMSOFT 2001, LNCS. Springer.

[6] J. Templ, 2003, TDL Specification and Report.Technical Report C059, Computer Science,University of Salzburg, Mar.

[7] B. Ravindran, 2002, Engineering DynamicReal-Time Distributed Systems: Architecture,System Description Language, andMiddleware.IEEE Trans. Softw. Eng.,28(1):30–57.

[8] H. Kopetz, 1997,Real-time Systems: DesignPrinciples for Distributed EmbeddedApplications. Kluwer.

[9] R. Belschner, J. Berwanger, C. Ebner,H. Eisele, S. Fluhrer, T. Forest, T. Fuhrer,F. Hartwich, B. Hedenetz, R. Hugel, A. Knapp,J. Krammer, A. Millsap, B. Muller, M. Peller,and A. Schedl, 2002,FlexRayCommunications System — RequirementsSpecification. BMW AG, DaimlerChryslerAG, Robert Bosch GmbH, GeneralMotors/Opel AG4, April. Version 2.0.2.

[10] Flexray website, 2004.www.flexray.com .

[11] BMW AG. ByteFlight Specification. availableatwww.byteflight.com .

[12] OSEK Group, 2001,OSEK/VDXTime-triggered Operating System Specification,Version 1.0, July.

[13] T.A. Henzinger and C.M. Kirsch, 2002, TheEmbedded Machine: predictable, portablereal-time code. InProc. ACM SIGPLANConference on Programming Language Designand Implementation (PLDI), pages 315–326.

[14] G. Stiegelbauer and A. Werner, 2004,Integration von TDL und Simulink:Komplexitatsreduktion beim Entwurf voneingebetteter Software. InProceedings of the

2nd Workshop: Automotive SoftwareEngineering, Ulm, Germany.

[15] MathWorks, www.mathworks.com.Simulink.

[16] C. Kirsch T. A. Henzinger, 2002, TheEmbedded Machine: Predictable, PortableReal-Time Code. InProceedings of the ACMSIGPLAN Conference on ProgrammingLanguage Design and Implementation (PLDI).

[17] T. Fuhrer, B. Muller, W. Dieterle, F. Hartwich,R. Hugel, and M. Walther, 2000, TimeTriggered Communications on CAN (TimeTriggered CAN - TTCAN). InProceedings 7thInternational CAN Conference, Amsterdam,Netherlands.

[18] Bosch, 1991,CAN Specification, Version 2.Robert Bosch GmbH, September.

[19] F. Hartwich, B. Muller, T. Fuhrer, andR. Hugel, 2000, CAN Network with TimeTriggered Communication. InProceedings 7thInternational CAN Conference, Amsterdam,Netherlands.

[20] WindRiver web site.http://www.windriver.com/ , 2004.

[21] H. Kopetz, 1995, TTP/A The fireworksprotocol. InSAE International Congress andExposition, Detroit, Michigan, February.

[22] R. Alur, L. Fix, and T.A. Henzinger, 1999,Event-clock automata: a determinizable classof timed automata.Theor. Comput. Sci.,211(1-2):253–273.

12 of 12