Distributed Control Lab-

A component-based application

OverviewArchitectureExperiments

Outline

MotivationThe Distributed Control Lab (DCL) Architecture“Foucault's Pendulum Details”

– Hardware / Software Architecture– Control Algorithms

“Hau den Lukas”– Hardware Architecture– Control Software

Additional ExperimentsMalicious Code Problem / Solutions

Motivation

Online access to physical experiments over the WebTest-bed for interconnected middleware-components and embedded systemsReach a predictable system behaviour in unstable environmentsStudy techniques to prevent malicious code damaging physical equipment Foucault's Pendulum demonstrates usage of dynamic reconfiguration for online replacement of user control

Distributed Control Lab

2001 project start at Hasso-Plattner-InstitutePractice of writing control algorithms for real-time control problemsstudy of system predictability, availability and security in context of middleware-based dynamic control systems Extensible architecture for hosting physical control experimentsInvestigation of algorithms for user code observation and replacement of control componentsExperiment : physical installation and specific control software

The Distributed Control Lab

Distributed Control Lab (V2)Architecture

Admin

ExperimentManager

Ticket Service

Job Queue

R

R

SOAPFrontend

R

RResult Manager

Job Results

ExperimentController

Tickets Users

DCLFrontend

User

R

SOAP

ExperimentController

R

RSOAP

R

Experiment Service

DISCOURSEUsers

ClientWeb-Browser

DCL Messaging Protocols

http/htmlASP.NET

Page

IIS ASP.NETWeb/Application

Server

DCL-Web Service

IIS ASP.NETWeb/Application

Server

DCL Experiment-/

Resultmanager

.NET RemotingService

DCLExperiment-

Controller

.NET RemotingService

EmbeddedControlDevice

RT-Linux PCBrick OS

...

DBMS

SOAP

.NETRemoting

ODBC

.NETRemoting

TCP/IP

IrDA

Problem : Malicous Code

Investigation of Solution for malicious code detection– Source code analysis– Experiment-specific languages / Language limitations – .NET code access security– Simulation before execution on physical experiment– Analytic Redundancy

Online observation of user programsReplacement of user programs before reach of uncontrollable stateDynamic reconfiguration of component-based control applicationMonitoring of environmental settings and component states

Foucault’s Pendulum

First installation 1848 by Leon Foucault in the Pantheon in ParisDemonstrates earth rotationToday many installation including one in UN-building in New York Problem : Pendulum must be kept swingingSolution : electro magnet under an iron ball Experiment: Find best control algorithm to keep the pendulum swinging

– Using minimum energy – Reaching the highest amplitude

USB Host-Controller

User-Controller

Safety-Controller

EventDuplication

Operating System I/OCheck HalfFull Flag

ConfigurationManager

Binary Reader / Filter, Smooth

Event Creation

Binary Writer

Binary Stream Generation

User Mode

Kernel Mode

Cypress EZ-USB

Read

Write

64 Byte

I/O request packetLight Sensor

Electro Magnet

FIFO-Memory 4 KBytes

ReadFile() WriteFile()

Pendulum Control – detailed• Laser light barriers sampled into

4KByte FIFO-memorys with 23,4 kHz

• USB-Controller checks half-full-Flag

• 64 Byte blocks of data transferred via USB 1.1

• Real-Time OS-Threads process in-coming signals / produce out-going bit stream

Pendulum-API : Control Event

public class ControlEvent{

public int nr; // sensor or actuator identifierpublic int state; // actuator 1 ON - 0 OFF

// light barrier // 1 light -> dark// 0 dark -> light

public long timestamp; // global time stamp}

Pendulum-API : Interface

public interface Pendulum{

// Dequeue next event// Blocks if no event present until next event occurspublic ControlEvent GetNext();

// Queue next event to put energy on / offpublic bool SendEvent(ControlEvent input);

// Get global time stamp 1 micro seconds logical resolution public long GetTime();

}

Pendulum: Example Control Code

while(true){

// get next eventControlEvent ev = pendel.GetNext();if(ev.state == 1) // pendulum enters light barrier{

// switch magnet onpendel.SendEvent(new ControlEvent(ev.timestamp+1000,0,1));// switch it off after 5 mspendel.SendEvent(new ControlEvent(ev.timestamp+6000,0,0));

}}

Pendulum Control ProcessGeneration

while(true){

// get next eventControlEvent ev = pendel.GetNext();if(ev.state == 1) // pendulum enters light barrier{

// switch magnet onpendel.SendEvent(new ControlEvent(ev.timestamp+1000,0,1));// switch it off after 5/23 secondspendel.SendEvent(new ControlEvent(ev.timestamp+6000,0,0));

}}

send to DCL while(true){

// get next eventControlEvent ev = pendel.GetNext();if(ev.state == 1) // pendulum enters light barrier{

// switch magnet onpendel.SendEvent(new ControlEvent(ev.timestamp+1000,0,1));// switch it off after 5/23 secondspendel.SendEvent(new ControlEvent(ev.timestamp+6000,0,0));

}}

Public Class UserControl

{

Public Static void Main(String[] args)

{

}

}

Configuration Code

Control Code

generate user control class

user control code

compile

control

Pendulum : Increase of amplitude

1. Get current oscillation time Toz ( time for a full oscillation )

2. Get current time the ball is covers the light barrier Ts ( indicates speed )

3. If ball enters light barrier :1. Queue Enable Magnet Event at Tcurrent+Toz/42. Queue Disable Mag. Event at :

Tcurrent + Toz/2 + Ts/2 – Tmf1. Tmf – duration of magnetic field shut down

4. Repeat steps 1-4

Code Access Security-CAS in .NET

Supplements operating system security mechanisms Trust to assemblies vary depending of origin and publisherBased on evidences of assemblies

– Application directory– Strong name– URL– Signature

Permissions restrict access to system resourcesDCL: Configuration of usable class library classes

Code Access Permissions

Represent access to a protected resource or the ability to perform a protected operationEnvironment VariablesFile DialogFile I/OIsolated Storage FileReflectionRegistry SecurityUser InterfaceDNS

PrintingEvent LogSocket AccessWeb AccessPerformance CounterDirectory ServicesMessage QueueService ControllerSQL Client

Evidence Based Security

Assembly evidence is matched against code groups to gain permissionCode group consist of – Membership condition– Set of code access permissions

Security policies contain code groups that map assemblies to permissionsPolicies in .Net : Enterprise, Machine, User, Application DomainPolicy Evaluation: top-down

.NET Security in the Pendulum Experiment

Code group in machine policy applies only to usercode.exe assembly

– Custom Permission Set assign to the code group

No permissions, except:– Execution permission– Access to driver component via .NET Remoting

Localhost, one specified socket, one URI– File I/O in the local directory

Security exception if any restriction is violated– Check required permissions before code execution– .NET Attributes

Pendulum ExperimentDynamic Properties

Code Access Security provides safe environment for execution of foreign codeThere are still dynamic properties at the experiment that can not be handled by CAS

– Expiration of available execution time (fairness)– Expiration of available energy– Overheating of the coil– the pendulum falls below a critical amplitude (energy) - so it

could not be kept swinging– Unexpected termination of user programs– Exaggerated use of memory

Our Approach : Dynamic Reconfiguration as safe-guard mechanism

Mapping of profiles to application configurations based on environmental conditions and component statesSelection of application configuration according to conditions provides best service for a given situationDefinition of

– observer : monitoring of environmental settings and component states

– profiles : mapping of environmental conditions to application configurations

– configurations of component-based applicationsOnline monitoring of environment and componentsChange of application configuration using dynamic reconfiguration if required (changed conditions)

Configuration Creator Tool

Pendulum Experiment Control Configurations

USB-Driver

Event Queuing

SafetyController

UserProgram

Event Duplicator

UserProgram

Configuration 1 : safety controller

Configuration 2 : user program (cold standby)

Configuration 3 : user program (warm standby)

USB

SafetyController

SafetyController

Measurements:Abnormal Termination of User Program

„Higher Striker“ – „Hau den Lukas“

Similar hardware to pendulum experiment– Parallel I/O / 38 kHz sample rate / 256 Byte buffer

Use of Real Time OS – Smaller Buffers, Higher Sampling Frequency – Short control delay– COTS x86 PC

Intel Celeron 633 MHz, 128 MB RAM (max 64 MB usable)10 Mbit/s LAN (NE 2000 PCI)

Combination of non-RT .Net and RT applicationCE-PC Windows Ce.Net 4.2

“Higher Striker” - Architecture

A/D TransformationFIFO-Memory

Optical Signal Gen.

CE-PCPlattformBuilder/

Debug Host

.Net DCLController

PC

OS Image

DebuggingParallel Port I/O

Download COMPILED User ControlAlgorithms

User Code

light barriers

Switch

Power

DCL

„Higher Striker“Minimal Program

INITIALIZE;do{READ();WRITE(buffer);GETSTATUS()if(EMPTY_FLAG_LS) Sleep(1);

}while(!EOF(buffer))

Controlling the Experiment

Abstract from Direct Hardware Programming Provide Higher Level API to the experiment2 possible control scenarios :– Calculation before runtime, analysis of result after

experiment execution for next run– Calculation of control signals during runtime – hard

deadlinesInvestigation of various operating systems.Net Compact Framework and real time

„Higher Striker“ - Event List - API

Definition before runtimeTransformation into byte stream before runtimeSimple checks possible (temperature of magnet)Generation of event list after runtime for analysis for next experiment runsNot flexible – but simple to implement

0;1;10;0;20;2;25;0;50;4;80;0;

11111111110000000000222220000000000000000000000000000088888888888888888888888888880000000000000

Higher Striker Control APIint HDLInitialize ( );int HDLInitialize ( unsigned char Writeahead,

unsigned char Initial[]);

int HDLStartClock( );int HDLStopClock( );

int HDLPerform( unsigned char * Write, unsigned char * Read );

int HDLGetStatus();int HDLGetError( int * LSError, int * MSError );

Higher Striker – Watch Dog

Init Active Stopsignal

not signaled for 5 reboots

if not signaled for 5 sec reboot

70 secpower on

power off

Hardware Watchdog connected / signaled via serial communication interface of control PC

In case of system hang-up control PC will be rebooted

Atmel AVR 8-Bit Tiny 12 microcontroller

Higher Striker Simulation

Used Buffer (0-255 Byte)

Cylinder Speed

Height

VideoAnimation

DCL - Grid Integration

Heterogeneous – X86,Itanium, PowerPC– Windows 2000/XP, Linux,

Mac OS X

DRMAA – Job Submission and Control for Clusters and GridsGLOBUSIDLE-Time

– Condor– Sun Grid Engine, Condor

Grid

Grid

- Increased Throughput

- Increased Response Time

Povray

Simulation

Fischertechnik Production Line

Control via Fischertechnik Intelligent Interface– Serial communication interface, 9600 Bits/sec– 4+4 Digital Outputs, 8+8 Digital Inputs, 2 Analog

Inputs– Installation:

4 Intelligent InterfacesPneumatic Actuators 1 robot arm ( 2 free degrees )

Controlling Fischertechnik using Beckhoff Industrial-PCs and the DCL

DCL

.NET Controller

IIF

ADS.NET Visual Studio Plug-In

TwinCat PLC-Control

Märklin Railway

Märklin Digital Model RailwayControl via serial interface 9600 Bit/secS-88 Bus and controllerConfigurable train speedFeedback via rail sensorsAbout 20 switchesControlled by x86 PCCurrently running W2K

Running .NET on devices

Why ?– Rapid software development for embedded devices– Object-oriented programming model, type safety– Many available tools, know-how, many experts– Code access security for remote lab experiments

How ? Possibilities:– PC-based execution – Communication to device for I/O– Native execution of translated IL-Code– Interpretation of stripped IL-Code on the device– Running .NET on more powerful embedded

devices to control small embedded devices

CLI2RCX- .NET runtime for Lego Mindstorm

Our gcc (Gnu Compiler Collection) frontend supports the fullECMA-335 standard and can parse any conformant .NET assembly. Port for Renesas/Hitachi H8-300 microcontroller underway

– Extremely small footprint (32 KB memory)– Variable and method definitions, method calls, integer ops up and

running

ildasm.NetAssemblyC#

VB .Net

C++ .Net

RCXbinary(srec -format)

targetbinaries

GCC

CILFront End RTL

HitachiH8-300

Back End

Backends

mnemonicCIL

Assembly

HitachiH8300

Backend

Mirco.NET - .NET Interpreter for Lego Mindstorm

• Application assemblies merged and striped (metadata, portable executable header, references)

• Application merged to one „micro“ assembly and transferred to device embedded device

• Interpreter reads “micro” executions according instructions using a separate runtime

MicroIL

Assembly

ApplicationAssembly Dependencies

Base ClassLibrary

DependenciesDependency

MicroIL

Assembly

Runtime

Interpreter

OperatingSystem

Hardware

Embeddded Device Host

stripmergedownload

CLI2RCX- .NET – Current State

Version 1.0 release implements the following features of the .NET platform:– primitive datatypes: bool, byte, short, int– classes, including instance attributes and properties.– static and instance methods, including parameters,

local variables, and constructors.– arithmethic operations– control flow operations: conditional and

unconditional branch instructions.

CLI2RCX- .NET – Current State

The following features are not yet supported (as are most features not mentioned in either list):

– most value types (enums, structs, delegates, floats, doubles)– strings– single-dimensional zero-based arrays (partially complete)– multi-dimensional or non-zero-based arrays– Inheritance, polymorphism, and late binding– interfaces– exceptions– static class attributes, class constructors, events– boxing and unboxing– arithmetic instructions that detect overflows– any predefined class except for System.Object

Related Work

“Verbund Virtuelles Labor” project at University Reutlingen / GermanyiLab project (WebLab) at MIT Virtual Lab at University of Hagen /GermanyTele-Laboratory at University of PisaTele-Lab / Simplex architecture “Ein ferngesteuertes Labor im Internet”www.remote-lab.de