1

Cooperative Computing for Distributed Embedded Systems*

Cristian Borcea, Deepa Iyer, Porlin Kang, Akhilesh Saxena, and Liviu Iftode

Division of Computer and Information SciencesRutgers University

* Work supported in part by the NSF under the ITR grant ANI-0121416

2

Programming Challenge

How to program ? A single computer

Parallel computers

Networks of computers

Next challenge: Networks of Embedded Systems (NES)

How to program NES to perform distributed tasks ?

3

Networks of Embedded Systems (NES)

Large scale, ad hoc networks Heterogeneous node functionality: sensors,

intelligent cameras, smart appliances Limited resources: CPU, memory, bandwidth, energy Wireless Communication: 802.11, Bluetooth Volatile, possibly mobile

WildlifeMonitoring

IntelligentHighways

CollaborativeRobots

SensorNetworks

4

Sensor Networks

Programmability issues: How to add a new application ? How to execute user-defined applications ?

Our goal: a general programming model for NES

Data dissemination Data collection

5

Traditional Distributed Computing Does Not Work for NES

Traditional distributed computing Stable configuration of functionally homogeneous nodes

Fixed-address naming and routing (e.g., IP)

Failures are exceptions

Networks of Embedded Systems Ad hoc configurations of nodes with different properties

Content-based naming and routing

Volatile nodes are common

6

Outline

Motivation

Cooperative Computing

Smart Messages (SM)

Tag Space

Node Architecture & SM Lifecycle

API & Examples

Prototype & Simulation Results

Conclusions

7

Execution Migration Example

Data migration

Bob’s dinner:

AppetizerEntreeDessert

Execution migration

8

Cooperative Computing Distributed computing over NES based on execution

migration Smart Messages (SM)

User-defined applications which migrate through the network Execute at each node in the path

• Application executes at nodes of interest

• Migration occurs between nodes of interest

Tag Space Name-based node memory Persistent across SM executions Used by SM for addressing, storage, synchronization, I/O

access

9

Smart Messages (SM)

Code Bricks: application & routing code Data Bricks: mobile data carried by SM during migration Execution State: used for execution migration Signature: access control to Tag Space Resource Table: tags to be accessed, memory, CPU cycles,

and generated traffic estimates

CodeBricks

DataBricks

ExecutionState

SignatureResource

Table

10

Tag Space

Tag structure

Application tags

Created by SMs

I/O tags

Provide interface to OS and I/O system

Name Signature Lifetime Data

Appetizer SM_Sign 10 hours 5$

Neighbors Node1,Node2

11

Node Architecture

AdmissionManager

VirtualMachine

Tag Space

OS & I/O

SM Arrival SM Migration

12

Admission Control

At arrival each SM presents its resource requirements

Admission manager decides whether or not to accept

an SM

Each accepted SM becomes a task ready for

scheduling

Optimization: code bricks are cached upon SM

acceptance

13

Execution

Takes place over a virtual machine (VM)

Non-preemptive, but time bounded

Interrupted by SM admission

May block on tags to be updated

Terminate or continue on another node (SM

migration)

14

Migration

Appetizer $5 Entree $10

migrate_SM is a user-level library call Implements content-based routing using

A one-hop migration primitive: sys_migrate Captures execution state, transfers SM one hop, resumes execution

Routing tags

eat(Appetizer);migrate_SM(Entree);eat(Entree);

Networkeat(Entree);

eat(Appetizer);migrate_SM(Entree);

15

Self Routing

1

Entree $10

2 3 4

route_to_Entree 3

migrate_SM(Entree, timeout)

sys_migrate(2) sys_migrate(3) sys_migrate(4)

route_to_Entree 2 route_to_Entree 4

Application controls routing in two ways:

Can choose among multiple migrate_SM

implementations

Can change its routing algorithm during execution

16

Cooperative SMs

if (!route_to_Entree) create_SM(DiscoverySM, Entree); block_SM(route_to_Entree);sys_migrate(next_hop);

Route_to_Entree ?

Network

DiscoverySM

DiscoverySM

if (!route_to_Entree) create_SM(DiscoverySM, Entree); block_SM(route_to_Entree);sys_migrate(next_hop);

Applications can be dynamic collections of SMs cooperating to perform a distributed task

route_to_Entree next_hop

17

Smart Messages API

Operations on Tag Space createTag(name, lifetime, data), deleteTag(name) readTag(name), writeTag(name, value)

SM Creation create_SM(code_bricks, data_bricks)

SM Spawning spawn_SM()

SM Synchronization block_SM(tag_name, timeout)

SM Migration migrate_SM(tag_names, timeout), sys_migrate(next_hop)

18

Code Example

migrate_SM(appetizer, timeout);eat(appetizer);migrate_SM(entree, timeout);eat(entree);migrate_SM(dessert, timeout);eat(dessert);migrate_SM(home, timeout);

while(!readTag(tag)) if (readTag(route_to_tag)) sys_migrate(readTag(route_to_tag)); else create_SM(DiscoverySM, tag); block_SM(route_to_tag, timeout);

Application:

migrate_SM:

//forward

do{

sys_migrate(All_Neighbors);

createTag(prev_to_tag, lifetime, prev_hop());

}while(!(readTag(tag) || readTag(route_to_tag)));

//backward

do{

sys_migrate(readTag(prev_to_tag));

createTag(route_to_tag, lifetime, prev_hop());

}while(readTag(prev_to_tag));

writeTag(route_to_tag, prev_hop());

DiscoverySM:

19

Evaluation goals

Expressiveness Ability to implement various user-defined distributed

applications

Performance Cost of execution migration vs. data migration

Practicality SM prototype

20

Prototype Implementation

Compaq’s iPAQs running Linux

802.11 & Bluetooth for wireless communication

Modified version of Sun’s Java KVM

21

Micro-benchmark Results

Tag Space Operation

Time (microsec)

createTag 55.8

deleteTag 30.8

readTag 25.0

writeTag 28.0

block_SM 24.6 Processor: 206 MHz Intel StrongARM Bandwidth: 11 Mbps (802.11), 1Mbps (Bluetooth) Single SM execution with one-hop discovery

SM: 829 bytes (731 code, 20 data, 78 stack) 55.2 ms with 802.11 452.8 ms with Bluetooth

22

Simulation

Event-driven simulator extended with support for SM execution Accounts for both communication and computation time

Accurate measurements for execution time by counting at the VM level the number of cycles per VM instruction

Each node is simulated by a Java thread

Thread scheduling controlled by simulator

Implemented two previously proposed applications for sensor networks using SMs: Data collection: Directed Diffusion [Estrin ’99]

Data dissemination: SPIN [Heinzelman ’99]

23

Smart Messages vs. Message Passing

Directed Diffusion SPIN

24

Related Work

Mobile Agents

Active Networks

Mobile ad hoc networking

Pervasive Computing

Sensor Networks

25

Conclusions

Propose Cooperative Computing and Smart Messages for

programming Networks of Embedded Systems (NES)

Implement and evaluate two SM distributed applications

Smart Messages offer a promising programming model

for NES

SM software distribution to be released this summer

26

Future Work

Security Distributed trust model for SM

Energy Use the simulator to design and analyze energy efficient

routing algorithms

Spatial Programming with SM We propose it as a novel paradigm for programming NES

Network resources represented as {space:tag} spatial

references

Translation from SP programs to SM programs

27

Thank you !

http://discolab.rutgers.edu/sm/

28

SM Admission Protocol

resource table and code brick IDs

IDs for code bricks not cached at destination

data bricks and missing code bricks

insert task in Ready Queue

Source Destination