S(o)OS

Realising resource-independent execution support on tera-device systems

S(O)OS IN A NUTSHELLTowards the Large-Scale OS

Presenter

04/19/20232

S(o)OSService-oriented Operating Systems

1. Background / Introduction: current problems of OS and HPC / tightly coupled systems

2. The basic idea behind S(o)OSa quick overview over the main concepts

3. Project structure & management approach

CONTENTStructure of this presentation

S(o)OSService-oriented Operating Systems

BACKGROUNDissues for large scale systems

Presenter

04/19/20234

S(o)OSService-oriented Operating Systems

• We face systems with– hundred of thousands of cores– Heterogeneous cores– Accelerators attached using wide range of technologies– Myriad of connection options

• Hypertransport/Quickpath• PCIExpress• Gigabit Ethernet• Infiniband• Lot of vendor specific connectivity

• This systems cannot easily be programmedThe potential cannot be exploitedCurrent approaches are working on the symptoms (PGAS, CUDA,

OpenCL, etc.)

HPC TRENDSBrief overview over current

Presenter

04/19/20235

S(o)OSService-oriented Operating Systems

• Focus on homogeneous resource infrastructures• Scale well wrt. processes but not wrt. processing

units• Are essentially centralistic => bottleneck, OS jitter ... Future environments will be

large-scale, heterogeneous and potentially widely distributed

Current OS architectures can not deal with this

ISSUESCurrent Operating Systems

Presenter

04/19/20236

S(o)OSService-oriented Operating Systems

COMMUNICATION

Current Operating Systems

Proc. Unit

Proc. Unit

Cach

eCa

che

Proc. Unit

Assign Process

Ope

ratin

g Sy

stem

Procedure CallContext SwitchMemory Access

Cross Core Comm.Load ProcessProcess Load

Assign ProcessProcedure CallContext SwitchMemory AccessCross Core Comm.Load ProcessProcess Load

Massive communication and management overhead

Presenter

04/19/20237

S(o)OSService-oriented Operating Systems

• Require a lot of knowledge about– The resource infrastructure– The relationship between algorithm and data– The potential distribution of the algorithms and – Its connectivity / communication– Etc.

Large scale tightly coupled systems will become a common good

Programming models must become more efficient and manageable

ISSUESDistributed Programming

S(o)OSService-oriented Operating Systems

TOWARDS S(O)OS

Revising OS Architectures

Presenter

04/19/20239

S(o)OSService-oriented Operating Systems

• Distribute operating system and code across resource infrastructure

• Rearrange running code and operating system according to– Availability of resources– Requirements of code– Linkage between data

GENERAL CONCEPT

Presenter

04/19/202310

S(o)OSService-oriented Operating Systems

OVERALL CONCEPT

Virtu

al M

emor

y SegmentedCode

deploymentresourcecapabilities

virt

ual e

xecu

tion

ProcessCode

actu

al e

xecu

tion

resourcerequirements

resourcerequirements

resourcerequirements

Phys

ical

Mem

ory

match

resourcecapabilities

resourcecapabilities

Presenter

04/19/202311

S(o)OSService-oriented Operating Systems

REAL VON NEUMANN

PU

ALU Control

In / Out

MU

Memory Control

In / Out

Data Bus

In / Out

Data Bus

Control

Memory

ALU

Presenter

04/19/202312

S(o)OSService-oriented Operating Systems

THE OS MONSTER

Presenter

04/19/202313

S(o)OSService-oriented Operating Systems

• Limited cache size• Communication is costly• Resource environment is dynamic or changes

between executions• Data consistency• Code consistency

OBSTACLES

Presenter

04/19/202314

S(o)OSService-oriented Operating Systems

• Distributed, Self-Managed Microkernels– Following the SOA / Grid principle OS functionalities are

separate “services”– “Code is where the data is” – Dynamic composition of elements according to code

segment requirements

BASIC PRINCIPLES 1

Scheduler Job Mgr

HAL I/OM

emor

yFile Management

Presenter

04/19/202315

S(o)OSService-oriented Operating Systems

• Runtime Code Behaviour Analysis – Identify code parts with strong

relationships– Primary and secondary data-

sets– OS relation– Distribute segments according

to requirements and availability

– Annotate memory with analysis results

BASIC PRINCIPLES 2

OS Modules

Code Segments

Procedure Calls etc.

Self-referencing

OS1 OS2

Annotated Virtual Memory

P1.B1

P1.B2

P2.B1

P2.B1

P1.B2P1.B1

D.B2D.B1

w:0.8

w:0.5w:0.9

w:0.7 w:0.3 w:0.6

f:0.5

f:0.2

f:0.8f:0.3

w:0.9

w:0.3

f:0.9

Address Type

&x00000000

Process 1

&x00000100&x00000200

&x00000300

&x00000400&x00000500&x00000600

Process 2&x00000700&x00000800

...

“Hacking” ApplicationsAddress Add. Inf. Type

&x00000000

accessed from: &x00000600&x00000C00&x00000F00

calls:&x0002AB00&x001D1F00

reads from: &x00011F00&x000BAE30

writes to:&x000BAE30

Process 1

&x00000100 accessed from: &x00000800

jumps to:&x00000600 ...

&x00000200&x00000300&x00000400&x00000500&x00000600

Process 2&x00000700&x00000800

...

Presenter

04/19/202317

S(o)OSService-oriented Operating Systems

• Distributed Execution Model– Whole code can be distributed, not just threads– Execution context may move between cores– Distribution may vary with infrastructure– Essential distribution information is maintained with code– Reduce communication overhead

BASIC PRINCIPLES 3

Virtual process space

Exec Env Exec Env

Presenter

04/19/202318

S(o)OSService-oriented Operating Systems

• New OS architectures / paradigms• New approaches and algorithms to deal with future

distributed execution systems • Proof-of-concept implementation of distributed

execution support tools

PROJECT GOALS

S(o)OSService-oriented Operating Systems

MANAGING S(O)OS

Notes on Project Structure

Presenter

04/19/202320

S(o)OSService-oriented Operating Systems

Two main strands:• Design Strand

Development of algorithms, architectures, reference implementations– WP2: looks at the development from a hardware perspective– WP3: examines (communication) protocols– WP4: deals with distributed execution models

• Integration & Testing StrandAligns the models, performs integrated, application related tests– WP5: OS model– WP6: Application

WORK PACKAGES

Presenter

04/19/202321

S(o)OSService-oriented Operating Systems

• HLRS, University of Stuttgart

• Instituto de Telecomunicações Aveiro

• RETIS Lab, Sant'Anna School of Advanced Studies

• CTIT, Universiteit Twente• Ecole Polytechnique

Fédérale de Lausanne• European Microsoft

Innovation Centre

PARTICIPANTS