CoreGRID Workpackage 5 Virtual Institute on Grid Information and Monitoring Services

Michał Jankowski , Paweł Wolniewicz, Jiří Denemark,

Norbert Meyer, Luděk Matyska

Virtual Environments Framework for

Virtualized Resource Access in the Grid

Cracow Grid Workshop

Krakow, September 16-18, 2006

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 2

Outline

1. Introduction

2. Virtual Accounts vs. Virtual Machines

3. Virtual Workspaces Approach

4. Architecture of the Framework

5. Accounting and Audit

6. Summary

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 3

Resource Management

• Provide controlled and secure access to Grid

resources

• Provide effective way of introducing/removing users

and granting/revoking privileges

• Provide proper isolation level between user tasks

• Accounting (economy) and audit (security)

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 4

From the user point of view…

the whole Grid should be seen as a single computer

with appropriate software, hiding all the technical

details connected with physical locations,

middleware, operating systems, etc.

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 5

Virtualization

• has a long history in computer science (virtual memory, virtual

machines, etc.)

• allows for partitioning or combining real components into virtual

entities

• abstracts from internal details of physical elements

• isolation and common interface for virtual elements

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 6

Virtual environment

• By the virtual environment we understand such

encapsulation of user jobs that will both guarantee

the limited set of privileges and also provides support

for identification of user and organization on behalf

he/she acts.

• Virtual accounts and virtual machines are examples

of different approaches to the implementation of

virtual environments.

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 7

Virtual Accounts vs. Virtual Machines

Virtual Accounts Virtual Machines

Flexibility in some extent very flexible

Job separation limited full

Accounting full limited

Audit full trusty

limited may be untrusty

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 8

Virtual Accounts vs. Virtual Machines cnt.

Virtual Accounts Virtual Machines

Administration easy difficult

SLA limited yes

integration with grid systems

easy difficult

Job checkpointing and migration

no support easy

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 9

Virtual Accounts vs. Virtual Machines cnt.

Virtual Accounts Virtual Machines

fine tune of environment (OS, software)

no yes

resource consumption insignificant small to large

purpose small clusters simple needs

many VOs, many OS es, many jobs at a time,SLA

implementations several,production quality

one?prototype

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 10

Virtual Workspaces Approach

• Mechanism for creation and management of distributed dynamic Virtual Workspaces (aka Environments) in the Grid

• Two separate implementations of Virtual Workspaces:– Virtual Accounts– Virtual Machines

• „Manual” workspace management• Jobs are submitted via any existing Grid services like Globus

GRAM• Possibility of creation of complex structures like virtual clusters• Heavy-weight deployment of VM implementation (Xen,

networking)

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 11

Virtual Workspaces and the Virtual Environment Framework

• VWs fulfill most needs

• Similar architecture and concepts

• BUT:

– VW management is not transparent for the client (broker or end user) –additional service calls, different interfaces, compatibility issues, etc.

We propose automatic management

– Problems with public address in case of VMs Proxy GRAM

– Support for accounting and audit VE logging framework and database

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 12

Architecture of the Framework

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 13

Proxy GRAM

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 14

Virtual Environment Information Subsystem

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 15

Summary

• Virtualization techniques simplify access and administration of

grid resources

• Choice of virtualization model (VM, VA) depends on

requirements

• Virtual Workspaces provide implementations of these models that

fulfills most of the needs

• The proposed framework allows for automatic VW management and

thus hiding create, destroy and set lifetime operations from the client

– Transparent for the user

– No integration with existing resource brokers needed

• Virtual Environment Information Subsystem supports accounting and

audit features

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 16

Thank You!