Overview  and  update  for  the  E3CWednesday,   3  August,   2016

Matt  RutkowskiIBM  STSM,  Cloud  Open  TechnologiesEditor,   Co-­‐Chair,   Simple  Profile  WG,  

Presenting  on  behalf  of  the  TOSCA  TC:

2

▪What  is  TOSCA?

▪What  Makes  TOSCA  Unique?

▪ Key  Modeling  Concepts

§ Topology,  Composition,  Portability,  Lifecycle  (management),  Policy

▪ TOSCA’s  Growing  Eco-­System

▪ in  open  source  &  standards

▪Work  Group  Activities  &  What’s  Next

An important Open standard, that is enabling a unique Cloud eco-systemsupported by a large and growing number of international industry leaders…

TOSCA  uses  a  domain-­specific   language  (DSL)  to  define  interoperable  descriptions  of  :

• Cloud  applications,  services,  platforms,  infrastructure  and  data  components,  along  with  their  relationships,  requirements,  capabilities,  configurations and  operational  policies…

• …thereby  enabling  portability and  automated managementacross  cloud  providers  regardless  of  underlying  platform  or  infrastructure  thus  expanding  customer  choice,  improving  reliability and  time-­to-­value  while  reducing costs.  

3

Associated  Companies• TOSCA  Version  1.0  Specification  approved  as  an  OASIS  Standard

— published  Nov  2013,  XML  format

• TOSCA  Simple  Profile  v1.0  Specification  (YAML  format)  

— TC-­‐approved,   implementable;  June  2016;    

— TOSCA  Simple  Profile  v1.1  Spec  (TC-­‐approved  Public  draft,  June2016)

§ Supports  Domain-­‐Specific  Profile  Specifications:  

– Network  Function  Virtualization  (NFV)  Profile  v1.0  (Revision  3  approved)

• Government  and  Corporate  Awareness:  

– OASIS:    600+  participant  organizations.  5000+  participants  spanning  65+  countries

– TOSCA  Committee:  175+  people    45+  companies/organizations

– International  Standards  &  Research:  ISO/IEC  JTC  1  liaison,  EU  FP7,  ETSI  NFV  liaison,  etc.  

• Multi-­‐company  Interoperability  Demonstrated:  

– EuroCloud  2013,  Open  Data  Center  Alliance  2014,  OSCON   2015,  OpenStack  Summit  2016  (Indigo  DataCloud) 4

Only  contributors,   reviewers,  implementers,  users,  and  supporters  of  the  TOSCA  Standard  within  OASIS  are  listed

incorporates  both  Data and  InformationModel features  and  concepts  …

…  but  brings  unique orchestration concepts focus  in  Lifecycle  mgmt.  and  State

Information  ModelsTypically,  used  to  model  a  constrained  domain  that  can  be  described  by  a  closed  set  of  entity  

types,  properties,  relationships and  operations.

Data  ModelsTypically,  describe  the  structure  (format),  enabling  manipulation   (via  interfaces)  of  the  data  stored  in  data  management  systems  assuring   integrity.

• Topology• Composition  • Requirements  -­‐ Capabilities• State  (Nodes,  Relationships)• Lifecycle (Management)• Policy

Intent  Model  Adds:

TOSCA  is  an  Intent  Model which  is  declarative  (integration  points  for  imperative)

• Structure  • Format• interfaces

• Types,  Relationships• Properties• Operations

ü TOSCA  is  can  work  with  imperative  scripts            (e.g.,  Ansible,  Chef,  Bash,  Ant,  etc.)  

ü TOSCA  can  include  other  data  models  (e.g.,  JSON,  YANG)

•Topology• Composition

• Portability

• Lifecycle

• Policy

Tier  (Group  Type)

TOSCA  is  used  first  and  foremost  to  describe    the  topology  of  the  deployment view for  cloud  applications  and  services

7

source_resource

Node_Type_A

target_resource

Node_Type_B

Requirement

connect_relationship

ConnectsToCapability

Nodes -­‐ are  the  resources  or  components  that  will  be  materialized  or  consumed  in  the  deployment  topology

Relationshipsexpress  the  dependencies  between  the  nodes  (not  the  traffic   flow)

ü Node  templates to  describe  components  in  the  topology  structure

ü Relationship  templates to  describe  connections,  dependencies,  deployment  ordering

Requirement  -­‐ CapabilityRelationships  can  be  customized  to  match  specific  source  requirements  to  target  capabilities

GroupsCreate  Logical,  Management  or  Policy  groups  (1  or  more  nodes)

8

TOSCA  Service  Template(container)Application  Tier  

(container)

Web  Server(container)

Web  App

PHP  Module

Database  Tier  (container)

DB  Server(container)

Database

Service  Templates  provide  the  “container”  to  exchange  and  reuse   topologies:  • Reusable  models extend  investments  by  making  it  easy  to  compose  more  valuable  

and  complex  apps  from  existing  apps• Determines   dependency  boundaries   to  maximize parallelism of  deployments  • Models   (dependencies)   can  be  validated  by  automation to  ensure  application-­‐aware,  

policy-­‐aligned  configuration,  deployment  and  operational  semantics

Containm

ent

Connectivity

Example:  a  simple,  2-­‐Tier  Cloud  application  expressed  in  a  TOSCA  Service  Template

• Topology

•Composition• Portability

• Lifecycle

• Policy

Application  Tier  (container)

Database  Tier  (container)

10

Logging/Monitoring  Tier  (ELK)

nodejs

WebServer

app_server

Compute

paypal_pizzastore

WebApplication

collectd

logstashSoftwareComponent

Requirements

Container

Capabilitieslog_endpoint

logstash_server

Compute

CapabilitiesContainer

elasticsearchSoftwareComponent

RequirementsContainer

Capabilitiessearch_endpoint

elasticsearch  _server

Compute

Capabilities

kibanaSoftwareComponent

Requirements

Container

kibana_server

Compute

Capabilities

search_endpoint

ConnectsTo

HostedOn HostedOn HostedOn

ConnectsTo

mongo_dbms

DBMS

mongo_server

Compute

mongo_db

Database

rsyslog

search_endpoint

ContainerContainer

ConnectsTo

Example:  Connect  a  Logging  /  Monitoring  Service  composed  of  ElasticSearch,  LogStash  and  Kibana  (ELK)  

Enabling  the  description  of  complex,  multi-­‐tier  (hybrid)  Cloud  applications

AnalyticsService

(Topology)

Cloud  Application(Topology)

Orchestrators  can  “substitute”  for  abstract  nodes……  as  long  as  all  declared  “requirements”  are  met:

• Monitoring  Service  can  be  substituted in  Cloud Application  • Analytics  Service  can  be  substituted in  Monitoring  Service  

Abstract  nodes  in  one  TOSCA  topology  can  be  substituted  with  another  topology

MonitoringService(Abstract)

JavaApplication

WebApplication

Server

SQLDatastore

Monitoring  Service(Topology)

Collector

Logger

MonitoringFramework

AnalyticsService(Abstract)

AnalyticsEngine

HadoopService  Template  1

Service  Template  2

Service  Template  3

• Topology

• Composition

•Portability• Lifecycle

• Policy

TOSCA  Service  Template

Storage

Compute1

DB

Compute2

App

Network

ScalingPolicy

§ TOSCA’s  defines  Normative  Types for  different  domains,  for  example:  ü Application,  IaaS  Types  are  part  of  

“core”  specificationü e.g.,  Web  Server,  Database,  Compute,  

Block  Storage,  Network

§ Cloud  Application’s  declarative  modelled  from  these  normative  types  …

§ …  Can  be  understood  by  any  Cloud  Provider

unfulfilledApplication   Requirements

can  be  exportedfor  Orchestrators  to  fulfill

Templates  include  (or  reference)  all  necessary  configuration  and  Infrastructure  requirements

TOSCA  applications,  using  normative  types,  are  portable  to  different  Cloud  infrastructures

TOSCA  Meta-­‐Model Normative  Types

Nodes• Properties• Attributes  

Relationships• Properties• Attributes

Capabilities

Interfaces(Operations)

Groups

Policies

Requirements

Interfaces

compo

sed  from

based  up

on

Example:  TOSCA  applications  are  portable  to  different  Cloud  infrastructures

Application   Requirements  

TOSCAOrchestration

TOSCA  Service  Template

Storage

Compute1

DB

Compute2

App

Network

ScalingPolicy

Cloud  Provider  C

Cloud  Provider  A

Cloud  Provider  B

by  expressing  application  Requirements…

independently  from  cloud  provider  Capabilities…

&  OptimizationAutomatic  Matching

Infrastructure   Capabilities

Orchestrators  concern  themselves  dealing  with  disparate  cloud  APIs   14

Capabilities

Requirements

• Topology

• Composition

• Portability

•Lifecycle• Policy

my_resource_name

My_Resource_Type

Lifecycle.Standard

create

configure

start

stop

delete

Node  Lifecycle

Operations

Implementations (e.g.,  imperative  scripts)  can  be  bound  to  operations.

source_resource

Type_A

A

target_resource

Type_B

B

my_relationship

ConnectsTo

Lifecycle.Configure

pre_config_target

post_config_target

add_target

remove_target

pre_config_source

post_config_source

add_source

remove_source

Operations

The  Orchestrator moves  the  nodes  through  their  Lifecycle States by  executing  their  LifecycleOperations in  topological  order• Orchestrators  can  work  to  deploy  nodes  in  parallel  based  upon  node  relationships

Relationship  LifecycleNodes have  their  own  Lifecycle  

Operations which  are  invoked  in  order  to  achieve  a  target  state

Relationships also  have  their  own  Lifecycle  Operations  to  configure  or  allocate  and  de-­‐configure  or  deallocate  Node related  resources

TOSCA  models  have  a  consistent  view  of  state-­‐based  lifecycle

• Topology

• Composition

• Portability

• Lifecycle

•Policy

v1.0  includes  the  groundwork  for  Placement  (Affinity),  Scaling  and  Performance  Policies‒ Orchestrators  can  evaluate  Conditions based  on  Events that  trigger  Automatic  or  Imperative  Actions

Policies  can  be  declared  independently  and  ttached to  various  points  in  your  models1. That  can  be  attached  to  Interfaces or  specific  Operations,  2. Nodes and    3. Groups of  Nodes  

my_app_1

Compute

Capabilities

Container

...Lifecycle

create

configure

...

Policy• Type• Event, Condition• Action

my_scaling_group

backend_app

Compute

Policy• Type• Event, Condition• Action

my_database

Computeweb-­‐app

ComputePolicy• Type• Event, Condition• Action

1

2

3

Scaling

“Policies  are  non-­‐functional  Requirements  independent  of  nodes”  

• Reference  by  other  Standards

• Open  Source• OpenStack

20

Topology,  Type  &  LCM  Designhttp://alien4cloud.github.io/

alien4cloud

Service  Orchestration  &  Managementhttp://getcloudify.org/

Data/computing  platform  targeted  at  scientific  communities

http://information-­‐technology.web.cern.ch/about/projects/eu/indigo-­‐datacloud

https://wiki.openstack.org/

Heat-­‐Translator(IaaS,  App  Orchestration)Tacker(Network  Function  Orchestration)

http://ariatosca.org//

Multi-­‐Cloud  Orchestration(Amazon,  Azure,  VMware,  OpenStack)

Open  Sourced  from  Cloudify

www.seaclouds-­‐project.eu/media.htmlOpen,  Multi-­Cloud  Management

ParserDeployment  Template  Translation

https://wiki.opnfv.org/display/parser/Parser

Note:  ETSI  NFV  ack.  TOSCA  can  be  used  as  aninput  model/format

• Interoperability  (Conformance)• Goal:  Conformance   test  suite  for  v1.0;  includes   tests  for  each  section  of  Simple  Profile  v1.0  specification.

• Each  test  is  a  TOSCA  Service  Templates  with  metadata  describing  test  using  the  OASIS  Test-­‐Assertion  (TAG)  Standard• Work  underway  to  publish   in  new  GitHub repo.,  announcement   (target  ~Sept  2016)

• Container  (Clustering)• Goal:  Finish   new  Cluster  capability  definitions,   Data  Cluster  use  cases.  for  Simple   Profile  v1.2

• Instance  Model• Goal:  new  schema  for  an  Instance  Model  (reuse  existing  schema  where  possible)

• Discussing  API  potentially  enabling  capture,  export and  management of  deployed   application

• Monitoring• Goal:  Create  normative  event  types  for  basic  operational  events

• Focus  on  events  types  for  Health,  Scaling &  Performance• Support   basic  “Red-­‐Yellow-­‐Green”  and  Percentage-­‐based  monitoring  for  dashboards

• Network  Function  Virtualization  (NFV)• Expanded   Scope:  include  Software-­‐Defined  Network  (SDN)  use  cases• Goal:  Complete  v1.0  Specification,   v1.0  Public  Review  Draft  3    Published   (17  March  2016)

• Can  model  complete  ETSI  MANO  specification:  Network  Services,  Virtual  Network  Functions  (NFV)s,  Virtual  Links,  with  Forwarding  Paths,    • Orchestration  demonstrated  with  OpenStack  Tacker  Project,  multi-­‐VNF  use  cases  for  next  release

22

• TOSCA  Technical  Committee  (TC)  Public  Page  (TC  approved  updates  on  documents,  strategy,  and  more)  — https://www.oasis-­‐open.org/committees/tc_home.php?wg_abbrev=tosca

• OASIS  TOSCA  LinkedIn  Group:  (latest  news,  community  and  eco-­‐system  updates,  etc.  Join  now  to  stay  informed!)  — https://www.linkedin.com/groups/8505536

• OASIS  YouTube  Channel,  TOSCA  Playlist  — https://www.youtube.com/user/OASISopen ,http://bit.ly/1BQGGHm

• TOSCA  Simple  Profile  in  YAML  v1.0   (TC-­‐approved  Committee  Spec,  implementable;  12,  June  2016)— http://docs.oasis-­‐open.org/tosca/TOSCA-­‐Simple-­‐Profile-­‐YAML/v1.0/cs01/TOSCA-­‐Simple-­‐Profile-­‐YAML-­‐v1.0-­‐cs01.zip

• TOSCA  Simple  Profile  for  NFV  v1.0  (latest  public  review  draft,  17  March  2016)– http://docs.oasis-­‐open.org/tosca/tosca-­‐nfv/v1.0/tosca-­‐nfv-­‐v1.0.html

• Contact  the  Technical  Committee  Co-­‐Chairs  (also  part  of  TOSCA  TC  Panel):– Paul  Lipton,  paul.lipton@ca.com; John  Crandall,  jcrandal@brocade.com

• Today’s  TOSCA  TC  Panel  also  Included:  Karsten Beins,  Karsten  Beins@ts.fujitsu.com;  Lowell  Higley,  Lowell.Higley@ca.com;  Chris  Lauwers,  lauwers@ubicity.com;  Derek  Palma,  dpalma@vnomic.com;  Matt  Rutkowski,  mrutkows@us.ibm.com

Q&A

Version  1.0• Approved  Committee  Specification,  12  June  2016

• Target  Fall  2016  for  full  OASIS  Standard• http://docs.oasis-­‐open.org/tosca/TOSCA-­‐Simple-­‐Profile-­‐YAML/v1.0/cs01/TOSCA-­‐Simple-­‐Profile-­‐YAML-­‐v1.0-­‐cs01.html

Version  1.1• Approved  Public  Draft  01,  June  2016• Metadata (completed)

• now  supported   in  all  Types  (Node,   Relationship,   Capability,   Data,  etc.)• Conformance   Testing  metadata

• Group Type  (completed)• Expanded  Group   Type  to  allow  management  of  member  resources  (i.e.,  Lifecycle)  • Has  its  own  Capabilities   and  Requirements

• Policy Definition (completed)• Event-­‐Condition-­‐Action   model  • Includes  Event  Filters and  Triggers

• Workflow (completed)• Intermix  declarative  with   Imperative   (e.g.,  Ansible,  Chef,  Ant,  Bash)• Preserve   investment   in  existing  scripts  for  complex  installations  /  configurations

Version  1.2• Target  Sept.  2016,  public  draft.• Improve  import  of  Service  Templates  

• Using  template  naming  /  versions  (expressions)   to  be  used  with  Catalogs  /  Repostiories• Allow  Composition  of  Group  Type

• Provide  use  cases  using  Clusters   Differentiate   from  Abstract  Node   Types

• Cluster Type (75%  completed)• Add  support   for  Cluster  normative   type;  based  upon  new  Group   Type• Will  support   new  normative  LoadBalancer ,  Scalable and  Router Capability  Types• Data  Clusters   (e.g.,  Cassandra,  MongoDB,  etc.)  – In-­‐Progress

24

Complete  Source:  https://github.com/openstack/heat-­‐translator/blob/master/translator/toscalib/tests/data/tosca_single_server.yaml

tosca_definitions_version:  tosca_simple_yaml_1_0

description:  >Template  for  deploying  a  single  server  with  predefined  properties  and  input  parameter

topology_template:inputs:

cpus:type:  integerdescription:  Number  of  CPUs  for  the  server.constraints:-­‐ valid_values:  [  1,  2,  4,  8  ]

node_templates:my_server:type:  tosca.nodes.Computecapabilities:host:properties:num_cpus:  {  get_input:  cpus }disk_size:  10  GBmem_size:  512  MB

os:properties:architecture:  x86_64  type:  linux    distribution:  rhel    

outputs:server_address:description:  IP  address  of  server  instance.value:  {  get_attribute:  [server,  private_address]  }