anynines - building a european paas

70
Building a European Cloud Mittwoch, 16. Oktober 13

Upload: anynines

Post on 24-May-2015

109 views

Category:

Technology


4 download

DESCRIPTION

This talks explains why there should be a European Cloud and how to build it. Sharing, the foundation of every Cloud leads to the question why not share IaaS and PaaS globally? Looking at latest security news in conjunction with having a look at Safe Harbour and Patriot Act leads to the question where to draw the line between security and freedom. Building a European cloud helps to allow European customers to draw their own line. OpenStack and Cloud Foundry are suitable open source technologies to build such a cloud.

TRANSCRIPT

Page 1: Anynines - Building a European PaaS

Building a European Cloud

Mittwoch, 16. Oktober 13

Page 2: Anynines - Building a European PaaS

European Cloud?

Mittwoch, 16. Oktober 13

Page 3: Anynines - Building a European PaaS

Hungarian Cloud?

Mittwoch, 16. Oktober 13

Page 4: Anynines - Building a European PaaS

Budapest Cloud?

Mittwoch, 16. Oktober 13

Page 5: Anynines - Building a European PaaS

Your personal Cloud??

Mittwoch, 16. Oktober 13

Page 6: Anynines - Building a European PaaS

The cloud is about sharing.

Mittwoch, 16. Oktober 13

Page 7: Anynines - Building a European PaaS

Spare capacity,Virtualization,

Share spare capacityPay as you go

Mittwoch, 16. Oktober 13

Page 8: Anynines - Building a European PaaS

So why not share globally?

Mittwoch, 16. Oktober 13

Page 9: Anynines - Building a European PaaS

Privacy

Mittwoch, 16. Oktober 13

Page 10: Anynines - Building a European PaaS

Any transfer of personal data of EU citizen to a non-EU

state with a lower data privacy level compared to EU

standards is prohibited.- Directive 95/46/EC

Mittwoch, 16. Oktober 13

Page 11: Anynines - Building a European PaaS

EU Safe Harbor

Mittwoch, 16. Oktober 13

Page 12: Anynines - Building a European PaaS

• is a EU directive

• regulates the processing of personal data within the European Union

Mittwoch, 16. Oktober 13

Page 13: Anynines - Building a European PaaS

U.S. - EU Safe Harbor

Mittwoch, 16. Oktober 13

Page 14: Anynines - Building a European PaaS

• Self(!)-certification process

• = swear to the United States Department of Commerce to comply to EU privacy laws

Mittwoch, 16. Oktober 13

Page 15: Anynines - Building a European PaaS

A memo from the EU commision:

Mittwoch, 16. Oktober 13

Page 16: Anynines - Building a European PaaS

"The Safe Harbour agreement may not be so safe after all."

European CommissionMEMO/13/710 19/07/2013

http://rh.gd/1hBKIrf

Mittwoch, 16. Oktober 13

Page 17: Anynines - Building a European PaaS

Patriot Act

Mittwoch, 16. Oktober 13

Page 18: Anynines - Building a European PaaS

"Uniting (and) Strengthening America (by) Providing Appropriate Tools

Required (to) Intercept (and) Obstruct Terrorism Act of 2001."

Mittwoch, 16. Oktober 13

Page 19: Anynines - Building a European PaaS

• United States federal law

• Significantly enhanced and broadened federal government powers in the realm of 

• Electronic Surveillance 

• Anti-money laundering 

• Border Security, ...

Mittwoch, 16. Oktober 13

Page 20: Anynines - Building a European PaaS

10 Titles of the Patriot Act

Mittwoch, 16. Oktober 13

Page 21: Anynines - Building a European PaaS

• Title I: Enhancing domestic security against terrorism

• Title II: Surveillance procedures

• Title III: Anti-money-laundering to prevent terrorism

• Title IV: Border security

• Title V: Removing obstacles to investigating terrorism

• Title VI: Victims and families of victims of terrorism

• Title VII: Increased information sharing for critical infrastructure protection

• Title VIII: Terrorism criminal law

• Title IX: Improved Intelligence

• Title X: Miscellaneous          

Mittwoch, 16. Oktober 13

Page 22: Anynines - Building a European PaaS

Patriot Actbeats

Safe Harbor

Mittwoch, 16. Oktober 13

Page 23: Anynines - Building a European PaaS

Where security meets freedom

Mittwoch, 16. Oktober 13

Page 24: Anynines - Building a European PaaS

The story oflavabit.com

Mittwoch, 16. Oktober 13

Page 25: Anynines - Building a European PaaS

• Encrypted email service (*2004) by Ladar Levison

• Used by Edward Snowden

• Ordered to turn over its SSL private key

Mittwoch, 16. Oktober 13

Page 26: Anynines - Building a European PaaS

Levison's was put to the decision: shutdown or “become complicit in

crimes against the American people”.

Mittwoch, 16. Oktober 13

Page 27: Anynines - Building a European PaaS

Lavabit.com was shut down on August 8, 2013

Mittwoch, 16. Oktober 13

Page 28: Anynines - Building a European PaaS

"This experience has taught me one very important lesson: without

congressional action or a strong judicial precedent, I would strongly

recommend against anyone trusting their private data to a company with

physical ties to the United States".- Ladar Levison, Lavabit.com

Mittwoch, 16. Oktober 13

Page 29: Anynines - Building a European PaaS

• It's not about having data on European servers

• It's not about having a European company

Mittwoch, 16. Oktober 13

Page 30: Anynines - Building a European PaaS

It‘s aboutstaying completely off any US provider and don‘t tie

to the US in person or with your company.

Mittwoch, 16. Oktober 13

Page 31: Anynines - Building a European PaaS

Relying on open source software is a good choice, too.

Mittwoch, 16. Oktober 13

Page 32: Anynines - Building a European PaaS

How to build a European cloud?

Mittwoch, 16. Oktober 13

Page 33: Anynines - Building a European PaaS

Cloud Building

Mittwoch, 16. Oktober 13

Page 34: Anynines - Building a European PaaS

Cloud,a term

that has beenoverdone

Mittwoch, 16. Oktober 13

Page 35: Anynines - Building a European PaaS

IaaSPaaSSaaS

Mittwoch, 16. Oktober 13

Page 36: Anynines - Building a European PaaS

IaaSPaaSSaaS

Mittwoch, 16. Oktober 13

Page 37: Anynines - Building a European PaaS

A 2013 proposal for an open source based

Cloud

Mittwoch, 16. Oktober 13

Page 38: Anynines - Building a European PaaS

Mittwoch, 16. Oktober 13

Page 39: Anynines - Building a European PaaS

Hardware

Mittwoch, 16. Oktober 13

Page 40: Anynines - Building a European PaaS

Hardware

Infrastructure as a Service (IaaS)

Servers, Network,Storage

Mittwoch, 16. Oktober 13

Page 41: Anynines - Building a European PaaS

Hardware

Infrastructure as a Service (IaaS)

Servers, Network,Storage

PaaS (PaaS)

VMs, Network,Storage

Mittwoch, 16. Oktober 13

Page 42: Anynines - Building a European PaaS

Hardware

Infrastructure as a Service (IaaS)

Servers, Network,Storage

PaaS (PaaS)

VMs, Network,Storage

Applications

CF API (deploy, scale, services, ...)

Mittwoch, 16. Oktober 13

Page 43: Anynines - Building a European PaaS

Mittwoch, 16. Oktober 13

Page 44: Anynines - Building a European PaaS

Hardware

Mittwoch, 16. Oktober 13

Page 45: Anynines - Building a European PaaS

Hardware

OpenStack (IaaS)

Servers, Network,Storage

Mittwoch, 16. Oktober 13

Page 46: Anynines - Building a European PaaS

Hardware

OpenStack (IaaS)

Servers, Network,Storage

Cloud Foundry (PaaS)

VMs, Network,Storage

Mittwoch, 16. Oktober 13

Page 47: Anynines - Building a European PaaS

Hardware

OpenStack (IaaS)

Servers, Network,Storage

Cloud Foundry (PaaS)

VMs, Network,Storage

Applications

CF API (deploy, scale, services, ...)

Mittwoch, 16. Oktober 13

Page 48: Anynines - Building a European PaaS

OpenStack

Mittwoch, 16. Oktober 13

Page 49: Anynines - Building a European PaaS

OpenStack architecture

Mittwoch, 16. Oktober 13

Page 50: Anynines - Building a European PaaS

Key-Stone

Mittwoch, 16. Oktober 13

Page 51: Anynines - Building a European PaaS

Nova

Mittwoch, 16. Oktober 13

Page 52: Anynines - Building a European PaaS

Glance

Mittwoch, 16. Oktober 13

Page 53: Anynines - Building a European PaaS

Cinder

Mittwoch, 16. Oktober 13

Page 54: Anynines - Building a European PaaS

Swift

Mittwoch, 16. Oktober 13

Page 55: Anynines - Building a European PaaS

Neutron

Mittwoch, 16. Oktober 13

Page 56: Anynines - Building a European PaaS

OpenStack provides usan IaaS ready to deploy

Cloud Foundry.

Mittwoch, 16. Oktober 13

Page 57: Anynines - Building a European PaaS

Cloud Foundry

Mittwoch, 16. Oktober 13

Page 58: Anynines - Building a European PaaS

• CF = large distributed system

• Inner shell vs. outer shell

• Bosh = Bosh outer shell > deploy CF

Mittwoch, 16. Oktober 13

Page 59: Anynines - Building a European PaaS

SimplifiedCloud Foundry

Architecture

Mittwoch, 16. Oktober 13

Page 60: Anynines - Building a European PaaS

Service(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

Page 61: Anynines - Building a European PaaS

Cloud Controller

• Offers the CF API endpoint

• System authority for issuing commands

• Start apps

• Create service

• Binding services

Service(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

Page 62: Anynines - Building a European PaaS

DEA

• droplet = dea.staging(app_code)

• Staging = executing buildpacks

• Warden

• Starts and runs dropletsService

(e.g. MySQL)Services

(e.g. MySQL)Services

(e.g. MySQL)Services

(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

Page 63: Anynines - Building a European PaaS

Health Manager

• compares desired system state with actual system state

• sends advice to CC

• CC actsService(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

Page 64: Anynines - Building a European PaaS

Router

• knows on which DEAs your app instances are

• routes incoming requests to the right DEAs

Service(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

Page 65: Anynines - Building a European PaaS

Services

• Create service = provision

• Bind = create credentials

• Apps bind to services

• Credentials as ENV variables

Service(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

Services(e.g. MySQL)

RouterRouter

DEA

RouterHealth Manager

RouterCloud Controller

Cloud ControllerDatabase

Get desired states

Request droplet start/stop

DEADEADEADEA

Droplet / Service metadata

API request Droplet request

Droplet changenotifications

Droplet heartbeat & exit messages

Consume a service

Mittwoch, 16. Oktober 13

Page 66: Anynines - Building a European PaaS

What you get?

Mittwoch, 16. Oktober 13

Page 67: Anynines - Building a European PaaS

Mittwoch, 16. Oktober 13

Page 68: Anynines - Building a European PaaS

Questions?

Mittwoch, 16. Oktober 13

Page 69: Anynines - Building a European PaaS

Thank you!

Mittwoch, 16. Oktober 13

Page 70: Anynines - Building a European PaaS

Coderequire "fileutils"

require "find"

require "fog"

class Blobstore

  def initialize(connection_config, directory_key, cdn=nil, root_dir=nil)

    @root_dir = root_dir

    @connection_config = connection_config

    @directory_key = directory_key

    @cdn = cdn

  end

  def local?

    @connection_config[:provider].downcase == "local"

  end

  def exists?(key)

    !file(key).nil?

  end

  def download_from_blobstore(source_key, destination_path)

    FileUtils.mkdir_p(File.dirname(destination_path))

    File.open(destination_path, "w") do |file|

      (@cdn || files).get(partitioned_key(source_key)) do |*chunk|

        file.write(chunk[0])

      end

    end

  end

  def cp_r_to_blobstore(source_dir)

    Find.find(source_dir).each do |path|

      next unless File.file?(path)

      sha1 = Digest::SHA1.file(path).hexdigest

      next if exists?(sha1)

      cp_to_blobstore(path, sha1)

    end

  end

  def cp_to_blobstore(source_path, destination_key)

    File.open(source_path) do |file|

Mittwoch, 16. Oktober 13