gcp intro-20160721

Introduction

CDNetworks inc. Product Manager / Haeseung Lee

before we start…

•Pros & Cons of Cloud Computing?

Pros of Cloud Computing• say ‘goodbye’ to costly systems

• access from innumerable options

• software expense

• cooked food

• lower traditional servers’ cost

• data centralization

• data recovery

• sharing capabilities

• cloud security

• free cloud storage

• instantly test

Cost Saving Reliability Manageability Strategic Edge = Edge Computing

Cons of Cloud Computing

• net connections

• low bandwidth

• affected quality

• security issues

• non-negotiable agreements

• cost comparison

• no hard drive

• lack of full support

• incompatiblity

• lack of insight into your network

• minimal flexibility

Downtime Security and Privacy Vulnerability to attack Limited control and flexibility Platform Dependency(aka ‘Lock in’) Cloud Computing costs

Cloud Vendors? = Our Competitors!

first runners…

Public Cloud Adoption

30©2016 RightScale, Inc. All rights reserved. This work by RightScale is licensed under a Creative Commons Attribution 4.0 International License.

This year, we looked at IaaS and PaaS for Google and Azure separately as well as combined. The

separate IaaS and PaaS numbers reveal how each service is doing on its own, while the combined

numbers show the number of unique respondents that use any Azure (IaaS or PaaS or both) or any

Google (IaaS or PaaS or both) service. Because many users leverage both IaaS and PaaS offerings

from these vendors, the combined adoption is less than the sum of the adoption numbers for each

individual service.

Whereas 17 percent are using Azure IaaS and 13 percent are using Azure PaaS, 20 percent are using

either or both Azure offerings. 7 percent are using Google App Engine and 6 percent are using

Google IaaS, while 10 percent are using either or both of Google IaaS/PaaS.

Private Cloud Adoption 2016 vs. 2015

Private Cloud Adoption

35©2016 RightScale, Inc. All rights reserved. This work by RightScale is licensed under a Creative Commons Attribution 4.0 International License.

Private cloud adoption grows across all providers.The 2016 State of the Cloud Survey reveals that adoption of private cloud is growing across

all providers. Across all sizes of organizations, 44 percent percent of respondents leverage vSphere

environments as private clouds. OpenStack and VMware vCloud Suite tied for second at 19 percent.

In its first year on the survey, bare-metal clouds are being used by 15 percent of respondents.

29 percent of respondents are experimenting with or planning to use OpenStack, and 27 percent

are planning to use Microsoft Azure Pack.

In comparison to last year, there were noticeable increases in adoption rates of every private cloud

technology. VMware vSphere/vCenter went up eleven percent (33 to 44 percent). OpenStack and

VMware vCloud Suite both increased six percent (13 to 19 percent).

MS Azure Service

Why choose Google Cloud Platform?

• enables developers to build, test and deploy applications on Google’s highly-scalable, secure, and reliable infrastructure.

• computing, storage, big data/machine learning, and application services for web, mobile, analytics, and backend solutions.

Google’s Infrastructure

. Google has been building for the past 15 years: Massive, Powerful infrastructures such as datacenter and high-speed fiber optic networks.

. Data Centers / Backbone / 70+ edge POPs in 33 Countries / edge caching platform

Cloud Regions and Zones

• Regions • Regions are specific geographical locations where you can run your

resources • are collections of zones • Regional resources are available to resources in any zones in the region

• Zones • Isolated locations within a region • Zonal resources are only available in that zone

• Multi-regional resources - below are one or more multi-regional deployment in addition to any regional deployment • Google App Engine and its feature • Google Cloud Database • Google Cloud Storage • Google BigQuery

Innovative, Customer-Friendly Pricing

• Sub-Hour billing • Sustained-use discounts

• automatically reward users who run virtual machines for over 25% of any calendar month

• Compute Engine custom machine types • pay only for the resources you need for your application

• online pricing calculator

https://cloud.google.com/products/calculator/

©Google Inc. or its affiliates. All rights reserved. Do not distribute. 12

Image by Connie Zhou

NextFully automated

NowUser managed, user

configured, user maintained

1st wave:Colocation

2nd wave:Virtualized datacenters

3rd wave:A global,

elastic cloud

The Future of Cloud Computing

Notes: The concept of cloud computing began with colocation. Instead of operating your own data center, you rented space in a colocation facility. This was the first wave of outsourcing IT. With colocation, the transfer of ownership was minimal - you still owned the machines and you maintained them. Traditionally, colocation is not thought of as cloud computing, but it did begin the process of transferring IT infrastructure out of your organization.

Today, cloud computing involves virtualized datacenters - virtual machines and APIs. Virtualization provides elasticity. You automate infrastructure procurement instead of purchasing hardware. With virtualization you still maintain the infrastructure. It is still a user-controlled/user-configured environment. This is the same as an on-premises datacenter, but now, the hardware is in a different location. Virtualization does provide a number of benefits: your development teams can move faster and you can turn capital expenses into operating expenses.

The next wave of cloud computing is a fully automated, elastic cloud. This involves a move from user-maintained infrastructure to automated services. In a fully automated environment, developers do not think about individual machines. The service automatically provisions and configures the infrastructure used to run your applications. Google is uniquely positioned to

©Google Inc. or its affiliates. All rights reserved. Do not distribute.

IaaS and PaaS

14

PaaSPreset run-times

Java, Go, PHP, Python...Focus is application logic

Pay for what you useLess management overhead

IaaSRaw compute, storage and

networkMore granular control

Pay for what you allocateMore management overhead

Towardsmanaged

infrastructure(DevOps)

Towardsmanaged services(NoOps)

Compute Engine App Engine

Notes: This slide highlights the shift in cloud computing. Virtualized datacenters brought you infrastructure as a service (IaaS) and platform as a service (PaaS) offerings. At Google, we have services that provide these options. As cloud computing has evolved, the momentum has shifted toward managed infrastructure (DevOps) and managed services (NoOps). As the course progresses, you see how the Google Cloud Platform products and services are positioned to propel you into the next wave of cloud computing - a fully automated, elastic cloud.


Google Cloud Platform

15

BigQuery

Big Data

Pub/Sub Dataflow Dataproc Datalab

Compute

Compute EngineApp Engine Container Engine

Storage

Cloud Storage

Cloud SQL Cloud Datastore

Bigtable

Machine Learning

Speech APIMachine Learning

Translate APIVision API

Notes: Google Cloud Platform’s products and services can be broadly categorized as Compute, Storage, Big Data, Machine Learning, Networking, and Operations/Tools. This slide highlights many of the services that form the core of the Google Cloud Platform. Throughout this course, you explore the Cloud Platform products and services in lectures and in the hands-on labs. For now, it is helpful to see the breadth of the Cloud Platform offerings at a glance.

For more information, see: https://cloud.google.com/products/

Google Cloud Platform Console• https://console.gloud.google.com

https://console.gloud.google.com

Projects

• All GCP services are associated with a project that is used to: • Track resources and quota ranges • Enable billing • Manage permissions and credentials • Enable services and APIs

• Projects use three identifying attributes: • Project Name • Project Number • Project ID - aka Application ID

• Interact with projects using the Cloud Console or the Cloud Resource Manager APIAlpha

https://cloud.google.com/resource-manager/reference/rpc/

Project Permissions - primitive roles

Owner Billing AdministratorViewerEditor

Invite members Remove members Can delete project includes Editor rights

Deploy applications Modify code Configure services includes Viewer rights

Read-only access Manage billing Add administrators Remove administrators

A project can have multiple owners, editors, viewers and billing administrator. - Primitive roles -above - curated roles - new IAM roles that give finer-grained access control than the primitive roles


Identity and Access Management

8

Who can do what on which resource

IAM roles - Curated Roles


IAM Roles - Curated Roles

9

IAM

✔ compute.instances.delete✔ compute.instances.get✔ compute.instances.list✔ compute.instances.setMachineType✔ compute.instances.start✔ compute.instances.stop . . .

List of Permissions

InstanceAdminRole

project_a

GoogleGroup

Notes:The “can do what” part is defined by an IAM role. An IAM role is a collection of permissions. Most of the time to do any meaningful operations you need more than 1 permission. For example to manage instances in a project, you need to create, delete, start, stop and change an instance. So the permissions are grouped together into a role to make it easier to manage.

To give a user the desired permissions, you grant a role to the user on a resource. In this example we are granting a group of users the InstanceAdmin role on project a so the user can manage instances in the project. Whenever possible, it is a best practice to use groups. You should also strictly control the ability to change policies and group memberships which will allow additional users to gain access to resources.

For a complete list of roles by product, see: https://cloud.google.com/iam/docs/#supported_cloud_platform_services

Service Accounts and IAM

• Service Accounts authentication with keys • google manage keys, key rotation for

Compute engine and App engine

• Can assign an IAM role to the service account

• Can also assign serviceAccountActor role to users/groups


Service Accounts and IAM

14

● Service accounts authenticate with keys

○ Google manages keys, key rotation for Compute Engine and App Engine

● Can assign an IAM role to the service account

● Can also assign ServiceAccountActor role to users/groups

Service Account InstanceAdmin Role Compute Instances

Service Account

Identity IAM Role Resource

User/Group ServiceAccountActor Role

Notes:Users require a username and password to authenticate. Apps use a key. One or more keys can be generated for each IAM service account. Keys are sensitive and need to be carefully managed because they give you access to resources. When you run applications on Compute Engine or App Engine, Google manages the keys for you and automatically rotates them. You never have the risk of losing/exposing your key. When you run apps elsewhere, you can generate and download the keys to use in your code. Keep them safe and rotate them.

A service account is both an identity and a resource. A service account is used as an identity for your application to authenticate; for example, a Compute Engine VM running as a service account. To give the VM access to the necessary resources, you need to grant the relevant IAM roles to the service account. At the same time, you need to control who can create VMs with the service account so random VMs cannot assume the identity. Here, the service account is the resource to be permissioned. You assign the ServiceAccountActor role to the users you trust to use the service account.

Google App Engine and Google Cloud Datastore



NetworkingCompute Big Data Machine Learning Storage Operations and

Tools

Compute Engine

Container Engine

App Engine

Notes: App Engine is one of several Google Cloud Platform compute options for running your applications.

What is Google App Engine

• A platform(platform as a service) for building scalable web applications and mobile backends

• App Engine makes deployment, maintenance, and scalability easy so you can focus on innovation


700 million

“App Engine enabled us to focus on developing the application. We wouldn’t have gotten here without the ease of development that App Engine gave us.”Bobby Murphy, CTO

Snapchat sends

photos and videos each day Google App Enginescaled seamlesslyduring growth to

millions of users

Small team is ableto innovate quickly

and expandglobally

Snapchat

Notes:● Today, Snapchat’s photo messaging app is among the top 10 most

downloaded mobile apps of any type for both Android and iOS.● They launched in 2011, and just two years later, had tens of millions of

users across the world sending hundreds of millions of photos and videos each day.

● Before the era of Cloud computing, supporting this level of growth with a developer and operations team of just a few people would have been impossible.

● With Google App Engine, Snapchat doesn’t have to worry about managing the underlying infrastructure. Cloud Platform allows their app to scale with minimal effort from their team.

● But it’s not just about scale. App Engine also allows Snapchat to quickly release new features or versions of the application - a difficult task when your app is constantly in use by millions of people.

Read more about Snapchat here: https://cloudplatform.googleblog.com/2016/03/Snapchat-shares-security-best-practices-for-running-on-GCP-practices.html.

App Engine - Best Practice

App Engine Standard Environment

• Managed runtimes for specific versions of Java, Python, PHP & Go

• Autoscale workloads to meet demand • Free daily quota, usage based pricing • SDKs for development, testing and deployment • Need to conform to sandbox constraints:

• no writing to local file system

• request timeouts at 60 seconds

• Limit on 3rd-party software installations


Example App Engine Standard Workflow - Web Applications

10

Project

App EngineApp Servers

Application instances



App Engine can access a variety of services using dedicated APIs1 Develop & test the web

application locally

2Use the SDK to deploy to App Engine

3App Engine automatically scales & reliably serves your web application

Task queues

Scheduled tasks

Search

Memcache

Logs

Notes:There are many services available to your applications running in the App Engine Standard environment. For the most up-to-date information on App Engine services, see: https://cloud.google.com/appengine/docs/about-the-standard-environment#index_of_features.

Below are details on some of the more commonly used services available to your App Engine Standard applications.

Users APIApp Engine Standard environment applications can authenticate users using Google Accounts or accounts on your own Google Apps domains. An application can detect whether the current user has signed in, and can redirect the user to the appropriate sign-in page to sign in or, if your app uses Google Accounts authentication, create a new account. While a user is signed in to the application, the app can access the user's email address. The app can also detect whether the current user is an administrator, making it easy to implement admin-only areas of the app.

Modules APIModules are used to factor large applications into logical components that can share stateful services and communicate in a secure fashion. An app that

App Engine Flexible EnvironmentBeta

• Build, deploy containerized apps with a click • Standard runtimes - Python, Java, Go, Node.js

- with no sandbox constraints • Custom runtime support for any languages

that support HTTP requests • During beta pricing based on Compute Engine

usage • Local development relies on Docker • Standard runtimes can access App Engine

services: Datastore, Memcache, task queues, logging, users, and so on.


App Engine Standard vs Flexible Environment

Standard Environment Flexible Environment

Instance startup Milliseconds Minutes

SSH access No Yes (not default)

Scaling Manual, basic, automatic Manual, automatic

Write to local disk No Yes (ephemeral)

Support for 3rd party binaries

No Yes

Network access Via App Engine services Yes

Customizable stack No Yes

18

Notes: You can run an App Engine application in two environments, the standard environment and the flexible environment. You can use both environments in your application at the same time if you structure your application using the microservices architecture.

Flexible environment versus Compute Engine

While flexible environment runs modules in Compute Engine VM instances, it differs from Compute Engine in the following ways:● Flexible environment VM instances are restarted on a weekly basis.

During restarts Google's management services will apply any necessary operating system and security updates.

● You always have root access to Compute Engine VM instances. SSH access to VM instances in the flexible environment is disabled by default. If you choose, you can enable root access to your app's VM instances.

● Flexible environment VM instances are automatically located by geographical region according to the settings in your project. Google’s management services will ensure that all the VM instances for a project are co-located for optimal performance.

Google Cloud Endpoints

• Build your own API running on App Engine Standard

• Expose your API using a RESTful interface • Includes support for OAuth 2.0

authorization • Generate client libraries • Support Java and Python server-side

code • Includes App Engine features

• scaling

• denial of service protection

• high availability

• Support iOS, Android, and JavaScript Clients



Compute Machine Learning Storage Operations and

Tools

Cloud Storage

Cloud SQL

CloudDatastore

CloudBigtable

Networking Big Data

Notes: Cloud Datastore is a highly-scalable NoSQL database for your applications. Cloud Datastore automatically handles sharding and replication, providing you with a highly available and durable database that scales automatically to handle your applications' load. Cloud Datastore provides a myriad of capabilities such as ACID transactions, SQL-like queries, indexes and much more.

Google Cloud Datastore

• Database designed for application backends

• NoSQL store for billions of rows • Schemaless access, no need to think

about underlying data structure • Local development tools • Automatic scaling and fully managed • Built-in redundancy • Supports ACID transactions • Includes a free daily quotas • Access from anywhere through a

RESTful interface




Tools

Cloud Storage

Cloud SQL

CloudDatastore

CloudBigtable

Networking Big Data

Notes:Google Cloud Platform has many storage options that satisfy nearly every customer use case. Cloud Datastore was discussed previously. In this module, we turn our attention to the remaining core storage options: Google Cloud Storage, Google Cloud SQL, and Google Cloud Bigtable. First, is Cloud Storage.

Google Cloud Storage

• High performance, internet-scale, immutable BLOB(binary large object) storage

• Not a file system(but can be accessed as one via 3rd-party tools such as Cloud Storage Fuse)

• Simple administration and does not require capacity mgmt

• Data encryption in-flight and at rest

• All storage classes accessed through the same APIs

Cloud Storage Classes


Cloud Storage Classes

6

Standard Storage

DRA Storage

Nearline Storage

Standard Storage provides the highest durability, availability and performance with low latency and is ideal for use with website content distribution and video streaming

Durable Reduced Availability Storage offers the same durability as Standard Storage but with a lower availability SLA at a reduced cost

Nearline Storageoffers low-cost, highly durable storage service for data archiving, online backup, and disaster recovery, without having to wait hours or days to retrieve or access your data

Each option comes with detailed pricing

Notes: Google Cloud Storage offers three types of storage: Standard Storage, Durable Reduced Availability (DRA) Storage, and Nearline Storage with different availability, latency, and throughput characteristics. All storage classes offer the same high level of durability.

Cloud Storage Integration


Cloud Storage Integration

11

Import and exporttables

Object storage, logs, Datastore backups

Startup scripts, images and general object storage

Import and export tables

App Engine

Compute Engine

Cloud Storage

Cloud SQL

BigQuery

Notes:Google Cloud Storage is tightly integrated with many of the Google Cloud Platform products and services. Cloud Storage is often the ingestion point for data being moved into the cloud, and is frequently the long-term storage location for data.

Integration between Cloud Storage and other services includes using Cloud Storage to:● Store tables imported into BigQuery and to store tables exported from

BigQuery● Store App Engine logs, Cloud Datastore backups, and objects used by

App Engine applications (such as images)● Store instance startup scripts, Compute Engine images, and objects

used by Compute Engine applications● Store tables imported into Cloud SQL and to store tables exported from

Cloud SQL

For more information on Cloud Storage integration with other Google Cloud Platform services, see: https://cloud.google.com/storage/docs/google-integration.

DEMO 1. Getting Started with GCP 2. Deploying Applications using Google App Engine and Cloud Datastore

Google Cloud SQL

• Google-managed MySQL relational database in the cloud

• Pay-per-Use model • RESTful API for management • Affordability and performance • Good Security • Vertical scaling(read and write) • Horizontal scaling(read) • Seamless integration with App

Engine and Compute Engine


Managed backups

Google security

Automatic replication

Familiar Flexible pricing

Cloud SQL Features

Notes:FamiliarBuild and deploy for the cloud faster because Cloud SQL offers standard MySQL databases. Use standard connection drivers and built-in migration tools to get started quickly. Cloud SQL supports most MySQL statements and functions:● Stored procedures● Triggers● Views

Cloud SQL does not support:● User-defined functions● MySQL-esque replication● Statements and functions related to files and plugins

Flexible pricingFor First Generation instances, Google offers two billing plans: Packages and Per Use. Which plan you should select depends on how your database will be used, but generally it is more economical to use the Package plan if your instance is used for more than 450 hours each month. If your instance is used sporadically, you can reduce costs by using the Per-use plan and setting your activation policy to ON DEMAND. Learn more.


Cloud SQL Integration

Cloud SQL can be used with App Engine using standard drivers like Connector/J for Java or MySQLdb for Python.

App Engine applications are authorized to access Cloud SQL, and the instance can be configured to follow one application.

Compute Engine instances can be authorized to access Cloud SQL instances using an external IP address.

Cloud SQL instances can be configured with a preferred zone - to stay close to the Compute Engine infrastructure.

Cloud SQL can be used with external applications and clients by authorizing IP addresses or networks using CIDR notation.

Standard tools like MySQL Workbench can be used to administer databases. External read replicas can be configured.

Externalservice

Google Cloud SQL second generationBeta

• Same features as first generation with higher performance, storage capacity at lower cost • up to 7x throughput and 20x storage

capacity of first generation instances

• less expensive than first generation for most use cases


Comparing Storage Options (1 of 2)

NoSQLBLOB SQL

Cloud Storage

Cloud Bigtable

Cloud Datastore

Cloud SQL

Good for:Structured and unstructured binary or object data

Good for:Getting started, App Engine applications

Good for:“Flat” data, Heavy read/write, events, analytical data

Good for:Web frameworks,existing applications

Use cases:Images, large media files, backups

Use cases:User profiles,product catalog

Use cases:AdTech, Financial and IoT data

Use cases:User credentials, customer orders

Notes:Google Cloud Platform delivers various storage service offerings which remove much of the burden of building and managing storage and infrastructure. Like our other cloud services, cloud storage will free you to focus on doing what you do best and differentiating at the application or service layer.

Our storage offerings range from SQL, NoSQL, Blob and Block storage depending on what you are trying to do, and it's easy to mix and match. ● Cloud SQL gives you fully managed MySQL so you have relational DB

and a more traditional approach to queries. ● Cloud Datastore provides a nearly infinitely scalable, schemaless

solution. ● If you want a disk you can mount Persistent Disk as a block store that

can be used by Compute Engine.● For just pure data and blobs, Cloud Storage can deliver what you need. ● Google Cloud Bigtable offers companies a fast, fully managed, infinitely

scalable NoSQL database service ideal for web, mobile and IoT applications.


Comparing Storage Options (2 of 2)

33

Cloud Datastore Cloud Storage Cloud SQL (1st Generation)

Bigtable

Storage type NoSQL, document

Object (BLOB) store

Relational SQL NoSQL, wide-column

Overall capacity Terabytes + Petabytes + up to 500 GB Petabytes +

Unit size 1 megabyte / entity

5 TB / object Standard MySQL limits

Recommended - Individual values: ~10 MB per cellAll values per row: ~100 MB

Transactions Yes No Yes No

Complex queries No No Yes No

Notes:● Cloud Bigtable is not a relational database; it does not support SQL

queries or joins, nor does it support multi-row transactions. Also, it is not a good solution for small amounts of data (< 1 TB).

● If you need full SQL support for an online transaction processing (OLTP) system, consider Google Cloud SQL.

● If you need interactive querying in an online analytical processing (OLAP) system, consider Google BigQuery.

● If you need to store immutable blobs larger than 10 MB, such as large images or movies, consider Google Cloud Storage.

● If you need to store highly structured objects, or if you require support for ACID transactions and SQL-like queries, consider Cloud Datastore.

Google Container Engine Google Compute Engine

What is a Container?

• Virtualization at the operating system layer

• Separates operating system from application code and dependencies

• Isolated individual processes

• Popular implementations include Docker and rkt

Why use Containers?

• Support consistency across development, testing, and production environments

• Loose coupling between application and operating system layers

• Much simpler to migrate workloads between on-premises and cloud environments

• Support agile development and operations ©Google Inc. or its affiliates. All rights reserved. Do not distribute.

● Virtualization at the operating system layer

● Separates operating system from application code and dependencies

● Isolates individual processes

● Popular implementations include Docker and rkt

3

What is a Container?

Operating system

Shared libraries

ContainerCode and libraries



Virtual Machine

Kubernetes(aka ‘k8s’)

• Open Source container cluster orchestration system • automates deployment,

scaling, and operations for container clusters

• Based on Google’s experience over 10+ years

• Built for a multi-cloud world: • public, private, hybrid

Features of Kubernetes

• Workload portability • run in many environments, across cloud providers

• implementation is open and modular

• Rolling updates • upgrade application with zero downtime

• Autoscaling • automatically adapt to changes in workload

• Persistent storage • abstracts details of how storage is provided from how it is consumed

• Multi-Zone clusters • run a single cluster in multiple zones

• Alpha on Google Cloud Platform

• Load balancing • external IP address routed traffic to correct port

Competitors of k8s




Tools

Compute Engine

Container Engine

App Engine

Notes: Container Engine is one of several Google Cloud Platform compute options for running your applications.


IaaS and PaaS

14

PaaSPreset run-times

Java, Go, PHP, Python...Focus is application logic

Pay for what you useLess management overhead

IaaSRaw compute, storage and

networkMore granular control

Pay for what you allocateMore management overhead

Towardsmanaged

infrastructure(DevOps)

Towardsmanaged services(NoOps)

Compute Engine App Engine

Notes: This slide highlights the shift in cloud computing. Virtualized datacenters brought you infrastructure as a service (IaaS) and platform as a service (PaaS) offerings. At Google, we have services that provide these options. As cloud computing has evolved, the momentum has shifted toward managed infrastructure (DevOps) and managed services (NoOps). As the course progresses, you see how the Google Cloud Platform products and services are positioned to propel you into the next wave of cloud computing - a fully automated, elastic cloud.

Google Container Engine(1/2)

• Fully managed cluster management and orchestration system for running containers • based on Kubernetes

• uses Compute Engine instances and resources

• Complimentary services: • Google Cloud Container Builder Beta -

Create Docker container images from app code in Google Cloud Storage

• Google Container Registry - Secure, Private Docker image storage

Google Container Engine(2/2)

• Uses a declarative syntax to manage applications • declare desired application configuration,

Container Engine implements, manages

• Decouples operational, development concerns

• Manage and maintains • logging, health management, monitoring

• Easily update Kubernetes versions as they are released

Container Engine=k8s :: concepts

- Nodes: node is working machine in a k8s cluster, and in Google Container engine, the machine is always a Computer Engine Instance.

- Pods: Pod is a group of one or more containers, shared storage, and configuration data relating to those containers. It is common for production applications running in k8s to include multiple, relatively tightly-coupled containers in a single pod.

- Replication Controllers: Replication Controller works to ensure that the requested number of pod replicas are always available and running at a given time. automatically adds or remove pods as required to maintain a desired state.

- Services: defines a logical set of pods and a way to access them using an IP address and port number pair.


Comparing Compute Options

9

Compute Engine

Container Engine

App Engine Standard

App Engine Flexible

Language support

Any Any Java, Python, Go & PHP

Any

Service model IaaS Hybrid PaaS PaaS

Primary use case

General computing workloads

Container-based workloads

Web and mobile applications

Web and mobile applications, container-based workloads




Tools

Compute Engine

Container Engine

App Engine

Notes: Compute Engine is one of several Google Cloud Platform compute options for running your applications.

Google Compute Engine(1/2)

• Run large-scale workloads on virtual machines hosted on Google’s infrastructure

• Robust networking features • default, custom networks

• firewall rules

• regional HTTP(S) load balancing

• network load balancing

• subnetworks

• High CPU, high memory, standard and shared-core machine types

Google Compute Engine(2/2)

• Persistent disks • standard, SSD, local SSD

• snapshots

• Resize disks, migrate instances with no downtime

• Instance metadata and startup scripts

• Advanced APIs for auto-scaling and instance group management

• Innovative pricing • per-minute billing, sustained use discounts • preemptive instances

• high throughput to storage at no extra cost

• custom mashine types - only pay for the hardware you need

DEMO 1. Deploying Applications using Google Container Engine 2. Deploying Applications using Google Compute Engine

Google Cloud Networking Google Cloud Operations and Tools Google Big Data Google Machine Learning(“TensorFlow”) Amazon Web Services case Messaging system (such as ‘Kafka’, etc.) Object Storage, File system introduction …

goes to next part.

References

. Google Cloud Platform Fundamentals: https://sites.google.com/a/google.com/cloud-platform-training/cloud-platform-training/cp100-v2 . Google Developers - codelab: https://codelabs.developers.google.com/?cat=Cloud

https://sites.google.com/a/google.com/cloud-platform-training/cloud-platform-training/cp100-v2

https://codelabs.developers.google.com/?cat=Cloud




Tools

Cloud Storage

Cloud SQL

CloudDatastore

CloudBigtable

Networking Big Data

Notes:The next storage option on Google Cloud Platform is Google Cloud Bigtable.

Google Cloud Bigtable

• Fully managed, NoSQL, wide-column database service for large workload application - TB to PB

• Integrated • accessed using HBase API

• native compatibility with big data, Hadoop ecosystem

• Protected • replicated storage

• data encryption in-flight and at rest

• Role-based ACLs

• Proven • drives major applications such as Google

Analytics and Gmail


Bigtable Access Patterns

Cloud Bigtable

Application APIData can be read from and written to Cloud Bigtable through a data service layer like: Managed VMs, the HBase REST Server, a Java Server using the HBase client. Typically this will be to serve data to applications, dashboards and data services.

StreamingData can be streamed in (written event by event) through a variety of popular stream processing frameworks like: Cloud Dataflow Streaming, Spark Streaming, Storm.

Batch ProcessingData can be read from and written to Cloud Bigtable through batch processes like: Hadoop MapReduce, Dataflow, Spark. Often, summarized or newly calculated data is written back to Cloud Bigtable or to a downstream database.


Cloud Bigtable Integration

Cloud Bigtable

Google Cloud DataflowUse Cloud Dataflow connector for Bigtable for batch and streaming operations in pipelines

Google Cloud DataprocUse Bigtable HBase client to integrate Hadoop jobs with Cloud Dataproc

On-premises, cloud-based HadoopUse Bigtable HBase client to integrate with Hadoop clusters

Externalservice

Notes: Dataflow Connector for Cloud BigtableThe Cloud Dataflow connector for Cloud Bigtable makes it possible to use Cloud Bigtable in a Cloud Dataflow pipeline. You can use the connector for both batch and streaming operations. The connector is written in Java and is built on the HBase client for Cloud Bigtable. You can find the connector on GitHub in the repository GoogleCloudPlatform/cloud-bigtable-client.

For more information on the Cloud Dataflow connector, see the full API documentation.

Cloud Bigtable and the HBase APIOne way to communicate with Cloud Bigtable is through the Cloud Bigtable HBase client, which is a customized version of the standard Apache HBase client. The Cloud Bigtable HBase client allows you to write Java applications that communicate with Cloud Bigtable through the open-source HBase API. Source code for the client is available in the GitHub repository GoogleCloudPlatform/cloud-bigtable-client.

When you use the Cloud Bigtable HBase client, you don't need to run the HBase server. You are simply using a modified version of the HBase client interface. Google runs and maintains the servers for the Cloud Bigtable




Tools

Cloud Storage

Cloud SQL

CloudDatastore

CloudBigtable

Networking Big Data

Notes:The next storage option on Google Cloud Platform is Google Cloud SQL.

gcp intro-20160721

Technology