deploying high-performance database services amazon aurora · – pet’s ability to eat must be...

© Copyright 2015. Apps Associates LLC. 1

Deploying High-Performance Database Services

Amazon Aurora

November 5 2015

Bill Stewart Founder and CEO

PetTrax

Lynn Ferrante BDM, Databases

AWS Business Development

Kshitij Kumar CTO

Apps Associates


Agenda

• About Amazon Aurora RDS

• PetTrax’s move to Aurora

• Migration Best Practices


Amazon RDS Aurora

Lynn Ferrante BDM, Databases

AWS Business Development


Agenda

• What is Amazon RDS

• What is RDS Aurora

– Value Proposition

– Benefits

– Performance and Scalability


Amazon Aurora Summary

• MySQL compatible -existing MySQL applications can run without changes

• Fast – up to 5 times the throughput of standard MySQL

• Available and Durable--replicates 6 copies of data across 3 availability zones

• Highly Scalable -- scale up to 32 vCPUs,244GiB Memory with 15 replicas

• Highly Secure-Isolates data within VPC & encrypt data in-transit and at-rest.

• Inexpensive-- one tenth of the cost of the high end commercial databases


Fastest growing service

in AWS history

Amazon Aurora Customer Adoption


• Create a database in minutes

• Automated patching

• Push-button scale compute

• Automated continuous backups to S3

• Instantly create user snapshots

• Automatic failure detection and failover

• Read Replicas as failover targets

• Minimal replica lag

• Automatically scale up to 64 TB

• Pay only for storage you use

• 6 copies of data in 3 AZs

Amazon Aurora’s Recipe for Success


Write-only workload

1,000 connections

Query cache (default on for Amazon Aurora, off for MySQL)

-

10

20

30

40

50

60

70

10 100 1,000 10,000

Tho

usa

nd

s o

f W

rite

s p

er

Seco

nd

Number of Tables

Write Performance and Table Count

AuroraMySQL on I2.8XLMySQL on I2.8XL with RAM DiskRDS MySQL with 30,000 IOPS (Single AZ)

Up to

11x faster

Writes Scale with Table Count


OLTP Workload

Variable connection count

250 tables

Query cache (default on for Amazon Aurora, off for MySQL)

-

20

40

60

80

100

120

50 500 5,000

Tho

usa

nd

s o

f W

rite

s p

er

Seco

nd

Concurrent Connections

Write Performance and Concurrency

Aurora RDS MySQL with 30,000 IOPS (Single AZ)

Up to

8x faster

Better Concurrency


OLTP workload

1,000 connections

250 tables

Query cache on/off tested

-

50

100

150

200

250

300

350

400

100/0 50/50 0/100

Tho

usa

nd

s o

f O

pe

rati

on

s/Se

con

d

Read/Write Ratio

Performance with query cache on and off

Aurora without CachingAurora with CachingRDS MySQL;30,000 IOPS (Single AZ) - without cachingRDS MySQL;30,000 IOPS (Single AZ) - with caching

R/W Ratio

Amazon Aurora

Without Caching

Amazon Aurora With

Caching

RDS MySQL 30K IOPS Without Caching

RDS MySQL 30K IOPS

With Caching

100/0 160,000 375,000 35,000 19,000

50/50 130,000 93,000 24,000 20,000

0/100 64,000 64,000 16,000 16,000

Caching Improves Performance


Write workload

250 tables

Query cache on for Amazon Aurora, off for MySQL (best settings)

2.6 3.4 3.9 5.4

1,000 2,000 5,000 10,000

0

50,000

100,000

150,000

200,000

250,000

300,000

350,000

Updates per Second

Re

ad R

ep

lica

Lag

in M

illis

eco

nd

s

Read Replica Lag

Aurora RDS MySQL;30,000 IOPS (Single AZ)

Updates/ Second

Amazon Aurora

RDS MySQL 30K IOPS

(Single AZ)

1,000 2.62 ms 0 s

2,000 3.42 ms 1 s

5,000 3.94 ms 60 s

10,000 5.38 ms 300 s

Replicas have up to 400 times less Lag


• 4 client machines with 1,000 threads each

WRITE PERFORMANCE READ PERFORMANCE

• Single client with 1,000 threads

• MySQL SysBench

• R3.8XL with 32 cores and 244 GB RAM

SQL Benchmark Results


Connecting Pets and Families

Bill Stewart (617) 875-7360 [email protected] www.pettraxinc.com

mailto:[email protected]

http://www.pettraxinc.com/


Agenda

• Overview

• Problem

• Solution

• The AWS Advantage

• Apps Associates Partnership

• Beta Solution

• Summary


Ricky – The Gobbler Teddy – The Nibbler

Overview Our Story


Problem Many overweight pets, many health issues


Problem Pet families lead busy lives

• Out of the home much of the day

– Difficult to keep track of diet and exercise

• Multiple caregivers in the home

– People err on side of caution and end up overfeeding

• Need to work late tonight

– Leave out a big bowl of food

• Pet seems to be OK, no need to go in for check-up

– Only going to vet when health issue is advanced

• Multi-pet households

– Problem grows exponentially for each additional pet in home


Solution Intelligent feeding, monitoring, and notifications

• Intelligent feeding

– Pet’s ability to eat must be authorized and automated

– Authorization must evaluate multiple criteria

• Food type allowed

• Times during the day allowed

• Daily & feeding-session calorie allowance

• Intelligent monitoring

– Track all pet interactions

• Feeding/drinking sessions: duration, time of day, calories

• Feeding attempts: at own feeder and other pet’s feeder

• Level of activity: sleeping, moving

• Intelligent notifications

– Ricky just finished eating. 45 calories consumed; all normal

– Teddy has not eaten in 10 hours. This is not normal; check on Teddy

– Health concern! Bailey’s activity pattern has deviated significantly over the past 8 hours; check on Bailey


Solution Intelligent Feeder – Alpha Version

RFID Reader for pet identification

Motion Sensor

Food Storage and Dispensing

Camera

Retractable Food Tray

Temp/Humidity Sensor

Processor, Memory, I/O, and Comm


Solution Customized for any pet household

• Single cat or single dog homes

• Multi-cat or multi-dog homes

– Same food (same feeder)

– Different food (multiple feeders)

• Mixed cat and dog homes

– Multiple feeders


Solution What’s missing?

Connected Feeder

Scalable Compute Resources

Mobile Application


The AWS Advantage Why leverage cloud infrastructure?

• Up and running in a day

– Compute

– Database

– Storage

– Networking

– Management Tools

– Security and Identity

• Low start up costs

– Low monthly cost after first year

– Only pay for what you need

• Focus on your business


The AWS Advantage Add services and scale infrastructure as you grow

• Additional AWS services that may benefit PetTrax

– Analytics

– Internet of Things (IoT)

– Mobile Services

– Application Services

• Scale up existing infrastructure

– Supporting company growth

– Only pay for what you need

• Leverage 3rd party integrators

– Apps Associates and others


Apps Associates Partnership AWS Premier Consulting Partner

• Infrastructure sizing and deployment – Compute resources – Web (Apache) – Application (Tomcat) – Database (MySQL Aurora) – Messaging (MQTT)

• Application development – server – Java REST services – Database design and optimization – MQTT configuration

• Application development – mobile – iOS – Android


Migrating to Aurora

• Preparing for the future • MYSQL Version 5.6 • Stood up a separate application stack for Aurora to

avoid interference with ongoing development • Migration done in 7 days • Changed one line of code in the application

– Database connection string

• Testing – simulated load on application with Aurora • Everything worked in Aurora as in MYSQL • De-commissioned MYSQL technology stack • Tested failover and recovery • Ready for the future


Beta Solution 6 months later

Intelligent Feeders • Authorization-based feeder and monitor

Mobile Application

Cloud Infrastructure • Compute • Web Service • Application Service • Database • Messaging


Summary Convergence of forces pet awareness and enabling technology

• PetTrax business focus

– Pets are family members now

– Health of our loved ones is paramount

– Pet spending continues to increase

– Pet-Social Media interactions will drive sales

• AWS & Apps Associates technology focus

– Internet of Things having exponential growth

– Mobile monitoring is a reality

– Information will drive more services

– Robust/scalable AWS Cloud infrastructure enables growth


Migration to Aurora

Kshitij Kumar Global VP of BI/EPM and CTO Apps Associates LLC


About Apps Associates

• High Growth History

• HC Worldwide: 650

• Global Presence: US, Europe, Asia

• Market Leader in Several Segments of NE US with Rapid Expansion Across US

• Strong AWS Partnership

Market Leader: Oracle on AWS MSP Certification – within first 10 partners WW 1 of 7 partners for MMN Accelerator Program Recently selected for Partner Advisory Board &

DB Accelerator partner


• Migrate existing My SQL

– Lift and Shift

– MY SQL Version 5.6

– InnoDB Framework

– EC2 Classic to VPC

1. AWS Data Migration Service 2. AWS Schema Migration Service

Migrate Commercial Databases like Oracle

– Custom Application

– Web Application

– One Off Departmental Reporting Application

– Pilot project

Migrating to Aurora


Typical Migration Timeline (4 Weeks)

Assessment

( 2 days)

• Analysis of workload

• Analysis of current DB objects / code

• Understand Performance characteristics

• Develop TCO

• Prepare migration approach

Build and Test

( 8 + 4 days)

• Build/Configure AWS platform; security model

• Migrate from MyISAM to Innodb

• Setup replication from dev to Aurora Dev

• Complete migration

• Support testing

• Documentation & signoff

Production Migration & Support

( 5 days)

• Perform production migration

• Support Go-Live cutover process

• Provide post-production setup

• Configure maintenance jobs

• Knowledge Transfer & Sign-off

Part – I (2 to 3 weeks) Part – II (1 Week)


Define Design Move Develop Test Validate

Design Slice

Database

Application

Aurora

In-DB Code

Web App

Client Server

DB Objects (Tables etc.)

Packages etc.

Re-connect

Host on AWS

Ease of Migration

Backup / Recovery

Access

Perf

Security

Business

Success Criteria Business

Objective

Select DB

Test Criteria

Success Criteria Business

Objective

Demonstrate

Next Steps

Project Management / Oversight

Aurora Migration Approach

Co

re S

ervi

ces

Op

tio

nal

Ser

vice

s

DB

Tra

inin

g A

pp

Ch

ange

Tra

inin

g


Aurora Migration Utilities : Data Migration and Schema Conversion



AWS Schema Conversion Service


AWS Data Conversion Service


Closing



RDS Aurora – Test Drive Lab


Service Offering : Pricing Model


Connect with Us

Web: www.appsassociates.com

Email: [email protected]

YouTube: www.youtube.com/user/AppsAssociates

LinkedIn: www.us.linkedin.com/company/Apps-Associates

Twitter: @AppsAssociates

Facebook: www.facebook.com/AppsAssociatesGlobal

Google+: www.plus.google.com/+AppsAssociatesGlobal/

http://www.appsassociates.com/

mailto:[email protected]

http://www.youtube.com/user/AppsAssociates

http://www.us.linkedin.com/company/apps-associates



https://twitter.com/AppsAssociates

https://twitter.com/AppsAssociates

http://www.facebook.com/AppsAssociatesGlobal

http://www.plus.google.com/+AppsAssociatesGlobal/

Thank You!

deploying high-performance database services amazon aurora · – pet’s ability to eat must be...

Documents