© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

7 July 2016

Deep Dive on Amazon Relational Database Service

Martin Minnock, Centre for Innovation & Analytics, AonPaul Burne -­ Technical Account Manager, AWS

Toby Knight -­ Manager, Solutions Architecture, AWS

What to expect

• Amazon RDS overview (super quick)• Security• Customer story• Migrating to RDS• Metrics and monitoring• Scaling on RDS• Backups and snapshots• High availability

No infrastructure management

Scale up/downCost-­effective

Instant provisioning

Application compatibility

Amazon Relational Database Service (Amazon RDS)

Amazon RDS engines

Commercial Open source Amazon Aurora

Amazon Aurora vs. MySQLFeature RDS Aurora RDS MySQL

Number of replicas Up to 15 Up to 5

Replication type Asynchronous (milliseconds) Asynchronous (seconds)

Replication performance impact on primary

Low High

Replica can act as failover target Yes (no data loss) Yes (potentially minutes of loss)

Storage Up to 64 TB, auto growth Up to 6 TB, specify storage limit

Automated failover Yes, to replica Yes, to standby

User-­‐defined replication delay No Yes

Replica support for different data or schema vs. primary

No Yes

Cross-­‐region replication No Yes

Data cache survives Yes No

Trade-­offs with a managed service

Fully managed host and OS• No access to the database host operating system• Limited ability to modify configuration that is managed on the host operating system

• No functions that rely on configuration from the host OSFully managed storage

• Max storage limits• SQL Server—4 TB• MySQL, MariaDB, PostgreSQL, Oracle—6 TB• Aurora—64 TB

• Growing your database is a process

Selected Amazon RDS customers

Security

Amazon Virtual Private Cloud (Amazon VPC)Securely control network configuration

Availability Zone

AWS Region

10.1.0.0/16

10.1.1.0/24Manage connectivity

AWS Direct Connect

VPN Connection

VPC Peering

Internet Gateway

Routing Rules

Security groupsDatabase IP firewall protection

Protocol Port Range SourceTCP 3306 172.31.0.0/16

TCP 3306 “Applicationsecurity group”

Corporate address admins

Application tier

Compliance

Singapore MTCS

27001/900127017/27018

MySQL and Oracle• SOC 1, 2, and 3• ISO 27001/9001• ISO 27017/27018• PCI DSS• FedRamp• HIPAA BAA• UK government programs• Singapore MTCS

Compliance

SQL Server and PostgreSQL• SOC 1, 2, and 3• ISO 27001/9001• ISO 27017/27018• PCI DSS• UK government programs• Singapore MTCS

SSL

Available for all six engines

Using SSL to encrypt a connection to a DB instance

mysql -h myinstance.c9akciq32.rds-eu-west-1.amazonaws \--ssl-ca=rds-combined-ca-bundle.pem --ssl-verify-server-cert.com

At-­rest encryption

• DB instance storage• Automated backups• Read Replicas• Snapshots

• Available for all six engines• No additional cost• Support compliance requirements

AWS KMS — RDS standard encryption

Two-­tiered key hierarchy using envelope encryption• Unique data key encrypts customer data• AWS KMS master keys encrypt data keys

Benefits:• Limits risk of compromised data key• Better performance for encrypting large data• Easier to manage small number of master keys than millions of data keys

• Centralized access and audit of key activity

Data Key 1

Amazon S3 Object

Amazon EBS Volume

Amazon Redshift Cluster

Data Key 2 Data Key 3 Data Key 4

CustomApplication

Customer MasterKey(s)

Enabling encryptionAWS Command Line Interface (AWS CLI)

aws rds create-­db-­instance -­-­region us-­west-­2 -­-­db-­instance-­identifier sg-­cli-­test \-­-­allocated-­storage 20 -­-­storage-­encrypted \-­-­db-­instance-­class db.m4.large -­-­engine mysql \-­-­master-­username myawsuser -­-­master-­user-­password myawsuser

aws rds create-­db-­instance -­-­region us-­west-­2 -­-­db-­instance-­identifier sg-­cli-­test1 \-­-­allocated-­storage 20 -­-­storage-­encrypted -­-­kms-­key-­id xxxxxxxxxxxxxxxxxx \-­-­db-­instance-­class db.m4.large -­-­engine mysql \ -­-­master-­username myawsuser -­-­master-­user-­password myawsuser

Amazon RDS + AWS KMS useful hints

• You can only encrypt on new database creation• Encryption cannot be removed• Master and Read Replica must be encrypted• Unencrypted snapshots cannot be restored to encrypted DB• Cannot restore MySQL to Aurora or Aurora to MySQL• Cannot copy snapshots or replicate DB across regions

IAM governed accessYou can use AWS Identity and Access Management (IAM) to control who can perform actions on RDS

Users and DBAApplications DBA and Ops

Your database RDS

Controlled with IAMControlled with database grants

IAM governed accessPolicies

"Action": ["rds:Describe*","rds:ListTagsForResource","ec2:DescribeAccountAttributes","ec2:DescribeAvailabilityZones","ec2:DescribeSecurityGroups","ec2:DescribeVpcs”,"cloudwatch:GetMetricStatistics","logs:DescribeLogStreams", "logs:GetLogEvents"],"Effect": "Allow","Resource": "*"

"Action": ["rds:*","cloudwatch:DescribeAlarms","cloudwatch:GetMetricStatistics","ec2:DescribeAccountAttributes","ec2:DescribeAvailabilityZones","ec2:DescribeSecurityGroups","ec2:DescribeSubnets","ec2:DescribeVpcs","sns:ListSubscriptions","sns:ListTopics", "logs:DescribeLogStreams", "logs:GetLogEvents" ],"Effect": "Allow","Resource": "*"

Read Only

Full Access

Prepared by Aon Inpoint | July 2016

RDS Deep DiveMartin Minnock -­ Aon Centre for Innovation & Analytics

130+ staffData Analysts | Data Scientists | Business Analysts | IT Development, Database & Infrastructure Specialists

Platforms, Projects & Servicesmulti-­channel web portals | ad-­hoc reporting | statistical analysis | machine learning initiatives

Dublin Centre for Innovation and Analytics at the heart of Aon Inpoint

Agile Scrum16 cross-­functional teams Agile Scrum & Kanban2 weekly sprints | Incremental releases

Aon Inpoint & ACIA (Dublin)

ACIA Reference Architecture for Analytics

Data Transformation & AnalysisData Lake Ingestion

Database

File/Object Storage

Message Channel

consume

Data Warehouses

Advanced Analysis

Mart

Marts

Mart

Mart

Analytics Distribution

Bespoke Analysis

Reports

APIs

Web Portal

Dashboards

Application Middleware

OrchestrationData Sources

Transactional Systems

Documents

PublicSources

ReferenceData

Logs

SQL

APIsJSON/XML

SFTP/PUT

Metadata Workflow & BatchMessaging

Technology Management

MonitoringSecurity Backup & RecoveryITIL Service Management

integrate

Logging & Audit.

Drivers for AWS Cloud Adoption

Performance and ProductivityPoor server performanceRe-­purposing/refreshing hardwareCapacity planning failsCumbersome work practices

Engagement Focus on business differentiationPromote experimentation & fail-­fastDrive innovationDevelop careers

Costs and Risks Poor utilisationResponsiveness to changeEmerging security standardsAgeing hardware / EoLSeparation of duties

Platform for GrowthGlobal user base Data increase across 4V’s Auto-­scaling analytics Democratisation of dataRelentless business appetite

Backend Databases for:

Analytics Delivery

Analytics Engine

NewProducts

Lift & Shift Targets

Short-­Life POC systems

Precedent for native AWS services

How ACIA uses RDS

Risk/View – Analytics Platform for Market & Risk Insights

Rapid Updates, Agile delivery

Customisable Future-­Proofed, Flexible

Focused on Self-­Service & Automation

Highly Available

Resource Intensive

Challenges (and Solutions)

3rd Party ToolsDatabase Refreshes

Missing Functionality EC2 (& BA)

RDS in the Ecosystem AWS DMS

Complete Lift & Shift – 100% AWS

Data Lake – feat. S3, EMR, and ECS

New Product Development

RDS for PostgreSQL, AWS Lambda for Python

Innovation! Data Science & Machine Learning

Intentions for the Future – RDS and Beyond

© Aon plc or its affiliates ("Aon"). All rights reserved.

NOTE: Aon does not provide or express an opinion or recommendation regarding any matter mentioned in this presentation.The recipient understands that neither Aon nor its employees makes or shall make any representation or warranty as to the accuracy or completeness of any information contained in this presentation. Aon shall not have any liability to the recipient or any other party resulting from the use of such information by the recipient or any other party.The information contained in this presentation may not be reproduced in any way or disseminated to any other party without the prior written consent of Aon.

Aon has endeavoured to ensure that this presentation is free of any virus or any other thing that would affect the recipient’s computer system. However, Aon cannot guarantee the security status of this presentation when accessed by the reader and shall not have any liability to the reader, recipient or any other party resulting from access to or use of the information contained herein.

Disclaimer

Migrating onto RDS

Historically, Migration = Cost, Time

Commercial data migration and replication software

Complex to setup and manage

Legacy schema objects, PL/SQL or T-­SQL code

Application downtime

Database Migration – 2 Steps

Step 1: Schema Conversion Overview

ü Move data to the same or different database engine

ü Keep your apps running during the migration

ü Start your first migration in 10 minutes or less

ü Replicate within, to, or from Amazon EC2 or RDS

AWS Database Migration Service

Customerpremises

Application Users

AWS

Internet

VPN

Start a replication instanceConnect to source and target databaseSelect tables, schemas, or databases

Let the AWS Database Migration Service create tables, load data, and keep them in syncSwitch applications over to the target at your convenience

Keep your apps running during the migration

Flexible Migration Approach

Replication instance

Source Target

Target

Target

Multiple targets

Replication instance

Source Target

Source

Source

Multiple sources

Source

L

Target

Replication instance instanceSelective

Metrics and monitoring

Summary of Metrics and Monitoring

• Amazon RDS Metrics• Event Notifications• Log Files• Cloudtrail

Accessing Amazon RDS Metrics

Amazon RDS Standard Metrics45 MetricsChange Time Period

Dive Deeper

Create Alarms

Amazon RDS Enhanced Monitoring

Access to over 50 metrics in 7 categories:

• Memory, • I/O, • CPU, • File system, • Load, • Swap• Processes

Amazon RDS Event Notifications

• Get Notified when events occur on your database instances

• 17 different event categories (availability, backup, configuration change, and so on)

• Uses Amazon Simple Notification Service (Amazon SNS)

Scaling on RDS

Scale out with Read Replicas

Relieve pressure on your master node for supporting reads and writes.

Bring data close to your customer’s applications in different regions

Promote a Read Replica to a master for faster recovery in the event of disaster

Replicas within and cross-­region• MySQL, MariaDB,

PostgreSQL• Aurora

Engines Needing Other Tools• Oracle • Microsoft SQL Server

Creating and Prompting Read Replicas Read Replica creation and promotion are accessed from the Instance Actions button in the RDS console

Creating and Promoting Read Replicas

Creating and Promoting Read Replicas With CLI

Creating and Promoting Read Replicas With CLI

Scaling Up and Down

• Handle higher load or lower usage

• Control costs

Scaling Up and DownConsole

Backups and snapshots

RDS Backups

MySQL, PostgreSQL, MariaDB, Oracle, SQL Server• Scheduled daily backup of entire instance• Archive database change logs• Up to 35 day retention for backups• I/O suspension as backup is initiated (but not with multi-­AZ deployment)• Multiple copies in each AZ where you have instances for a deployment

Aurora• Automatic, continuous, incremental backups• Point-­in-­time restore• No impact on database performance• 35 day retention

RDS Restore

• Restoring creates an entire new database instance• You define all the instance configuration just like a new instance

Snapshots

• Full copies of your Amazon RDS database that are different from your scheduled backups

• Backed by Amazon S3• Typical use cases

• Resolve production issues• Nonproduction environments• Point-­in-­time restore• Final copy before terminating a database• Disaster recovery• Cross-­region copy• Copy between accounts

High availability

Minimal deployment—single AZ

Availability Zone

AWS Region

10.1.0.0/16

10.1.1.0/24

Amazon Elastic Block Store Volume

High availability—Multi-­AZ

Availability Zone A

AWS Region

10.1.0.0/16

10.1.1.0/24

Availability Zone B

10.1.2.0/24

Replicated storage

Same instance type as master

High availability—Multi-­AZ to DNS

dbinstancename.1234567890.us-­west-­2.rds.amazonaws.com:3006

High availability—Amazon Aurora storage• Storage volume automatically grows up to

64 TB• Quorum system for read/write;; latency

tolerant• Peer-­to-­peer gossip replication to fill in

holes• Continuous backup to Amazon S3 (built for

11 9s durability)• Continuous monitoring of nodes and disks

for repair • 10 GB segments as unit of repair or hotspot

rebalance• Quorum membership changes do not stall

writes

AZ 1 AZ 2 AZ 3

Amazon S3

High availability—Aurora nodes• Aurora cluster contains primary node and up to 15 secondary nodes

• Failing database nodes are automatically detected and replaced

• Failing database processes are automatically detected and recycled

• Secondary nodes automatically promoted on persistent outage, no single point of failure

• Customer application can scale out read traffic across secondary nodes

AZ 1 AZ 3AZ 2

PrimaryNodePrimaryNodePrimaryNode

PrimaryNodePrimaryNode

SecondaryNode

PrimaryNodePrimaryNode

SecondaryNode

Aurora-­DNS Failover

AppRunningFailure Detection DNS Propagation

Recovery Recovery

DBFailure

MYSQL

AppRunning

Failure Detection DNS Propagation

Recovery

DBFailure

AURORA WITH MARIADB DRIVER

15 -­ 3 0 s e c

5 -­ 2 0 s e c

1 5 -­ 3 0 s e c

Driver benefits

Thank You!

ContactsMartin MinnockCloud Product Owner & Database Manager Aon Centre for Innovation & Analyticsmartin.minnock@aon.ie Paul Burne

Technical Account ManagerAmazon Web Servicespaulburn@amazon.co.uk

Toby KnightManager, Solutions ArchitectureAmazon Web Servicestobyk@amazon.co.uk

@martinminnock

Please remember to rate this session under My Agenda on

awssummit.london