aws re:invent 2016: how netflix achieves email delivery at global scale with amazon ses (mbl204)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

MBL204

November 29, 2016

How Netflix Achieves Email Delivery at Global Scale

with Amazon SESBest Practices for Building Scalable and Reliable Email Applications using Amazon SES

Devika Chawla, Netflix

Kadir Rathnavelu, Amazon SES

Challenges with Email Sending

• Delivery of email to your recipient’s inbox is hard

• Managing the complaint feedback loop is complex

• The undifferentiated heavy lifting of setting up an email

infrastructure

Netflix Email Delivery at Global Scale with

Amazon SES

Agenda

• Netflix Email Overview

• History, Challenges & Motivation for Amazon SES

• Move to Amazon SES

Netflix Email Overview

Netflix Ecosystem

Sign up flow

Sign up flow

Sign up flow

Sign up flow

Non Member – Remarketing Series

Sign up flow

Members - Welcome Series

Members – Personalized Messages

Cancel Membership

Former Member - Rejoin Series

Emails across the customer lifecycle

Acquisition

Engagement Transactional

Rejoin

Message Classification

Member - Transactional

Most Important

Very Low Risk

Member - Other

Very Important

Low Risk

Non/Former Member

Important

Medium Risk

History, Challenges &

Motivation for Amazon SES

Messaging Platform in AWS & Send from Data Center

us-west-2 us-east-1 eu-west-1

Challenges

• Keep email delivery software running in Data Center

• Optimize settings for ISPs

• Process bounce logs

• Work with 3rd party for complaint feedback loop

• Process complaint feedback

Challenges

Requirements for Netflix Email

• Great Deliverability for 100 million plus messages/day

• Linear scale with global subscriber growth, at cost

• Multiregion delivery of transactional messages

• Risk Isolation for different types of messages

• Operational simplification

Move to Amazon SES

Dedicated Pools of IPs

Messages

Message

Processor

Message

Metadata

Service

SES

Transactional

Pool

SES

Member

Pool

SES

Non/Former

Member

Pool

Password

Reset

New

Season

Re-

marketing

Multi Region Presence

us-west-2 us-east-1 eu-west-1

SESSES SES

Streamlined Feedback for Delivery, Bounces, Complaints

Delivered/sec Bounces/sec

Complaints/sec Latency – 95th Percentile

Gradual Ramp up to Amazon SES (IP Warming)

us-west-2 us-east-1 eu-west-1

SESSES SES

IP Reputation (from 3rd Party vendor)

Inbox Deliverability (from 3rd Party vendor)

Migration

Monitor

Bounces

IP Reputation

Inbox Deliverability

Increase Traffic to SES

100% Emails on Amazon SES

us-west-2 us-east-1 eu-west-1

SESSES SES

Building Scalable and Reliable Email

Applications

Reiterating the challenges with sending emails

• The reputation of the email sender

• Managing the complaints feedback loop is complex

• The undifferentiated heavy lifting of setting up an email

infrastructure

Amazon SES eliminates these challenges

• Delivering emails to an inbox is a shared responsibility

• Amazon SES provides…

• Higher deliverability via proactive reputation management

• Maintains complaint feedback loops with major ISPs

• Elasticity

• High availability

• The breadth of features to build reliable sending programs

Fundamentals of Setting up an

Email Program

Setting up email authentication

• Verify an email address or a domain

Setting up email authentication

• Verify an email address or a domain

• Set up DomainKeys Identified Mail (DKIM)

DomainKeys Identified Mail (DKIM)

• An authentication standard that allows you to sign your email

cryptographically

• ISPs decode the cryptographic signature using a public key to

ensure that your emails are not modified in transit

• Set up DKIM for your domain with DKIM from Amazon SES Console

Setting up email authentication

• Verify an email address or a domain

• Set up DomainKeys Identified Mail (DKIM)

• Add an Sender Policy Framework (SPF) record

Sender Policy Framework (SPF)

• SPF indicates which mail servers are authorized to send email for

their domain

• Receiving mail server compares MAIL FROM domain’s DNS record

for authorization

• Two ways to set up SPF on Amazon SES

• Use default MAIL FROM domain of Amazon SES

• Publish an SPF record

Setting up email authentication

• Verify an email address or a domain

• Set up DomainKeys Identified Mail (DKIM)

• Add an Sender Policy Framework (SPF) record

More on set up

• Use Mailbox Stimulator for scalability testing

• Use HTTP instead of SMTP

• Send in Parallel

Feedback Processing

Why do you want to process feedback?

• Listen to feedback about the emails you send

• Delivered

• Bounce

• Complaints

• Acting on bounces and complaints improves the deliverability of

your emails

• Improves customer experience

Monitoring your sending activity

Monitoring Method How to Access Data Level of Detail Granularity

Feedback Notifications Amazon SNS, or Email Details on each

event

Across entire AWS Account

Event Publish Amazon CloudWatch, or Amazon

Kinesis Firehose

Details on each

event

Fine grained (based on

user-defined email

characteristics)

Amazon SES Event Publishing

• Fine-grained metrics based on a definable email characteristics like

marketing vs. transactional, product, etc.

• If you are selling music, you could define and track by ‘genre’

• Publish to Amazon CloudWatch or Amazon Kinesis Firehose

Amazon SES Event Publishing

• Create a configuration set

• Provide the name of the configuration

set, and one or more message tags

Amazon SES Event Publishing

• Amazon CloudWatch is best suited for aggregated metrics

• Metrics are available under Amazon SES namespace in Amazon

CloudWatch

• Event are aggregated by type and combination of specified dimensions

• Amazon Kinesis Firehose enables you to see individual events

• Stream data in to S3, Amazon Kinesis Analytics, Amazon Redshift, or

Elasticsearch Service Cluster

Dedicated IPs

Shared IPs

• Amazon SES customers send their emails from a single set of IP

addresses

• Everyone is responsible for the reputation of the IP addresses

What are dedicated IPs?

• Dedicated IPs are Amazon SES IP addresses that are reserved

exclusively for your email sending

• Take more control and responsibility over the deliverability

• Scenarios:

• Segmenting mail streams for separate programs, or

engagement metrics

• Sending transactional mail, such as order confirmations, in

isolation from your marketing mail

• Send diagnostic or operational email from a certain set of IPs,

and reject all other mails

Dedicated IP addresses

• Your IP reputation will ultimately be driven by your own

sending practices

• Gradual warm up will reduce throttling and potential

rejects from the ISPs

• Start small

• Measure results

• Make corrections

• Ramp up the sending

Dedicated IP addresses

• Failure to keep the IPs warmed up will result in worse

deliverability

• You are responsible for estimating any potential volume

increases

Thank you!

Remember to complete

your evaluations!