serverless best practices for configuration management and... · summit © 2019, amazon web...

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMMIT

Serverless best practices for configuration management and cost optimization

Alex Casalboni


About me

• Software Engineer & Web Developer

• Startupper for 4.5 years

• Serverless Lover & AI Enthusiast

• ServerlessDays Organizer

• AWS Customer since 2013


Lambda permission model

Fine-grained security controls for both execution and invocation

Execution policiesDefine what AWS resources/API calls can this function access via AWS Identity and Access Management (IAM)

Used in streaming invocations

For example, “Lambda function A can read from DynamoDB table users”

Function policiesUsed for sync and async invocations

For example, “Actions on bucket X can invoke Lambda function Z"

Resource policies allow for cross account access


Action: “s3:*”… make puppies cry!Action: “dynamodb:*"

Action: “sns:*“

Photo by Matthew Henry on Unsplash

https://unsplash.com/photos/2Ts5HnA67k8?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText

https://unsplash.com/search/photos/sad-dog?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText


Fine-grained IAM policy with AWS SAM

MyFunction:

Type: AWS::Serverless::Function

Properties:

Handler: index.handler

Runtime: python2.7Policies:

- AWSLambdaExecute # Managed Policy

- Version: '2012-10-17'

Statement:

- Effect: Allow

Action:

- dynamodb:GetItem

Resource: !GetAtt MyDynamoDBTable.Arn


Hardcoded secrets make fish cry!

Photo by Julieann Ragojo on Unsplash


AWS Lambda environment variables

Key-value pairs that you can dynamically pass to your function

Available via standard environment variable APIs (based on runtime)

Can optionally be encrypted via AWS Key Management Service (AWS KMS)

Allows you to specify in IAM what roles have access to the keys to decrypt the information

Useful for creating environments per stage (such as dev, test, prod)


AWS Systems Manager―Parameter Store

Centralized store to manage your configuration dataSupports hierarchies

Plaintext or encrypted with AWS KMS

Can send notifications of changes to Amazon Simple Notification Service (Amazon SNS) or Lambda

Can be secured with IAM

Calls recorded in AWS CloudTrail

Can be tagged

Available via API/SDK

Useful for centralized environment variables, secrets control, feature flags


Parameter Store access via SDK

import json, boto3

ssm = boto3.client('ssm')

def get_parameter():

response = ssm.get_parameter(

Name='LambdaSecureString’,

WithDecryption=True

)

return response['Parameter']['Value']

def lambda_handler(event, context):

value = get_parameter()

print(”value = %s" % value)


Parameter Store access via SDK with ssm_cache

import json, boto3

ssm = boto3.client('ssm')

def get_parameter():


Name='LambdaSecureString’,

WithDecryption=True

)



value = get_parameter()


from ssm_cache import SSMParameter

param = SSMParameter(‘LambdaSecureString’)


value = param.value


github.com/alexcasalboni/ssm-cache-python


AWS Secrets Manager

Allows you to manage, retrieve, and rotate credentials

Helps you rotate secrets regularly without breaking stuff

Keeps track of different password versions

Implements security controls associated with credential management

Built-in support for Amazon RDS


AWS Secrets Manager + Parameter Store

Uniform and consistent access to both services

You can reference Secrets Manager secrets with PS APIs

Rotation & Refresh delegated to the client

As simple as using a prefix: /aws/reference/secretsmanager/


Secrets access via Parameter Store

import json, boto3

ssm = boto3.client('ssm’)

prefix = ‘/aws/reference/secretsmanager’

def get_secret():


Names=[‘%s/my_secret’ % prefix],

WithDecryption=True

)



value = get_secret()



Secrets access via Parameter Store with ssm_cache

import json, boto3

ssm = boto3.client('ssm’)

prefix = ‘/aws/reference/secretsmanager’

def get_secret():


Names=[‘%s/my_secret’ % prefix],

WithDecryption=True

)



value = get_secret()


from ssm_cache import SecretsManagerParameter

secret = SecretsManagerParameter(‘my_secret’)


value = secret.value


github.com/alexcasalboni/ssm-cache-python


Parameters & secrets grouping with ssm_cache

from ssm_cache import SSMParameterGroup

group1 = SSMParameterGroup(max_age=300) # 5min cache

param1 = group.parameter('param_1’)

param2 = group.parameter('param_2’)

group2 = SSMParameterGroup(base_path="/Foo") # common prefix

foo_bar = group2.parameter('/Bar') # will fetch /Foo/Bar

baz_params = group2.parameters('/Baz') # will fetch /Foo/Baz/1 and /Foo/Baz/2

secret = group2.secret(‘my_secret’) # will fetch /aws/reference/secretsmanager/my_secret

group1.refresh()

group2.refresh()


Anatomy of a function

Your function

Language runtime

Function container

Compute substrate


The request lifecycle

Bootstrap

the runtime

Start your

code

Cold

start

Warm

start

Download

your code

Start new

container

AWS optimization Your optimization


Same view in AWS X-Ray


Efficient function code

Avoid “fat”/monolithic functions

Control the dependencies in your function's deployment package

Optimize for your language

Node – Browserfy, Minify


Ephemeral function environment

Lambda processes a single event per-container

No need for non-blocking execution on the frontend

REMEMBER – containers are reused

Lazily load variables in global scope

Don’t load it if you don’t need it


Ephemeral function environment

import boto3

client = None

def my_handler(event, context): if not client:

client = boto3.client("s3")

# process


Concise function logic

Separate Lambda handler from core logic

Use functions to TRANSFORM, not TRANSPORT

Read only what you needQuery filters in Amazon Aurora

Use Amazon S3 select


Concise function logic (example)

from mylib import MyLibClass


operation = event['Operation’]

myobj = MyLibClass()

if operation == ‘do_this’:

my_obj.do_this()

elif operation == ‘do_that’:

myobj.do_that()

else:

raise ValueError(‘Invalid op’)


Concise function logic (example)

import boto3

ddb = boto3.client(‘dynamodb’)

class MyLibClass(object):MY_CONSTANT = ‘blabla’def __init__(…):

# constructordef do_this(self):

# use ddb to do thisdef do_that(self):

# use ddb to do that

from mylib import MyLibClass


operation = event['Operation’]

myobj = MyLibClass()

if operation == ‘do_this’:

my_obj.do_this()

elif operation == ‘do_that’:

myobj.do_that()

else:

raise ValueError(‘Invalid op’)


Small changes, big difference

# Download and process all keys

for key in src_keys:

response = s3_client.get_object(…)

contents = response['Body'].read()

for line in contents.split('\n')[:-1]:

line_count +=1

try:

data = line.split(',')

srcIp = data[0][:8]

…

# Select IP Address and Keys for key in src_keys:

response = s3_client.select_object_content(expression=“SELECT SUBSTR(obj._1, 1, 8),

obj._2 FROM s3object as obj”)

contents = response['Body'].read()

for line in contents: line_count +=1 try:

…

After (95s, $0.028)Before (200s, $0.112)

https://github.com/awslabs/lambda-refarch-mapreduce


SMART RESOURCE ALLOCATION

Match resource allocation (up to 3 GB!) to logic

Stats for Lambda function that calculates 1000 times all prime numbers <= 1000000

128 MB 11.722s $0.024628

256 MB 6.6789s $0.028035

512 MB 3.1949s $0.026830

1024 MB 1.4659s $0.024638


“AWS Lambda Power Tuning”

Data-driven cost & performance

optimization for AWS Lambda

github.com/alexcasalboni/aws-lambda-power-tuning

Don’t guesstimate!


No orchestration in codeS

TA

RT

JO

B

JO

B #

X S

TA

RT

ED

HT

TP

PO

ST

HT

TP

PO

ST

AR

E W

E T

HE

RE

YE

T?

NO

PE

!

WE

’RE

DO

NE

!ZzZz

OR

time.sleep(10)


No orchestration in code


Gateways & routers

Choose suitable entry point for client applications

Single, custom client? Use the AWS SDK

Not end user facing?Use regional endpoints on API Gateway

Discard uninteresting events ASAP

S3 – Event prefix

SNS – Message filtering (new!)


Queue based

Simple

No event store

Stream based

Think concurrent, not TPS


Resilient: retry policies

Understand retry policiesSync never retried

Async retried 2 times

Streams retried all the time

Leverage Dead Letter Queues (DLQ)SQS or SNS for replays

REMEMBER: Retries count as invokes


Concurrency Controls

Concurrency a shared pool by default

Separate using per function concurrency settingsActs as reservation

Also acts as max concurrency per functionEspecially critical for data sources like RDS

“Kill switch” – set per function concurrency to zero


Should my

Lambda

function be

in a VPC?

Does my function

need to access

any specific

resources in a

VPC?

Does it also need to

access resources or

services in the

public internet?

Don’t put the

function in a

VPC

Put the function

in a private

subnet

Put the function

in a subnet with

a NAT’d route

to the internetYes Yes

No No

Do I need a VPC?

serverless best practices for configuration management and... · summit © 2019, amazon web...

Documents