serverlessconf: serverless for the enterprise - rafal gancarz
TRANSCRIPT
2
So you want to do Serverless?
fun, cool, cutting edge, cost effective, agile, competitive, fast to deliver
@RafalGancarz
3
… but you are an Enterprise
security, operability, support, compliance
@RafalGancarz
4
… or you just want to build something more substantial
multiple development teams, CI/CD, DevOps, automated testing, loose coupling,
microservices, event-driven@RafalGancarz
Simple use case - chat bot
5@RafalGancarz
Simple use case - image resizer
6@RafalGancarz
• API-first platform for the cloud management vendor
• 10s of APIs • CRUD, scheduled/triggered
background jobs, event-driven processing
• Strict security policies • SAML SSO integration • Rich RBAC model • Legacy system integration • CI/CD • Operations • Tooling
7
Advanced use case
?@RafalGancarz
• Tight coupling • Sprawling web of
dependencies • All or nothing deployments • Team dependencies/
coordination challenges
8
Serverless monolith
ABC
API
@RafalGancarz
• Scoped around domain bounded contexts or platform capabilities
• Isolated infrastructure stack (Terraform state)
• Consistent naming of resources • Dedicated build/deployment
pipeline • 0 .. N Lambda functions • Explicitly defined security policies
and dependencies • Optionally exposes an API • Optionally exposes domain events • Optionally subscribes to event
sources
9
Serverless Components
A B C
API API
@RafalGancarz
• Synchronous call to another component • via HTTPS API • via direct Lambda
invokation • no buffering • you pay for the wait
time :)
10
Integration patterns
@RafalGancarz
• Asynchronous call to another component • send a message to SQS
queue or SNS topic • fire and forget • decoupled • flexible message formats
helpful
11
Integration patterns
@RafalGancarz
• Asynchronous notifications • subscribe to SNS topic • decoupled • flexible message formats
helpful
12
Integration patterns
@RafalGancarz
• Swagger for API specifications • API definitions can be used for model
validation and contract testing • API versioning (for non backward-
compatible changes) • Developer portal for API discovery
and documentation
13
APIs
14
Continuous delivery/deployment
@RafalGancarz
15
Continuous delivery/deployment
@RafalGancarz
16
Infrastructure provisioning
@RafalGancarz
17
Code deployments
ES 2015
+
@RafalGancarz
(babeljs.io)(rollupjs.org)
(apex.run)
18
Testing
Unit testing (local/CI)
Acceptance testing (test environment)
Smoke testing (post deployment)
testing pyramid
@RafalGancarz
19
Testing
+ +
Unit testing (local + CI)
+
Acceptance/functional testing (AWS)
+
Smoke testing (AWS)
+
@RafalGancarz
(mochajs.org) (chaijs.com) (sinonjs.org) (proxyquire)
• Least privilege policy based on IAM roles
• IAM credentials and STS used for operations/tooling access
• Credential/key rotation • Encryption in transfer • Encryption in storage (for sensitive
data)
20
Security principles
@RafalGancarz
21
Security in depth
@RafalGancarz
• Authentication • Bespoke SSO solution integrated with the legacy system • Serverless SAML SSO + JWT token
• Authorisation • Custom authorisation library • API Gateway custom authorisers not flexible enough
• Sensitive configuration values • Encrypted objects in the component’s S3 bucket
• Secret management/storage component • Bespoke serverless solution based on DynamoDB and
KMS
22
Security
@RafalGancarz
• Bunyan logging library • JSON formatted logs • business specific metadata • transaction tracing
• CloudWatch Logs for log collection • Log aggregation pipeline (Lambda + Kinesis + Lambda) • Bespoke log filtering solution • Elastic Search for log storage • Kibana for log browsing
23
Logging
@RafalGancarz
• CloudWatch metrics and alarms • API Gateway • Lambda • DynamoDB • Billing
• Bespoke Operations Health Dashboard application • CloudTrail for auditing
• API Gateway • AWS API calls
24
Monitoring & Auditing
@RafalGancarz
• Serverless NPM registry (S3 + Lambda + API GW) • Jenkins Pipeline DSL scripts (Groovy) • Deployment framework (Python) • Operations Dashboard (Node) • more to come …
25
Tooling
@RafalGancarz
• Serverless is still in its infancy - lots of problems to solve (sometimes not the problems you’d like to be solving)
• Serverless evolves rapidly - new tools/solutions are emerging, new features becoming available, new ideas&patterns are being shared
• Strong dependency on the cloud provider (SDKs, tools, support, limits)
• Plan/adapt your capacity (DynamoDB, Kinesis) • Serverless frameworks are great for some use cases but
quite opinionated and limiting for others (particularly around stack provisioning/management)
• Tooling is sparse, a lot of ‘build your own’
Lessons learned
26@RafalGancarz
