amazon efs: deploying scalable, shared file systems
TRANSCRIPT
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Introducing
Amazon Elastic File System
(Amazon EFS)
Timothy Harder
Agenda
1. Provide an overview of Amazon EFS
2. Introduce Amazon EFS technical concepts
3. Conduct a walkthrough of creating a file system
4. Discuss file system security mechanisms
5. Explore the Amazon EFS regional availability and
durability model
Overview of Amazon EFS
The AWS storage portfolio
Amazon S3• Object storage: Data presented as buckets of objects
• Data access via APIs over the Internet
Amazon
EFS• File storage (analogous to NAS): Data presented as a file system
• Shared low-latency access from multiple EC2 instances
Amazon
Elastic Block
Store
• Block storage (analogous to SAN): Data presented as disk volumes
• Lowest-latency access from single Amazon EC2 instances
Amazon
Glacier• Archival storage: Data presented as vaults/archives of objects
• Lowest-cost storage, infrequent access via APIs over the Internet
What is Amazon EFS?
• Fully managed file system for EC2 instances
• Provides standard file system semantics
• Works with standard operating system APIs
• Sharable across thousands of instances
• Elastically grows to petabyte scale
• Delivers performance for a wide variety of workloads
• Highly available and durable
• NFS v4–based
Amazon EFS is designed for a broad range of
use cases, such as:
• Content repositories
• Development environments
• Home directories
• Big data
Operating shared file storage today is a pain
Application owner
or developer
IT administrator
Business owner
• Estimate demand
• Procure hardware
• Set aside physical space
• Set up and maintain hardware (and network)
• Manage access and security
• Provide demand forecasts/business case
• Add lead times and extra coordination to your schedule
• Limit your flexibility and agility
• Make up-front capital investments, over buy, stay on a
constant upgrade/refresh cycle
• Sacrifice business agility
• Distract your people from your business’s mission
We focused on changing the game
Amazon EFS
is simple
Amazon EFS
is elastic
Amazon EFS
is scalable
1 2 3
Amazon EFS is simple
• Fully managed
– No hardware, network, file layer
– Create a scalable file system in seconds!
• Seamless integration with existing
tools and apps
– NFS v4—widespread, open
– Standard file system semantics
– Works with standard OS file system APIs
• Simple pricing = simple forecasting
1
Amazon EFS is elastic
• File systems grow and shrink
automatically as you add and remove
files
• No need to provision storage capacity
or performance
• You pay only for the storage space you
use, with no minimum fee
2
• File systems can grow to petabyte
scale
• Throughput and IOPS scale
automatically as file systems grow
• Consistent low latencies regardless
of file system size
• Support for thousands of concurrent
NFS connections
Amazon EFS is scalable3
Why does this matter…
… to app owners
and developers?
… to your
business?
• Easy to move existing code, applications, and tools
used today with existing NFS servers to the AWS cloud
• Simple shared file storage solution for new cloud-native
applications
• Predictable pricing with no up-front investment
• Increased agility
• Spend less time managing file storage and more
time focusing on your business
… to IT
administrators?
• Eliminates need to manage and maintain file system
storage at scale
Diving In
Some key AWS concepts to understand
• Region
• Availability Zone (AZ)
• Amazon Virtual Private Cloud (VPC)
Region
• Geographic area where
AWS services are available
• Customers choose
region(s) for their AWS
resources
• 11 regions worldwide
REGION
Availability Zone (AZ)
• Each region has multiple,
isolated locations known as
Availability Zones
• Low-latency links between
AZs in a region
• When launching an EC2
instance, a customer
chooses an AZ AVAILABILITY ZONE 3
EC2
AVAILABILITY ZONE 2
AVAILABILITY ZONE 1
EC2EC2
EC2
REGION
Amazon VPC
• Logically isolated section
of the AWS cloud, virtual
network defined by the
customer
• When launching instances
and other resources,
customers place them in a
VPC
• All new customers have a
default VPC
AVAILABILITY ZONE 1
REGION
AVAILABILITY ZONE 2
AVAILABILITY ZONE 3
VPC
EC2EC2
EC2
EC2
What is a file system?
• The primary resource in Amazon EFS
• Where you store files and directories
How to access a file system from an instance
• You “mount” a file system on an EC2 instance
(standard command); the file system appears like a
local set of directories and files
• An NFS v4 client is standard on Linux distributions
mount –t nfs4
[file system DNS name]:/
/[user’s target directory]
What is a mount target?
• To access your file system from instances in a VPC, you create mount targets in the VPC
• A mount target is an NFS v4 endpoint in your VPC
• A mount target has an IP address and a DNS name you use in your mount command
AVAILABILITY ZONE 1
REGION
AVAILABILITY ZONE 2
AVAILABILITY ZONE 3
VPC
EC2EC2
EC2
EC2
Mount
target
How does it all fit together?
AVAILABILITY ZONE 1
REGION
AVAILABILITY ZONE 2
AVAILABILITY ZONE 3
VPC
EC2EC2
EC2
EC2
Customer’s file
system
There are three ways to set up and
manage a file system
• AWS Management Console
• AWS Command Line Interface (CLI)
• AWS Software Development Kit (SDK)
The AWS Management Console, CLI, and SDK each
allow you to perform a variety of management tasks
• Create a file system
• Create and manage mount targets
• Tag a file system
• Delete a file system
• View details on file systems in your AWS account
Setting up and mounting a file system takes
less than a minute
1. Create a file system
2. Create a mount target in each AZ from which
you want to access the file system
3. Enable the NFS client on your instances
4. Run the mount command
Securing Your File System
Several security mechanisms
• Control network traffic to and from file systems (mount
targets) by using VPC security groups and network ACLs
• Control file and directory access by using standard
Linux/Windows directory-level/file-level permissions
• Control administrative access (API access) to file systems
by using AWS Identity and Access Management (IAM)
Only EC2 instances in the VPC you specify can access
your Amazon EFS file system
Customer’s file
system
VPC
EC2EC2
EC2
EC2
VPC
EC2EC2
EC2
EC2
VPC
EC2
EC2
Security groups control which instances in your VPC
can connect to your mount targets
Customer’s file
system
Security group:
sg-allowed
Security group:
Permit inbound traffic
from “sg-allowed”
Security group:
sg-not-allowed
Amazon EFS supports user-level file and directory
access permissions
• Set file/directory permissions to specify read-write-
execute permissions for users and groups
Integration with IAM provides administrative security
• Use IAM policies to control who can use
the administrative APIs to create, manage,
and delete file systems
• Amazon EFS supports action-level and
resource-level permissions
Regional Availability and Durability
In what regions can I use Amazon EFS?
• US-West (Oregon) *Preview
• US-East (Northern Virginia)
• EU (Ireland)
Data is stored in multiple AZs for high availability
and durability
• Every file
system object
(directory, file,
and link) is
redundantly
stored across
multiple AZs in
a region
AVAILABILITY
ZONE 1
REGION
AVAILABILITY
ZONE 2
AVAILABILITY
ZONE 3
Amazon
EFS
Data can be accessed from any AZ in the region
while maintaining full consistency
• Your EC2 instances can connect to your Amazon EFS file system from any AZ in a region
• All reads and writes will be fully consistent in all AZs; that is, a read in one AZ is guaranteed to have the latest data, even if the data is being written in another AZ
AVAILABILITY
ZONE 1
REGIONVPC
EC2EC2
EC2
AVAILABILITY
ZONE 2
AVAILABILITY
ZONE 3
EC2
Write
Read
Wrapping Up
Simple and predictable pricing
• With Amazon EFS, you pay only for the storage space you use
– No minimum commitments or up-front fees
– No need to provision storage in advance
– No other fees, charges, or billing dimensions
• Amazon EFS price: $0.30/GB-month
What to do next?
• Learn more at aws.amazon.com/efs
• Request an invite for our preview
Timothy Harder
NEW YORK