amazon web services - rjug€¦ · amazon ec2 elastic compute cloud resizable compute capacity in...
TRANSCRIPT
Amazon Web ServicesBuilding in the Cloud
Amazon has Three Parts
AWS Principles
Easy to use
Fast
Elastic
Highly available
Secure
Pay as you go
The Utility Model
AWS Bandwidth Growth
AWS Storage Growth
Services Overview
Building block services that allow developers to innovate and make money
Infrastructure As a ServiceAmazon Simple Storage ServiceAmazon Elastic Compute CloudAmazon Simple Queue ServiceAmazon SimpleDB
Commerce As a ServiceAmazon Flexible Payments Service
Fulfillment Web Service
People As a ServiceAmazon Mechanical Turk
Alexa Web ServicesAlexa Web Information Service
Alexa Top Sites
Alexa Site Thumbnail
Alexa Web Search Platform
Amazon S3Simple Storage Service
Highly scalable data storage in-the-cloud
Programmatic access via web services API
Simple to get going, simple to use
Highly available and durable
Pay-as-you-go:
Storage: $0.15 / GB / month
Data Transfer: starts at $0.17 / GB
Requests: nominal charges
Content Delivery Network
Amazon EC2Elastic Compute Cloud
Resizable compute capacity in the cloudObtain and boot new server instances in minutesQuickly scale capacity, up or down, as your computing requirements change
Full root access to a blank Linux or Windows machineSimple Web service management interfaceChanges the economics of computing
Amazon EC2 FeaturesElastic Block Store
Persistent storage, up to 1 TBBacked by S3 Snapshots
Availability ZonesSeparate Data CentersIncreased Reliability
Elastic IP AddressesFixed IP can be quickly attached to an instanceDon't need to wait for DNS propagation delays
Predictions Cost Money
Amazon Machine ImagesPublic AMIs: Use pre-configured, template AMIs to get up and running immediately. Choose from Fedora, Ubuntu, Windows configurations and more
Private AMIs: Create an Amazon Machine Image (AMI) containing your applications, libraries, data and associated configuration settingsPaid AMIs: Set a price for your AMI and let others purchase and use it (Single payment and/or per hour)
Instance Types
Standard High CPUSmall Large Extra
LargeMedium Extra Large
Bits 32 64 64 32 64RAM 1.7 GB 7.5 GB 15 GB 1.7 GB 7 GBDisk 160 GB 850 GB 1690 GB 350 GB 1690 GBEC2 Compute Units
1 4 8 5 20
I/O Performance
Medium High High High High
Firewall Yes Yes Yes Yes YesPer Hour $0.10 $0.40 $0.80 $0.20 $0.80
Amazon SQSSimple Queue Service
Scalable Messaging - many application threads/servers can be used without worrying about overloading the messaging system Unlimited - any number of queues, any number of messages per queue Supports reader fail-over -messages become invisible to other readers when read the first time. Adjustable time before message is again visible to other readers
Amazon SimpleDBSimple to use - implements most common / simple database features behind a set of web service callsFlexible - doesn't require structured definition of data which allows you to add new data easilyScalable - scales easily as your application grows. Simply add more domains or data within your domainsFast - quick efficient storage and retrieval of your dataReliable - uses Amazon's high-availability data centers to provide strong and consistent performanceInexpensive - you pay only for the resources you consume, so you avoid significant up-front costs.
Eventual Consistency!
Using Amazon SimpleDBitem description color material123 Sweater Blue, Red456 Dress shirt White, Blue789 Shoes Black Leather
PUT (item, 123), (description, Sweater), (color, Blue), (color, Red)PUT (item, 456), (description, Dress shirt), (color, White), (color, Blue)PUT (item, 789), (description, Shoes), (color, Black), (material, Leather)
QueryDomain = MyStore[‘description’ = ‘Sweater’]
Leveraging AWSApplication hosting - Using EC2 to provided an elastic web tier
Batch processing - Using SQS, EC2 and S3 to process large amounts of dataSearchable Object Store - Combine S3 and SimpleDB to store metadata data about objects and run searches
Cloud-Bursting - Leveraging EC2 for excess capacity to augment your own data center
New York Times and AWS
24 hours later
Animoto scales
50 to 3500 servers in 3 days!
Using AWS with JavaWeb services provide language independence
WSDL can be used to build an access layer
Popular Open Source Java libraries include;Jets3t - Amazon S3 accesstypica - Amazon EC2, SQS, SimpleDB, FPS, DevPay
Jets3tAn open source library for interacting with Amazon S3 and objects stored there.Supports both SOAP and REST interfaces.
Well tested and very robust.
S3 APIpackage org.jets3t.service; Class to know about;
S3Service S3Bucket createBucket(String name) S3Object getObject(S3Bucket bucket, String key)
S3Bucket setAcl(AccessControlList acl)
S3Object setContentType(String type) setDataInputStream(InputStream iStr)
S3 Code Sample// code from a servlet that proxies data from S3 s3 = new RestS3Service(new AWSCredentials(awsAccessId, awsSecretKey)); // get the object details from S3 S3Object obj = new S3Object(objectId);S3Bucket bucket = new S3Bucket(bucketName);obj = s3.getObject(bucket, objectId); // use the stream from the S3Object to move data to the servlet response stream OutputStream oStr = response.getOutputStream();response.setContentType(obj.getContentType());InputStream in = obj.getDataInputStream();Copier.copyStreams(in, oStr);
TypicaTypica was designed initially to support SQS via the REST interface in a very reliable and thread-safe way. The initial single digit downloads were inspiring enough to get EC2 support added.
Incorporates patches from users and supports many popular services.
Current popularity means hundreds of downloads for each release.
SQS APIpackage com.xerox.amazonws.sqs;
Classes to know about;QueueServicemessageQueue getOrCreateMessageQueue(queueName)List<MessageQueue> listMessageQueuesMessageQueuesendMessage(msg)String receiveMessage()deleteMessage(msgId)deleteQueue(force)MessageString getMessageBody()
SQS Sample CodeQueueService qs = new QueueService(<accessId>, <secretKey>);
MessageQueue queue = qs.getOrCreateMessageQueue("testQueue");
queue.sendMessage("test Message");
Message msg = queue.receiveMessage();
logger.debug("message received : "+msg.getMessageBody());
EC2 APIpackage com.xerox.amazonws.ec2;
Classes to know about;Jec2List<ImageDescription> describeImages(imageIds)List<ReservationDescription> describeInstances(instanceIds)ReservationDescription runInstances(imageId, ...)ConsoleOutput getConsoleOutput(instanceId)ReservationDescriptionList<Instance> getInstances();
EC2 Sample CodeJec2 ec2 = new Jec2(<accessId>, <secretKey>);ReservationDescription res = ec2.runInstances("ami-1234567", 1, 1, null, null, "dak-keypair");String instanceId = res.getInstances().get(0).getInstanceId();while (!res.getInstances().get(0).getState().equals("running")) { try { Thread.sleep(5000); } catch (InterruptedException ex) {} res = ec2.describeInstances(instanceId);}logger.debug("Instance running : "+instanceId); ConsoleOutput cons = ec2.getConsoleOutput(instanceId);logger.debug("console output = "+cons.getOutput());ec2.terminateInstances(new String [] {instanceId});
SimpleDB APIpackage com.xerox.amazonws.sdb;
Classes to know about;SimpleDB
ListDomainsResult listDomains() Domain createDomain(String name)
Domain QueryResult listItems(String query) QueryWithAttributesResult listItemsWithAttributes(String query)
Item List<ItemAttribute> getAttributes() SDBResult putAttributes(List<ItemAttribute> attrs)
SimpleDB Sample CodeSimpleDB sdb = new SimpleDB(<accessId>, <secretKey>);Domain dom = sdb.getDomain("testDomain");logger.info("items:");String nextToken = "";while (nextToken != null) { ListDomainsResult result = dom.listItems(query, nextToken, 10); List<Item> items = result.getItemList(); for (Item item : items) { logger.info(item.getIdentifier()); } nextToken = result.getNextToken();}
Useful Linkshttp://aws.amazon.com/
http://code.google.com/p/typica/
http://jets3t.s3.amazonaws.com/index.html