aws day 3 create a batch processing cluster advance v2.5

Upload: cheo-garza

Post on 15-Oct-2015

114 views

Category:

Documents


0 download

TRANSCRIPT

  • Creating a Batch Processing Cluster

    Advanced

    v. 2.5_August, 2013 Copyright 2012-2013, Amazon Web Services, All Rights Reserved.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 2

    TABLE OF CONTENTS

    Table of Contents ......................................................................................................................................................................... 2

    Overview ..................................................................................................................................................................................... 4

    About the batch processing cluster .......................................................................................................................................... 4

    Learning objectives .................................................................................................................................................................. 4

    Download your EC2 Key Pair private key file ........................................................................................................................ 6

    Launch and configure a batch processing instance ...................................................................................................................... 8

    Create an IAM role .................................................................................................................................................................. 8

    Launch AN EC2 instance ........................................................................................................................................................ 8

    Create an AMI from your batch processing instance ............................................................................................................... 9

    Create two SQS task queues .................................................................................................................................................. 10

    Create an S3 bucket ............................................................................................................................................................... 10

    Launch worker nodes ............................................................................................................................................................. 10

    Dispatch work and view results ............................................................................................................................................. 11

    Monitoring the cluster ................................................................................................................................................................ 13

    Create an alarm ...................................................................................................................................................................... 13

    Create an SQS alarm .............................................................................................................................................................. 14

    Auto scaling the cluster.............................................................................................................................................................. 15

    Launch a CLI instance ........................................................................................................................................................... 15

    Create an auto scaling launch configuration .......................................................................................................................... 15

    Create a User Data Script ....................................................................................................................................................... 15

    Create a Launch Configuration .............................................................................................................................................. 16

    Create an auto scaling group .................................................................................................................................................. 16

    Create an auto scaling policy ................................................................................................................................................. 17

    Execute the auto scaling policy using CloudWatch ............................................................................................................... 18

    Test your policy ..................................................................................................................................................................... 18

    Conclusion ................................................................................................................................................................................. 19

    End Lab .................................................................................................................................................................................. 19

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 3

    Appendix A: Connecting to EC2 Instances ............................................................................................................................... 20

    Connect to your EC2 Instance via SSH (OS X and Linux) ................................................................................................... 22

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 4

    OVERVIEW

    In this lab, you use the AWS Management Console to build a basic batch processing cluster.

    You:

    Launch and configure an EC2 instance that will serve as the template for future worker

    nodes in your batch processing cluster

    Create an Amazon Machine Image (AMI) from that instance

    Use SQS to create task queues for passing message to your instances

    Launch a cluster of instances based on your AMI

    Schedule work via your task queues

    Observe the output in your output queue

    ABOUT THE BATCH PROCESSING CLUSTER

    The worker nodes in your cluster have a simple job: to convert some number of individual

    images into a single montage image. A worker node will download a list of images you provide

    and stitch them into a composite montage using the ImageMagick tool. While this is not the

    most CPU-intensive job, it does require some cycles, and the larger the size and number of

    images you provide for each job, the more work each node will have to do.

    For this lab, you will provide a newline-delimited list of image URLs. For example:

    https://us-east-1-aws-training.s3.amazonaws.com/arch-static-assets/static/20120728-DSC01265-L.jpg

    https://us-east-1-aws-training.s3.amazonaws.com/arch-static-assets/static/20120728-DSC01267-L.jpg

    https://us-east-1-aws-training.s3.amazonaws.com/arch-static-assets/static/20120728-DSC01292-L.jpg

    https://us-east-1-aws-training.s3.amazonaws.com/arch-static-assets/static/20120728-DSC01315-L.jpg

    https://us-east-1-aws-training.s3.amazonaws.com/arch-static-assets/static/20120728-DSC01337-L.jpg

    The node will download each image and produce an output:

    LEARNING OBJECTIVES

    By the end of this lab, you will be able to:

    Bootstrap an EC2 instance using UserData and CloudInit

    Create an AMI from a running instance

    Use the AWS Management Console to create an SQS queue

    Use the AWS Management Console to pass messages to, and read messages from, an

    SQS queue

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 5

    START YOUR QWIKLAB

    1) To the right of the Architecting on AWS, Day 3: Batch Processing link, click the Start

    Lab button to launch your qwikLAB.

    (Hint: If you are prompted for a token, use one you have been given or have

    purchased.)

    You will see the lab creation in progress.

    2) The instructions for this lab are located on the INSTRUCTION2 tab at the bottom of the

    window.

    3) On the exercise page, notice the lab properties.

    1) Duration - The time the lab will run for before automatically shutting down.

    2) Setup Time - The estimated time to setup the lab environment.

    3) AWS Region - The AWS Region where the lab resources are being created. (Note:

    The AWS Region in the image below is intentionally blank. Regions differ depending

    on the lab setup.)

    4) Copy the Password provided.

    (Hint: Selecting the value shown and pressing CTRL+C works best.)

    5) Click the Open Console button.

    6) Login to the AWS Management Console using the following steps:

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 6

    1) In the User Name field type awsstudent.

    2) Paste the password you copied from the lab details in qwikLAB into the Password

    field.

    3) Click the Sign in using our secure server button.

    In this step you logged into the AWS Management Console. Login credentials for the

    awsstudent AWS account are provisioned by qwikLAB using AWS Identity Access

    Management.

    7) In the AWS Management Console, click EC2.

    8) Confirm that the same AWS Region shown on the qwikLAB tab appears in the AWS

    Management Console toolbar.

    DOWNLOAD YOUR EC2 KEY PAIR PRIVATE KEY FILE

    1) In your browser, switch to the qwikLAB tab.

    2) Download the qwikLAB provided EC2 Key Pair Private Key file in PPK format (for

    Windows) or PEM format (for OSX or Linux). From the EC2 Key Pair Private Key drop-

    down list, choose the appropriate option for your operating system.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 7

    3) Save the file to your \Downloads directory (the default), or move it to another directory

    of your choice.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 8

    LAUNCH AND CONFIGURE A BATCH PROCESSING INSTANCE

    In this section, you use the AWS Management Console to launch and configure an EC2

    instance. This instance will be the template for the batch processing nodes in your cluster.

    CREATE AN IAM ROLE

    Your batch processing nodes will be communicating with SQS to receive processing

    instructions and to return results to S3. Start by creating an IAM Role that grants this access.

    1) From the Services menu, choose All AWS Services > IAM.

    2) In the Details section, click Roles.

    3) Click the Create New Role button.

    4) For Role Name, type: BatchProcessing and click Continue.

    5) For AWS Service Roles, select Amazon EC2.

    6) For Select Policy Template, choose Amazon SQS Full Access.

    7) At the Set Permissions panel, accept the default values and click Continue.

    8) Click Create Role.

    9) Select the newly created role in the list, and then in the Permissions tab below, click

    Attach Role Policy.

    10) For Select Policy Template, choose the Amazon S3 Full Access policy template, and

    then click Apply Policy.

    LAUNCH AN EC2 INSTANCE

    Next you launch an EC2 instance that forms the basis for your batch processing nodes.

    1) From the Services menu, choose All AWS Services > EC2.

    2) Click Launch Instance and choose the Classic Wizard.

    3) Click Continue.

    4) Select the following AMI: Amazon Linux AMI 2013.03.1 (64-bit).

    5) Verify the Instance Type is set to t1.micro and click Continue.

    6) For User Data, leave as text selected and paste (or type) the script below into the

    window (you may remove the comment lines beginning with #):

    #!/bin/bash

    # Install ImageMagick, a Python library, and create a directory

    yum install -y ImageMagick

    easy_install argparse

    mkdir /home/ec2-user/jobs

    # Download and install the batch processing script

    wget -O /home/ec2-user/image_processor.py \

    https://awsu-arch.s3.amazonaws.com/aux/technical-exercises\

    /day-2/lab_1_create_batch_processing_cluster/image_processor.py

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 9

    7) For IAM Role, choose BatchProcessing and click Continue until you reach the tags

    panel.

    8) On the tags panel, for the Name key, type the following in the Value column: Master

    Node.

    9) Click Continue.

    10) Leave the default key pair selected (starting with qwiklab) and click Continue.

    11) On the Configure Firewall panel, for Security group, choose Create a new security

    group (to allow SSH access from your workstation):

    1) For Group Name, type BatchMaster (and optionally enter a Description).

    2) Accept the default options to create an inbound TCP rule for port 22 (SSH) and click

    Continue.

    13) Click Launch and then click Close.

    14) When your instance is in the running state, SSH to the instance using the key pair you

    downloaded previously and user name ec2-user.

    Note: If you need assistance connecting to an EC2 instance, see Appendix A:

    Connecting to EC2 Instances.

    15) Confirm that the image_processor.py script and jobs directory are present by running

    the following command:

    ls

    14) You should receive the following output:

    image_processor.py jobs

    Note: It can take a few minutes for all of the commands you issued in the User Data field to

    run. If the items are not present, periodically try again.

    At this point, you have customized a running EC2 instance that you use as the basis for nodes

    in your batch processing cluster.

    CREATE AN AMI FROM YOUR BATCH PROCESSING INSTANCE

    In this section you use the AWS Management Console to create an Amazon Machine Image

    (AMI) from the running batch processing instance you created in the previous section. By

    creating an AMI from the running instance you configured, you can launch many more

    identically configured instances quickly and easily.

    1) Locate your instance in the EC2 Management Console. 2) Right-click your instance and choose Create Image (EBS AMI). 3) For Image Name type Worker Image (and optionally enter a Description). 4) Accept the remaining default values and click Yes, Create and then click Close.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 10

    Note: Your SSH session may be temporarily stopped this is normal.

    5) Click Images > AMIs to view the status of your AMI (it should change from pending to available).

    6) Copy the value in the AMI ID column you will need it later.

    After your AMI is ready, you can use it to launch pre-configured worker nodes in your batch

    processing cluster.

    CREATE TWO SQS TASK QUEUES

    In this section, you use the AWS Management Console to create two SQS queues to hold

    input and output tasks. You will eventually dispatch work via the input queue, and view the

    results provided by your worker nodes in the output queue.

    1) Choose Services > All AWS Services > SQS. 2) Confirm that the region shown in the top-right corner is the same region you used for

    EC2. 3) Click Create New Queue. 4) In the Queue Name field, type: input-queue. 5) Change Default Visibility Timeout to 60 seconds. 6) Accept the remaining default values and click Create Queue. 7) Repeat the previous steps to create an output task queue named: output-queue.

    Note: Remember the names of your input and output queues you will need them later.

    CREATE AN S3 BUCKET

    In this section you create an S3 bucket to hold the output your worker nodes create.

    1) Choose Services > All AWS Services > S3. 2) Click Create Bucket. 3) In the Bucket Name field, type image_bucket[your initials] and click Create. Note:

    Replace [your initials] with the first letter of your first and last name. If that name is already in use, choose another unique name for your bucket.

    Important: Use only lowercase letters for your bucket name.

    Note: Remember the name of your bucket you will need it later.

    LAUNCH WORKER NODES

    In this section, you launch the worker nodes in your batch processing environment.

    1) Choose Services > All AWS Services > EC2 (or click the link in the History section).

    2) Click Launch Instance and choose the Classic Wizard.

    3) Click Continue.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 11

    4) Click the My AMIs tab and select the Worker Image you created.

    5) On the Instance Details panel, for Instance Type, choose c1.medium.

    6) Accept the remaining default options and click Continue.

    7) On the Advanced Instance Options panel, in the User Data field, leave as text

    selected and paste (or type) the following code in the window (you may remove the

    comments if you wish). Replace the underlined text in the code with your SQS queue

    names and S3 bucket name:

    #!/bin/sh

    # REPLACE PARAMETERS WITH YOUR QUEUE NAMES AND BUCKET NAME

    INPUT_QUEUE=

    OUTPUT_QUEUE=

    S3_BUCKET=

    # Launch two processes to do work

    /usr/bin/python /home/ec2-user/image_processor.py --input-queue

    $INPUT_QUEUE \

    --output-queue $OUTPUT_QUEUE --s3-output-bucket $S3_BUCKET &

    /usr/bin/python /home/ec2-user/image_processor.py --input-queue

    $INPUT_QUEUE \

    --output-queue $OUTPUT_QUEUE --s3-output-bucket $S3_BUCKET &

    This User Data snippet will launch the two processes for the application that do image

    conversion. This makes optimum use of the two available cores on the

    c1.medium instance. If there were more or fewer cores available, you would adjust the

    number of processes accordingly.

    8) For IAM Role, choose BatchProcessing and click Continue.

    9) At the Storage Device Configuration panel, click Continue.

    10) On the tags panel, for the Name tag, type: Worker Node in the Value column.

    11) Click Continue.

    12) On the Create Key Pair panel, accept the default key pair and click Continue.

    13) On the Configure Firewall panel, for Choose one or more of your existing Security

    Groups, choose BatchMaster and click Continue.

    14) Click Launch then click Close.

    You are now ready to use SQS to schedule jobs and view the output.

    DISPATCH WORK AND VIEW RESULTS

    In this section you use the SQS Management Console to put a message in your SQS input

    queue. Your background nodes expect a newline-delimited list of URLs pointing to images.

    1) Navigate to the SQS Management Console. 2) Right-click your input-queue and click Send a Message.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 12

    3) Paste the following list of image URLs in the window:

    https://us-east-1-aws-training.s3.amazonaws.com/arch-static-assets/static/20120728-DSC01265-L.jpg

    https://us-east-1-aws-training.s3.amazonaws.com/arch-static-assets/static/20120728-DSC01267-L.jpg

    https://us-east-1-aws-training.s3.amazonaws.com/arch-static-assets/static/20120728-DSC01292-L.jpg

    https://us-east-1-aws-training.s3.amazonaws.com/arch-static-assets/static/20120728-DSC01315-L.jpg

    https://us-east-1-aws-training.s3.amazonaws.com/arch-static-assets/static/20120728-DSC01337-L.jpg

    4) Click Send Message then click Close. 5) Click the Refresh button and observe the Messages Available and Messages in Flight

    columns for each queue. Refresh periodically until you see 0 messages in your input queue and 1 message in your output queue.

    If the messages in your queue do not move:

    Ensure that you put the correct queue names and bucket names in the User Data

    field when creating the Worker Node.

    Ensure that your bucket name only has lowercase letters.

    Ensure that your Worker Node uses the BatchProcessing IAM role.

    After your first job has been processed, follow these steps to view the output and open the

    resulting link in a browser:

    1) Right-click your output-queue and click View/Delete Messages. 2) Click Start Polling for Messages. 3) Find your message and click More Details to view the message body. The message

    will contain a link to the output. 4) Select the link, right-click it, and choose Go to to view the montage image.

    Now that you have run one job, try using your input queue to schedule 30 or more jobs. You

    may re-use the same list of URLS that you used in your initial test.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 13

    MONITORING THE CLUSTER

    Next, use the AWS Management Console to monitor the CPU performance of your worker

    node.

    1) Navigate to the EC2 Management Console. 2) Click Instances > Instances. 3) Select your Worker Node from the list. 4) Select the Monitoring tab below. 5) Observe the available metrics, then click the Avg CPU Utilization metric to view more

    detail. 6) Change the Time Range filter to view the previous 6 hours of data.

    CREATE AN ALARM

    CloudWatch Alarms allow you to receive notifications when metrics are at a threshold you

    define. Create an alarm to alert you if your worker nodes CPU is over-utilized.

    1) On the Monitoring tab, click Create Alarm. 2) In the Create Alarm dialog, for With these recipients type your email address. 3) Configure the alarm to fire: Whenever Average of CPU utilization is >= 90% For at

    least 1 consecutive period(s) of 5 minutes.

    Hint: Adjust the value for CPU utilization and observe the graph to see how this alarm might work given your current metric.

    4) For Name this alarm, type: BatchProcessingNotifications.

    5) Click Create Alarm and then click Close. 6) Check your e-mail and click the link to opt-in to the notification subscription.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 14

    CREATE AN SQS ALARM

    In this section you create an SQS alarm to track the number of messages in the input queue.

    1) Navigate to the CloudWatch section of the Management Console. 2) Click Alarms. 3) Click Create Alarm. 4) Create an alarm to receive an e-mail when the number of messages in your input-queue

    is > 10 for (1) 5-minute period.

    Hint: The metric is named ApproximateNumberOfMessagesVisible, and the statistic

    you should use is Sum, not Average. For Action details, choose NotifyMe and validate

    your email address.

    If your messages are not being processed quickly enough, that is a good indicator that you

    should add more nodes to your cluster. In the next section, you create an Auto Scaling group

    to scale out automatically when the SQS Alarm you just created is fired.

    TEST YOUR POLICY

    Add enough messages to your input-queue to force your alarm to fire. Because CloudWatch

    statistics are not updated instantly, it may take a few moments before the alarm is fired and

    can be viewed in the CloudWatch Dashboard. Observe as your instances are scaled out and

    the number of messages in your queue begin to be processed more quickly.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 15

    AUTO SCALING THE CLUSTER

    Auto scaling is commonly configured using the Command Line Interface (CLI). Rather than

    manually installing the CLIs on your laptop, in this lab you use CloudFormation to quickly

    launch an EC2 instance and automatically install and configure all of the AWS CLIs for you.

    LAUNCH A CLI INSTANCE

    Use CloudFormation to launch an EC2 instance and configure all of the AWS command line

    tools (CLIs):

    1) Open the CloudFormation Management Console. 2) Click Create New Stack. 3) For Stack Name type: CLI. 4) In the Provide a Template URL field, paste the following URL:

    https://awsu-arch.s3.amazonaws.com/aux/technical-exercises/install-all-cli/install-all-cli.template

    5) Click Continue. 6) In the KeyName field, type the name of the qwiklab EC2 Key Pair you downloaded

    previously (you do not need to type the extension). 7) Check I acknowledge that this template may create IAM resources and click

    Continue. 8) Click Continue twice and then click Close to create your stack. 9) Periodically refresh the page until the stacks status changes to CREATE_COMPLETE. 10) Select the CLI stack, click the Outputs tab, and copy the DNS name of your instance

    (you need it connect to the instance using SSH).

    You now have an EC2 instance with all of the AWS CLIs installed. In the next section you will use the Auto Scaling CLIs to create and configure auto scaling for your batch processing cluster.

    CREATE AN AUTO SCALING LAUNCH CONFIGURATION

    The first step when using auto scaling is to define exactly how the service should launch an

    EC2 instance. Defining these parameters is done by creating a launch configuration. In this

    section, you will use the auto scaling CLI to create a launch configuration for your batch

    processing cluster.

    CREATE A USER DATA SCRIPT

    Because your launch configuration will need user data to automatically launch your image

    processing script, you should create a file containing that script before creating the launch

    configuration.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 16

    1) Copy the following script to a text editor and replace the underlined parameters with

    your queue names and bucket name (this is the same script used for your Worker

    Node):

    #!/bin/sh

    # REPLACE PARAMETERS WITH YOUR QUEUE NAMES AND BUCKET NAME

    INPUT_QUEUE=

    OUTPUT_QUEUE=

    S3_BUCKET=

    # Launch two processes to do work

    /usr/bin/python /home/ec2-user/image_processor.py --input-queue $INPUT_QUEUE \

    --output-queue $OUTPUT_QUEUE --s3-output-bucket $S3_BUCKET &

    /usr/bin/python /home/ec2-user/image_processor.py --input-queue $INPUT_QUEUE \

    --output-queue $OUTPUT_QUEUE --s3-output-bucket $S3_BUCKET &

    2) Log in to your new CLI instance using previous steps as a guide. 3) Use vi or nano (use nano if you are unfamiliar with vi) to create a file to hold your user

    data. The following command creates the file using nano:

    nano user-data.txt

    4) Type or paste your user data script. 5) When you are finished, in nano, press CTRL+X to exit, and press Y, followed by Enter

    when prompted to save the file.

    CREATE A LAUNCH CONFIGURATION

    1) Edit the following command (using a text editor) and replace the underlined parameters

    to include a name for your launch configuration, your Security Group, your AMI ID and

    your Key Pair name:

    as-create-launch-config \

    --group \

    --image-id \

    --key \

    --user-data-file user-data.txt \

    --instance-type c1.medium \

    --iam-instance-profile BatchProcessing

    2) Paste (or type) the edited command and execute it. Note: If you type the command, you do not need the closing \ at the end of each line.

    3) You should receive the message: OK-Created launch.

    Note: Remember the name of your launch configuration you need it later.

    CREATE AN AUTO SCALING GROUP

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 17

    Now that your launch configuration is created, create an auto scaling group to automatically

    launch your worker nodes.

    1) Copy the following command into a text editor and replace the underlined parameters to

    include a name for your group, the name of your launch configuration, and the

    availability zone of your worker node:

    as-create-auto-scaling-group \

    --launch-configuration \

    --tag "k=Name, v=Worker-Node" \

    --tag "k=Department, v=7500" \

    --tag "k=Application, v=image-processor" \

    --min-size 1 \

    --max-size 5 \

    --availability-zones

    2) Paste (or type) the edited command and execute it.

    3) You should receive the message: OK-Created AutoScalingGroup.

    Note: Remember the name of your Auto Scaling Group you need it later.

    If you receive the message The availability zone is not legal then the availability

    zone you entered is not available for your account. Edit the command to reference the

    availability zone of your worker node.

    4) Open the EC2 Management Console and look for the instance launched by auto scaling

    with the name you specified (Worker-Node).

    CREATE AN AUTO SCALING POLICY

    Now that your auto scaling group is created, create an auto scaling policy that will add capacity

    to your group. You ultimately configure CloudWatch to execute this policy automatically.

    1) Copy the following command into a text editor and replace the underlined parameters to

    include a name for your policy and the name of your auto scaling group:

    as-put-scaling-policy \

    --auto-scaling-group \

    --adjustment 1 \

    --type ChangeInCapacity

    2) Paste (or type) the edited command and execute it. 3) You should receive an arn for the policy such as:

    arn:aws:autoscaling:us-east-1:424012728564:scalingPolicy:526ae925-

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 18

    f46f-4384-80f5-

    d5f1a9353b90:autoScalingGroupName/cliautoscale:policyName/workerpolicy

    Note: Remember the name of your auto scaling policy, you need it later.

    EXECUTE THE AUTO SCALING POLICY USING CLOUDWATCH

    Now that your auto scaling policy is created, locate the CloudWatch alarm for SQS messages

    you created previously and have it execute the policy you just created.

    1) Navigate to the CloudWatch Management Console and click Alarms.

    2) Select the alarm you created earlier for SQS messages and click Modify.

    3) On the Configure Actions panel, add an action to execute the policy you just created

    for your auto scaling group.

    TEST YOUR POLICY

    Add enough messages to your input queue to force your alarm to fire. Observe as your

    instances are scaled out and the number of messages in your queue begin to be processed

    more quickly. You may need to modify some settings in order to observe the proper

    functionality.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 19

    CONCLUSION

    Congratulations! You now have successfully:

    Bootstrapped an EC2 instance via UserData and CloudInit.

    Created an AMI from a running instance.

    Used the AWS Management Console to create an SQS queue.

    Used the AWS Management Console to pass messages to, and read messages from,

    an SQS queue.

    For feedback, suggestions and corrections to this lab, please email aws-course-

    [email protected].

    END LAB

    1) Sign-out of the AWS Management Console.

    2) Close your Remote Desktop Client session.

    3) Click the End Lab button in qwikLAB.

    3) Give the lab a rating, optionally type a Comment, and click Submit. You may also close

    the dialog if you do not wish to provide feedback.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 20

    APPENDIX A: CONNECTING TO EC2 INSTANCES

    If you are unfamiliar with connecting to EC2 instances, here are some instructions.

    Important: Proceed to the next section entitled CONNECT TO YOUR EC2 INSTANCE VIA

    SSH (OS X AND LINUX) if you are using a Linux-based laptop to perform this exercise. This

    section is only for Microsoft Windows users. There is no need for you to perform the steps in

    each section. Only complete the section that matches your operating system.

    DOWNLOAD PUTTY

    1) Download PuTTY to a location of your choice using the following URL (unless you

    already have PuTTY):

    http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

    DOWNLOAD YOUR EC2 KEY PAIR PRIVATE KEY FILE

    4) In your browser, switch to the qwikLAB tab.

    5) In the Connection Details section, copy the Endpoint name to the clipboard or paste it

    into a text document. You will need this to connect to the image in PuTTY. Note the

    ec2-user user name preceding the @ symbol. If you place just the host name in the

    PuTTY client, you must log in as ec2-user.

    6) Download the qwikLAB provided EC2 Key Pair Private Key file in the PuTTY

    compatible PPK format. From the EC2 Key Pair Private Key drop-down list, click the

    Download PPK option.

    7) Save the file to your \Downloads directory (the default), or move it to another directory

    of your choice.

    CONNECT TO THE EC2 INSTANCE USING SSH AND PUTTY.

    1) Double-click putty.exe (your downloaded copy or your existing copy).

    2) Paste the Endpoint string you copied to your clipboard into the Host Name input in

    Putty (using CTRL+V).

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 21

    3) Expand the SSH category by double-clicking it.

    4) Select the Auth category by clicking it (not the + symbol next to it).

    5) Click Browse and locate the PPK file (ending in .ppk) in your \Downloads directory (or

    the location you chose).

    6) Select the file and click Open.

    7) In the PuTTY client, click Open.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 22

    8) When prompted, click Yes to allow PuTTY to cache the servers host key.

    CONNECT TO YOUR EC2 INSTANCE VIA SSH (OS X AND LINUX)

    Important: Only complete this section if you are using a Macintosh OS X or Linux-based

    laptop to perform this exercise. This section is only for Linux users. There is no need for you to

    perform the steps in each section. Only complete the section that matches your operating

    system.

    DOWNLOAD YOUR EC2 KEY PAIR PRIVATE KEY FILE

    1) In your browser, switch to the qwikLAB tab.

    2) Download the qwikLAB provided EC2 Key Pair Private Key file in PEM format. From

    the EC2 Key Pair Private Key drop-down list, click the Download PEM option.

    3) Save the file to your \Downloads directory (or some other directory of your choice).

    4) Copy the entire content of the Endpoint field to your clipboard (using CTRL+C).

    CONNECT TO THE EC2 INSTANCE USING THE OPENSSH CLI CLIENT

    1) Open the Terminal application.

    2) Enter the commands below.

    In the first command, change the path and file name (~/Downloads/qwiklab-l33-

    5018.pem) to the location of the .pem file you downloaded.

  • Architecting on AWS Exercise 1 - Creating a Virtual Private Cloud

    v. 2.5 Copyright 2012-2013, Amazon Web Services, All Rights Reserved. Page 23

    In the second command, replace username@hostname (ec2-user@ec2-23-22-87-

    238.compute-1.amazonaws.com) with the information you copied from the

    Endpoint field.

    chmod 600 ~/Downloads/qwiklab-l33-5018.pem

    ssh i ~/Downloads/qwiklab-l33-5018.pem ec2-user@ec2-23-22-

    87-238.compute-1.amazonaws.com