aws cost visualizer
DESCRIPTION
A presentation which is used as GIP participants’ progress report. This presentation will explain how the custom-built AWS cost visualizer works.TRANSCRIPT
AWS COST VISUALIZERHDE GIP - MTS JUNE 2014
Presented by Iskandar Setiadi & Saskya Irena
AWS COST VISUALIZEROVERVIEW
1. Problem Description2. Analysis & Implementation3. Result4. Demonstration
ENORMOUS NUMBER OF PRODUCTS
UNDETAILED BILLING REPORT (UI)
COMPLICATED BILLING REPORT
CSV DETAILS
InvoiceID,PayerAccountId,LinkedAccountId,RecordType,RecordId,ProductName,RateId,SubscriptionId,PricingPlanId
"Estimated" ,"LineItem","21610877191184803585998665","AmazonElastic
"Estimated" ,"LineItem","21942771010558985838334611","AmazonElastic
BUT THERE ARE 8.000.000 (八百万)ENTRIES PER MONTH!
HOW MUCH IS THE EXPENSE FOR:
1. Each Customer2. Each S3 Bucket3. Each Infrastructure4. Each Project (Tapirus, Nobita)5. Other Expenses
- DAILY / MONTHLY -
- INSTANCE, ETC -
SOLUTION
CREATE YOUR OWN TOOL!
IMPLEMENTATIONENVIRONMENT
1. Python 2.6.92. Flask Framework & Redis for Database3. Linux/UNIX based Operating System4. Amazon EC2 - t1.micro -> c3.large5. Github (HDE/aws-cost-visualizer)
FIELDS OF DATA (CUSTOMER / CI /PROJECT)
1. Company Name2. EC2 Instance Reserved3. EC2 Instance Non-Reserved4. EC2 EBS Standard5. EC2 EBS PIOPS6. EC2 Data Transfer7. EC2 Others8. S3 Usage Rate *9. S3 **
10. Others
*= CUSTOMER ONLY
** = CI & PROJECT ONLY
FIELDS OF DATA (S3)
1. Bucket Name2. Total Cost
FIELDS OF DATA (NON-CUSTOMER)
1. Product Name (SQS, etc)2. Total Cost
PARSING CONVENTION
EC2 Instance Reserved: BoxUsage, HeavyUsage
EC2 Instance Non-Reserved: BoxUsage, SpotUsage
EC2 EBS Standard: VolumeUsage, VolumeIOUsage,SnapshotUsage
EC2 EBS PIOPS: VPIOPS, VolumeUsage
EC2 Data Transfer: Regional, In-Bytes, Out-Bytes
Others: Idle EIP, LoadBalancerUsage, Monitoring (Alarm,etc), DataProcessing, etc
26 DAYS STATISTICS
7.220.000++ LINES
1147 SECONDS(19 MINUTES 3 SECONDS)
KEY IDEAS
CHECK THE LATEST 2 MONTHS REPORT
RUN IT ONCE / TWICE PER DAY (11 P.M. / 5 A.M.)
USE SCRIPT / CRON TO HANDLE FAILURE [PRODUCTION]
IF AWS MONTHLY REPORT IS GETTING BIGGER (5 TIMES) -> MIGRATE TO BETTER EC2 INSTANCE
NAMING CONVENTIONS / RESERVEDKEYWORDS
THERE'S NO UNDERSCORE '_' ON S3 BUCKET NAME
PLEASE APPEND KEYWORD LIST AS BRIEF AS POSSIBLE
ASSUMPTIONS
All conventions in AWS Billing Report, Usage Report, andCompany List are constant
But the nature of production environment is alwayschanging, so it's needed to maintain the application
If there's no specified date in AWS Billing Report, thisapplication will map it to 1st date
DEMONSTRATION TIME!
ANY QUESTION?THANKS FOR YOUR ATTENTION!
ありがとう ございました