System Architectureby Rex Chung - Founder

What we do?

• Video conversion API

Ankoder AmazonS3

Youtubedownloader

Custom Video

Channel

What we do?

• Video conversion API

Ankoder AmazonS3

Youtubedownloader

Custom Video

Channel

request

What we do?

• Video conversion API

Ankoder AmazonS3

Youtubedownloader

Custom Video

Channel

request

postback

Our status

Our status

• 3 people, 3 months from October 2007

Our status

• 3 people, 3 months from October 2007

• Demo usage - free.ankoder.com

Our status

• 3 people, 3 months from October 2007

• Demo usage - free.ankoder.com

• Rails Plugin released

Our status

• 3 people, 3 months from October 2007

• Demo usage - free.ankoder.com

• Rails Plugin released

• Ruby Gem released

Our status

• 3 people, 3 months from October 2007

• Demo usage - free.ankoder.com

• Rails Plugin released

• Ruby Gem released

• Still a lot to do...

Current Architecture

• Slicehost - VPS

• FFmpeg - video conversion

• Ruby Reliable-messaging

• Ruby drb workers on Amazon EC2

• Amazon S3 storage

Current Architecture

API Rails ApplicationMySQL

Reliable-messaging

Download queue

Conversion queue

EC2 EC2

ActiveResource

S3

Current Problems

Current Problems

• Uploads cannot scale

Current Problems

• Uploads cannot scale

• Bandwidth is doubled.

Current Problems

• Uploads cannot scale

• Bandwidth is doubled.

• File uploaded will have a delay

Current Problems

• Uploads cannot scale

• Bandwidth is doubled.

• File uploaded will have a delay

• Slicehost VPS is not fault tolerant

Moving to EC2

Problems with EC2

• No persistent storage

• No permanent IP

• Server up time is not guaranteed

Moving to EC2

Benefits with EC2

• Delay to S3 is small (700KB/s)

• Scalable Upload

• Scalable download by hiding S3

• More reliable network communication

EC2 Architecture

RAILSMySQL cluster

RAILSMySQL cluster

RAILSMySQL cluster

SQS

S3

Rubyworker

Rubyworker

DNS

Difficulties

• Bandwidth of each instances are limited

• No fixed IP means can only use DNS to route to different server.

• Since server uptime is not guaranteed, DNS config will need to be automatically updated.

• DNS may be the single point of failure.

Improvements from S3

Improvements from S3

• Direct upload to S3, but cannot handle custom form fields.

Improvements from S3

• Direct upload to S3, but cannot handle custom form fields.

• Increase packet buffer size to increase transfer speed.

Improvements from S3

• Direct upload to S3, but cannot handle custom form fields.

• Increase packet buffer size to increase transfer speed.

• SQS pricing now 100 times cheaper From $0.10/1,000 messages to $0.01/10,000 requests

Some links

• Rails plugin - http://code.google.com/p/ankoder/

• Ruby Gem - http://ankoder.rubyforge.org/

• Doc - http://doc.ankoder.com

• Youtube Downloader - http://free.ankoder.com

• Beta - http://beta.ankoder.com