how twitter works (arsen kostenko technology stream)

36
HOW TWITTER WORKS The Challenge

Upload: lviv-it-arena

Post on 14-Feb-2017

181 views

Category:

Technology


5 download

TRANSCRIPT

Page 1: How Twitter Works (Arsen Kostenko Technology Stream)

HOW TWITTER WORKS

The Challenge

Page 2: How Twitter Works (Arsen Kostenko Technology Stream)

… and more!

How Twitter Works

Page 3: How Twitter Works (Arsen Kostenko Technology Stream)

Tweets

How Twitter Works

Just 140 characters???

Page 4: How Twitter Works (Arsen Kostenko Technology Stream)
Page 5: How Twitter Works (Arsen Kostenko Technology Stream)

#oscars

Page 6: How Twitter Works (Arsen Kostenko Technology Stream)

#worldcup

Page 7: How Twitter Works (Arsen Kostenko Technology Stream)
Page 8: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Tweetsauthor

mention

actions

card

counts

Page 9: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Let’s Draw Twitter’s Architecture!

clients routing API logic storage

Page 10: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Twitter Architecture: The Old Days (~2010)

clients routing logic storage

MySQLCluster

TweetDeck

Twitter for iOS

Twitter for Android

twitter.com

Twitter API Clients

API

The Monorail

(Ruby)The Monorail

(Ruby)The Monorail

(Ruby)

Page 11: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Twitter Architecture: The Old Days (~2010)

Page 12: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Twitter Architecture: Microservices

clients routing logic storage

Macaw-SwiftMacaw-Swift

User StoreGizmoduck(User Service)

Web API

TFE (Twitter Front End)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

Tweet Store

Macaw-Tweets

Search Service

Direct Messages Service

Macaw-Users

Macaw-Search

Macaw-DM

Graph Store

Tweet Graph Store

Direct Messages Store

TweetDeck

Twitter for iOS

Twitter for Android

twitter.com

Twitter API Clients

Distributed Key-Value Store

API

Macaw-SwiftMacaw-SwiftTweet API

Macaw-SwiftMacaw-SwiftUsers API

Macaw-SwiftMacaw-SwiftSearch API

Macaw-SwiftMacaw-SwiftMessages API

Macaw-SwiftMacaw-SwiftTweet Service

Tweetypie (Tweet Service)

Macaw-SwiftMacaw-SwiftUser Service

Tweetypie (Tweet Service)

Macaw-SwiftMacaw-SwiftTimeline Service

Tweetypie (Tweet Service)

Macaw-SwiftMacaw-SwiftSocial Graph Service

Tweetypie (Tweet Service)

Macaw-SwiftMacaw-SwiftSearch Service

Tweetypie (Tweet Service)

Macaw-SwiftMacaw-SwiftDirect Messages Service

Macaw-UsersMacaw-SwiftMacaw-SwiftTimeline API

Page 13: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Twitter Architecture: Microservices

10-100x performance

hundreds of million tweets/day

Page 14: How Twitter Works (Arsen Kostenko Technology Stream)

143,200 peak tweets/sec

Page 15: How Twitter Works (Arsen Kostenko Technology Stream)
Page 16: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Twitter Clients

clients routing logic storage

Manhattan (Distributed DB)

Gizmoduck(User Service)

Web API Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

TBird (Tweet Store)

Tweets API

Search Service

Direct Messages Service

Users API

Search API

Messages API

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

TFE (Twitter Front End)

HTTPS

TweetDeck

Twitter for iOS

Twitter for Android

twitter.com

Twitter API Clients

Manhattan (Distributed DB)

API

Timeline API

Page 17: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Twitter Front End (TFE)

clients routing logic storage

Manhattan (Distributed DB)

Gizmoduck(User Service)

TFE (Twitter Front End)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

TBird (Tweet Store)

Search Service

Direct Messages Service

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

HTTPS HTTP

TweetDeck

Twitter for iOS

Twitter for Android

twitter.com

Twitter API Clients

Manhattan (Distributed DB)

API

Web API

Tweets API

Users API

Search API

Messages API

Timeline API

Page 18: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Presentation Layer Services

clients routing logic storage

Manhattan (Distributed DB)

Gizmoduck(User Service)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

TBird (Tweet Store)

Search Service

Direct Messages Service

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

HTTP

TweetDeck

Twitter for iOS

Twitter for Android

twitter.com

Twitter API Clients

HTTPS

Web API

Tweets API

Users API

Search API

Messages API

TFE (Twitter Front End)

Thrift

Manhattan (Distributed DB)

API

Timeline API

Page 19: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Application Logic Services

clients routing logic storage

Manhattan (Distributed DB)

TBird (Tweet Store)

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

HTTP

TweetDeck

Twitter for iOS

Twitter for Android

twitter.com

Twitter API Clients

HTTPS

TFE (Twitter Front End)

Thrift

Web API

Tweets API

Users API

Search API

Messages API

Gizmoduck(User Service)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

Search Service

Direct Messages Service

Manhattan (Distributed DB)

API

Timeline API

Page 20: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Storage Services

clients routing logic storage

HTTP

TweetDeck

Twitter for iOS

Twitter for Android

twitter.com

Twitter API Clients

HTTPS

TFE (Twitter Front End)

Thrift

Gizmoduck(User Service)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

Search Service

Direct Messages Service

Manhattan (Distributed DB)

TBird (Tweet Store)

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

Manhattan (Distributed DB)

API

Web API

Tweets API

Users API

Search API

Messages API

Timeline API

Page 21: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Write Path Example: Tweet

clients routing logic storage

HTTP

TweetDeck

Twitter for iOS

Twitter for Android

Twitter API Clients

HTTPS

TFE (Twitter Front End)

Thrift

Manhattan (Distributed DB)

TBird (Tweet Store)

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

Manhattan (Distributed DB)

twitter.com

API

Gizmoduck(User Service)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

Search Service

Direct Messages Service

Web API

Tweets API

Users API

Search API

Messages API

Timeline API

Page 22: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Write Path Example: Tweet

clients routing logic storage

HTTP

TweetDeck

Twitter for iOS

Twitter API Clients

HTTPS Thrift

Manhattan (Distributed DB)

TBird (Tweet Store)

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

Manhattan (Distributed DB)

TFE (Twitter Front End)

API

Twitter for Android

twitter.com

Gizmoduck(User Service)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

Search Service

Direct Messages Service

Web API

Tweets API

Users API

Search API

Messages API

Timeline API

Page 23: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Write Path Example: Tweet

clients routing logic storage

HTTP

TweetDeck

Twitter for iOS

Twitter API Clients

HTTPS Thrift

Manhattan (Distributed DB)

TBird (Tweet Store)

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

Manhattan (Distributed DB)

TFE (Twitter Front End)

API

Twitter for Android

twitter.com

Gizmoduck(User Service)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

Search Service

Direct Messages Service

Web API

Tweets API

Users API

Search API

Messages API

Timeline API

Page 24: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Write Path Example: Tweet

clients routing logic storage

HTTP

TweetDeck

Twitter for iOS

Twitter API Clients

HTTPS Thrift

Direct Messages Service

Manhattan (Distributed DB)

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

Manhattan (Distributed DB)

TFE (Twitter Front End)

Gizmoduck(User Service)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

Search Service

API

Twitter for Android

twitter.com

TBird (Tweet Store)

Web API

Tweets API

Users API

Search API

Messages API

Timeline API

Page 25: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Write Path Example: Tweet

clients routing logic storage

HTTP

TweetDeck

Twitter for iOS

Twitter API Clients

HTTPS Thrift

Direct Messages Service

Manhattan (Distributed DB)

TBird (Tweet Store)

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

Manhattan (Distributed DB)

TFE (Twitter Front End)

Gizmoduck(User Service)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

Search Service

API

Twitter for Android

twitter.com

Web API

Tweets API

Users API

Search API

Messages API

Timeline API

Page 26: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Building a TimelineConsider user timelines for three different Twitter users

Page 27: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Building a TimelineIf you follow these three users...

time

Page 28: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Building a Timeline… we can build your Twitter timeline as a list of tweets in reverse chronological order:

time

Although we can also rely on other pieces of information to create tweet timelines that are more relevant for our users.

Page 29: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Building a Timeline

clients routing logic storage

TweetDeck

Twitter for iOS

Twitter for Android

Twitter API Clients

twitter.com TFE (Twitter Front End)

Gizmoduck(User Service)

Tweetypie (Tweet Service)

Timeline Service

Social Graph Service

Search Service

Direct Messages Service

Manhattan (Distributed DB)

TBird (Tweet Store)

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

Manhattan (Distributed DB)

API

Web API

Tweets API

Users API

Search API

Messages API

Timeline API

Page 30: How Twitter Works (Arsen Kostenko Technology Stream)

Twitter Architecture - How Twitter Works

Building a Timeline

logic storage

Tweetypie (Tweet Service)

Timeline Servicec

Manhattan (Distributed DB)

TBird (Tweet Store)

Flock (Graph Store)

TFlock (Tweet Graph Store)

DMBird (Direct Msg Store)

Haplo (Redis cluster)

API

Web API

Tweets API

Users API

Search API

Messages API

Timeline API

FanoutService

Page 31: How Twitter Works (Arsen Kostenko Technology Stream)

TWITTER SERVICE STACK

How Twitter Works

Page 32: How Twitter Works (Arsen Kostenko Technology Stream)

Service StackMost services at Twitter run on the following stack:

Hardware

Twitter Service Stack - How Twitter Works

Distributed Resource Manager

Job Scheduler

Runtime

RPC Framework

Operating System

Network Server

REST Framework

Hardware

Mesos

Aurora

Scala + JVM

Finagle

Linux

TwitterServer

Finatra

Page 33: How Twitter Works (Arsen Kostenko Technology Stream)

Mesos and Aurora

Distributed resource manager

Twitter Service Stack - How Twitter Works

Hardware

OS

Hypervisor

OS

Hardware

OS

Mesos

Job

Job

Job

Job

Job

Job

OS

Job

OS

Job

OS

Job

OS

Job

Job scheduler

Page 34: How Twitter Works (Arsen Kostenko Technology Stream)

Scala, Finagle and Finatra

Twitter Service Stack - How Twitter Works

An object-oriented, functional programming language

Compiles to Java bytecode

Used by most production services at Twitter

An extensible RPC system for high-concurrency clients and

servers

Supports HTTP, Thrift, and other protocols

A framework for fast and testable HTTP services

Page 35: How Twitter Works (Arsen Kostenko Technology Stream)

#oneteamMike Cvet @mikecvet

Moses Nakamura @mnnakamura

Saša Gargenta @sasa

Anna Sulkina @asulkina

Ricardo Cervera-Navarro @ricardoc

John Zeman @zemanJulie Dilger @JulieDil

Nisha Antony @nantony345

Page 36: How Twitter Works (Arsen Kostenko Technology Stream)

Questions?