f - baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `eÑ t m,x ¡ 0 Ä Ï ua× z...

12
2.1 2.2 2.3 3 7 2.1 2.1 2.1 2.1 2.1 Elastic Compute Service ECS Open Storage Service OSS Open Table Service OTS Relational Database Service RDS Open Data Processing Service ODPS Aliyun Cloud Engine ACE Web

Upload: others

Post on 05-Jun-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

8 ��

2.1 2.2

2.33 7

2.1

2.1 2.12.1

2.1

Elastic Compute Service ECS Open Storage Service OSS

Open Table Service OTS Relational Database Service RDSOpen Data Processing Service ODPS

Aliyun Cloud Engine ACE Web

��

Page 2: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

2 �� 9

2.1

2.2

POSIXAPI

Online Service Batch Processing Job

1

Nuwa Coordination Service

Page 3: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

10 ��

Paxos [15] Server

UNIX

/ Root entry / /UNIX

Value 1MB

Publish/Subscribe One

Publisher/Many SubscriberMany Publisher/Many Subscriber Distributed Election

server1 nuwa://cluster/ myservice/server1 server2 nuwa://cluster/myservice/server2

nuwa://cluster/myserviceserver1 server2

2

Socket Remote Procedure Call RPC[9]

Kuafu RPC

7×24Gigabyte API RPC

RPC RPC Client URI RPC RPC Server

� TCP tcp://fooserver01:9000 � Nuwa nuwa://nuwa01/FooServer

Page 4: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

2 �� 11

stream TCP Message

std::string std::map string

RPC asynchronous synchronous

� RPC

� RPC

Unix Domain Socket

Kuafu Proxy TCP

TCP

Flow Control

3

Zhongkui

� Authentication

� Capability Authorization

Capability

Capability

Capability

Capability

Capability

Capability Capability

Pangu

Page 5: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

12 ��

� PB 1PB=1000TB � Metadata

3

� I/O �

Master Chunk Server

Paxos [15] 64MB

64MB

3 Replica3

3

OSS OTS

ECS

Master

Page 6: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

2 �� 13

Fuxi

Service Job

MapReduce[12]

Map-Reduce-Merge

Master/Worker MasterWorker Plan Worker Worker

Master Status Master MasterMaster

Master Job/Service FailoverMaster Job/Service

CPU Tubo Master

Master WorkerMaster Job/Service Master Master

Worker Worker

Service Master Service MasterService Master Service Master Service Worker

Failover Job Master Job Master Job

Master Job Master JobTask Task Task Worker Task Master

1

Service Service Master Role

Page 7: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

14 ��

Service Worker Service Master MasterMaster Child Master Service

Master Service Service Worker

Service Worker PartitionService Worker

Service Worker

ServiceLoad Unload

Split Migrate

2

Job Directed Acyclic Graph DAG Task Pipeline

Task Pipeline Task Task Job Master Job description

DAG DAG Task Task Task MasterMaster Task

Worker Instance Task WorkerTask Worker

Job InstanceWorker Snapshot Task

Task n Instance m Taskn�m Instance

1

Shennong

Master Inspector Agent

� Master AgentSubscription Master

Page 8: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

2 �� 15

� InspectorAgent

� Agent Agent InspectorAgent Master Master

Subscription Agent

I/O Master Subscription

MonitorService AnalysisService

� MonitorService Agent

� AnalysisServiceI/O

2

Dayu

P2P

Page 9: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

16 ��

2.3

ECSOSS OTS RDS ODPS

ACE �

OTS Partitioning

ECSECS ECS

IT

IT

Page 10: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

2 �� 17

Ping TCP UDP DNS POP3 SMTP FTPECS

DDOS

� Server Load Balancer SLB IP

SLBSLB

SLB

OSSOSS

RESTful API RESTful API B.1Web

OSS

OSS Object ObjectObject Metadata OSS Object

OSSOSS Object OSS Bucket Bucket Object

Bucket OSS Object BucketBucket OSS Bucket

Bucket OSS Bucketpublic-read-write public-read private

OTS

Page 11: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

18 ��

OTS

OTSRESTful API Web Portal

OTS

OTS OTS

OTS

RDS WebMySQL SQL Server

RDS

RDS

RDS RDS

RDS

RDS

ODPS RESTful API SQL MapReduce[12]

ODPS PB

ODPS SQL MapReduce API ODPS SQL SQLPB ODPS SQL

Page 12: f - Baiduimages.china-pub.com/ebook3800001-3805000/3800166/ch02.pdf · `EÑ t m,X ¡ 0 Ä Ï uA× z Ö L 5x2Ï4³ ,X Ï u ¤ oA× z á u È à Ê Õ A× ¡ hEó z,X ü4 á u ÄOnline

2 �� 19

SQL API DDLDML ODPS SQL

ODPS SQL ODPS SQL

SQL

ODPS MapReduce Hadoop[19] MapReducePB

Hadoop MapReduce ODPSOut-of-Box ODPS

ACE

ACE PHP Node.js

MySQL Memcache Cron Session StorageACE PHP ACE PHP Runtime PHP

99% ACEACE PHP

ACE APIACE D

ACE