streaming benchmark

16
Streaming Benchmark Vinaya M S Insight Data Science

Upload: vinaya-m-s

Post on 11-Feb-2017

273 views

Category:

Data & Analytics


0 download

TRANSCRIPT

StreamingBenchmark

Vinaya MSInsightDataScience

m4.large:3

r3.large:4

r3.large:4

r3.large:1

Howmanytweetsareprocessed/second?

Insertread_ts Insertwrite_ts

100tweets/second

Subsetof100tweets/second

Latency:

∑ (𝑤𝑟𝑖𝑡𝑒() − 𝑟𝑒𝑎𝑑())./010234./056786 /total_tweets_processed

winStart:windowstarttime;winEnd:windowEndtime

RESULTS

Whataboutstorm?

• Totalnumberoftweetsprocessedinstormaremore.•~41000in160sec(Flink:~18500in160sec)

•Noticedlowthroughputandhighlatencyinstorm.

•Thisisnottruealways.Performancetuningisrequired.

FewtuningsIconsidered

§Numberofcomputebolts

§Consumerbolts.

§Javaheapsize.Playsanimportantrole

§Tried2ofthegroupings.

AboutMe

3yearsexperience

MS,ComputerScience

Enjoy:TableTennisCooking

Thankyou😊

Storm:

Flink:

Eachtaskslotintaskmanagercanrunonepipelineofparallel task.