jeeconf. vanilla java
TRANSCRIPT
![Page 1: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/1.jpg)
Vanilla java or handling 10k req/sec per core
![Page 2: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/2.jpg)
Dmitriy DumanskiyBlynk
Co-Founder / Java / Android
Java blog : http://habrahabr.ru/users/doom369/topicsDOU : http://dou.ua/users/DOOM/articles/
![Page 3: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/3.jpg)
Experience
mGage - 3k req/secmobclix - 6k req/secxxxx - 11k req/sec
![Page 4: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/4.jpg)
Visa
Load : ~2000 req/sec
![Page 5: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/5.jpg)
Twitter : new tweets
Load : ~6000 req/sec
![Page 6: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/6.jpg)
Facebook messenger
Load : 300k req/sec
![Page 7: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/7.jpg)
![Page 8: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/8.jpg)
IoT Market
● 4.4 billion devices right now (70$ b)● 25 billion devices per 2020 (270$ b)
![Page 9: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/9.jpg)
Makers problem
+ = ?
![Page 10: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/10.jpg)
Problem
● Thousands of connections● Keep-alive● Security● Pulling / polling● Simplicity
![Page 11: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/11.jpg)
IoT Protocols
HTTP, MQTT, XMPP, AMQP, CoAP
![Page 12: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/12.jpg)
![Page 13: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/13.jpg)
Minimum packet size
● HTTP - 26 bytes (~70 bytes)● MQTT - 2 bytes● Blynk - 3 bytes
![Page 14: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/14.jpg)
Blocking IO
connection new Socket() new Thread() read / writeServer Socket
![Page 15: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/15.jpg)
Blocking IO
● 256kb stack● 1k threads == 250 MB of RAM● Сontext switching
![Page 16: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/16.jpg)
Non-Blockingnew Socket()
read / write
Selector
Thread
new Socket()
read / write
new Socket()
read / write
![Page 17: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/17.jpg)
Non-Blocking
● Few threads. Usually - n cores x 2.● No context switching● No memory consumption
![Page 18: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/18.jpg)
Non-blocking frameworks● Apache MINA● Grizzly● Xnio● Vert.x● Akka● Netty
![Page 19: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/19.jpg)
Non-blocking frameworks● Apache MINA● Grizzly● Xnio● Vert.x● Akka● Netty
![Page 20: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/20.jpg)
Non-blocking frameworks● Apache MINA● Grizzly● Xnio● Vert.x● Akka● Netty
![Page 21: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/21.jpg)
Non-blocking frameworks● Apache MINA● Grizzly● Xnio● Vert.x● Akka● Netty
![Page 22: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/22.jpg)
Non-blocking frameworks● Apache MINA● Grizzly● Xnio● Vert.x● Akka● Netty
![Page 23: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/23.jpg)
Netty
![Page 24: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/24.jpg)
Netty cons
Netty is hard. 2-3 weeks onboarding
![Page 25: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/25.jpg)
Netty cons
Bad documentation
![Page 26: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/26.jpg)
Netty cons
Too many abstractions
![Page 27: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/27.jpg)
Netty cons
Easy shot in the foot
![Page 28: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/28.jpg)
Netty cons
ByteBufin.readBytes(length) - memory leak
in.readSlice(length) - ok
![Page 29: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/29.jpg)
Netty cons
Still has bugs, 300 issues
![Page 30: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/30.jpg)
Netty pros
Full control
TCP_FASTOPEN, TCP_MD5SIG, TCP_CORK, TCP_KEEPCNT, TCP_KEEPIDLE, TCP_KEEPINTVL,
SPLICE
![Page 31: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/31.jpg)
Netty pros
Supports everything
TCP, UDP, UDT, SCTP, HTTP, HTTPS, HTTP/2, SPDY, Memcached, MQTT, etc
![Page 32: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/32.jpg)
Netty pros
A lot of default Handlers
![Page 33: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/33.jpg)
Netty pros
Fast
![Page 34: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/34.jpg)
Netty pros
Super quick community help
![Page 35: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/35.jpg)
Blynk architecture
![Page 36: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/36.jpg)
![Page 37: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/37.jpg)
https://github.com/blynkkk/blynk-server
Open source
https://github.com/blynkkk/blynk-library
![Page 38: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/38.jpg)
Blynk stats
● ~2000 of hardware always online● ~300 users always online● ~1000 local Blynk Server installations● ~3 Mbps in traffic● ~7 billions requests per month
![Page 39: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/39.jpg)
Blynk load
● 6% CPU with 2500 req/sec on 4 cores● 10k req/sec per core● 90% of requests - 3 handlers● Only 3% requests are HTTP
![Page 40: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/40.jpg)
Why so fast?
![Page 41: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/41.jpg)
No blocking IO
Request
BE
Blocking IO Workers
Response
![Page 42: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/42.jpg)
Batches
Request
BE
Response
Collector1 min batch File System
![Page 43: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/43.jpg)
What is your DB?
try (Writer writer = Files.newBufferedWriter(file)) { writer.write(user.toString());}
![Page 44: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/44.jpg)
![Page 45: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/45.jpg)
No synchronization
Client 1connect
Selector Thread 1assign
Client 2connect
Selector Thread 2assign
Client 2login
Logic Thread 1reregister
![Page 46: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/46.jpg)
No GC pressure
ctx.write(new Response(mId));vs
ByteBuf buf = ctx.alloc().directBuffer(1);buf.writeByte(mId);
ctx.write(buf);
![Page 47: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/47.jpg)
In memory
Set<User> users = JsonParser.parseUsers(dir);
+ 1 min batches
![Page 48: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/48.jpg)
Native transports
● Native epoll● Native OpenSSL
![Page 49: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/49.jpg)
![Page 50: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/50.jpg)
Version 1
Blynk monolith
80$
![Page 51: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/51.jpg)
Version 1. Tech stack
● Vanilla Java 8● Netty (HTTP, WebSockets, TCP/IP)
![Page 52: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/52.jpg)
Version 1. Pros
Simple single jar deploymentjava -jar server.jar
100ms start
![Page 53: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/53.jpg)
Version 1. Pros
Simple backup script
![Page 54: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/54.jpg)
Version 1. Pros
Single JVM, no cross-JVM shared state
![Page 55: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/55.jpg)
Version 1. Pros
Entry level
![Page 56: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/56.jpg)
Version 1. Cons
Single point of failure, single datacenter
![Page 57: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/57.jpg)
Version 1. Cons
Latency is bad for distant locations
![Page 58: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/58.jpg)
Version 1. Cons
Need to drop connections on redeploy
![Page 59: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/59.jpg)
Version 2
ClientGet DNS record EUROPE
USA
ASIAGeo DNS
50$
![Page 60: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/60.jpg)
Version 2
● Improved latency● No common storage anymore
![Page 61: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/61.jpg)
Version 2
USA
Clients
DC 1
EU
Clients
DC 2
ASIA
Clients
DC 3
![Page 62: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/62.jpg)
Version 3
USA
Clients
DC 1
EU
Clients
DB
ASIA
Clients
DC 3
![Page 63: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/63.jpg)
Version 3. DB Cassandra
Node 1
Node 3
Node 2
Node 3
Node 2
Node 1
Node 3
Node 2
DC 1 DC 2
Node 1
15$
![Page 64: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/64.jpg)
Version 3. DB Cassandra
● No typical batches● Minimum requirement 1GB of RAM● Performs poorly on low-end VMs● Much slower than Postgres
![Page 65: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/65.jpg)
Version 3. Final
USA
Clients
DC 1
EU
Clients
Postgres
ASIA
Clients
DC 365$
![Page 66: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/66.jpg)
Version 3. Final
● 60k req/sec for 65$ (25x for current load)● + 10x growth to 20 CPU● Samsung cloud - 1 req/sec per 6$
![Page 67: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/67.jpg)
![Page 68: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/68.jpg)
![Page 69: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/69.jpg)
![Page 70: JEEConf. Vanilla java](https://reader033.vdocument.in/reader033/viewer/2022051122/58ef44b01a28ab730e8b4611/html5/thumbnails/70.jpg)
Q & A