shoving your application code into the web serverlua - example 1 function allwords() 2 local...

Shoving Your Application Code Into the Web ServerUsing OpenResty and nginx to write high-performance web applications

Nikola Plejic

WebCamp Zagreb, October 6th, 2018

Who Am I?

I dabbling around the web for a whileI GNU/Linux, Rust, Lua, C++, Clojure, PythonI senior engineer, TVbeat (of course we’re hiring)

I mostly HTTP

Who Am I?

I dabbling around the web for a whileI GNU/Linux, Rust, Lua, C++, Clojure, PythonI senior engineer, TVbeat (of course we’re hiring)I mostly HTTP

web server

I “c10k” problemI 30%-40% of the public internet (*)

I (*) - lies, big lies, and statistics

I modular architectureI asynchronous, event-driven model

I “c10k” problem

I 30%-40% of the public internet (*)I (*) - lies, big lies, and statistics

I modular architectureI asynchronous, event-driven model

I “c10k” problemI 30%-40% of the public internet (*)

I (*) - lies, big lies, and statistics

I modular architectureI asynchronous, event-driven model

I “c10k” problemI 30%-40% of the public internet (*)

I (*) - lies, big lies, and statistics

I modular architectureI asynchronous, event-driven model

I “c10k” problemI 30%-40% of the public internet (*)

I (*) - lies, big lies, and statistics

I modular architecture

I asynchronous, event-driven model

I “c10k” problemI 30%-40% of the public internet (*)

I (*) - lies, big lies, and statistics

I modular architectureI asynchronous, event-driven model

modularity

I LDAP -> PostgreSQL -> upstreamI append X-Requested-By header to every HTTP requestI REST -> Protobuf

modularity

I LDAP -> PostgreSQL -> upstream

I append X-Requested-By header to every HTTP requestI REST -> Protobuf

modularity

I LDAP -> PostgreSQL -> upstreamI append X-Requested-By header to every HTTP request

I REST -> Protobuf

modularity

I LDAP -> PostgreSQL -> upstreamI append X-Requested-By header to every HTTP requestI REST -> Protobuf

what if we could script these nginx modules?

Lua

I small, dynamic programming language

I think: JavaScript, Scheme, TclI ANSI C: highly portable and embedableI LuaJIT: probably the best dynamic language JIT compiler in the world (*)

(*) - lies, big lies, and benchmarks

Lua

I small, dynamic programming languageI think: JavaScript, Scheme, Tcl

I ANSI C: highly portable and embedableI LuaJIT: probably the best dynamic language JIT compiler in the world (*)

(*) - lies, big lies, and benchmarks

Lua

I small, dynamic programming languageI think: JavaScript, Scheme, TclI ANSI C: highly portable and embedable

I LuaJIT: probably the best dynamic language JIT compiler in the world (*)

(*) - lies, big lies, and benchmarks

Lua

I small, dynamic programming languageI think: JavaScript, Scheme, TclI ANSI C: highly portable and embedableI LuaJIT: probably the best dynamic language JIT compiler in the world (*)

(*) - lies, big lies, and benchmarks

Lua

I small, dynamic programming languageI think: JavaScript, Scheme, TclI ANSI C: highly portable and embedableI LuaJIT: probably the best dynamic language JIT compiler in the world (*)

(*) - lies, big lies, and benchmarks

Lua - example

1 function fact(n)2 if n == 1 then3 return 14 end5

6 return n * fact(n - 1)7 end8

9 print(fact(128))

Lua - example

1 function allwords ()2 local line = io.read()3 local pos = 14 return function ()5 while line do6 local s, e = string.find(line, "%w+", pos)7 if s then -- found a word?8 pos = e + 1 -- update next position9 return string.sub(line, s, e)

10 else11 line = io.read()-- word not found; try next line12 pos = 1 -- restart from first position13 end14 end15 return nil -- no more lines: end of traversal16 end17 end

“Programming in Lua”, 1st ed. §10.2

Lua - embedable in C apps...nginx - written in C...

Lua - embedable in C apps...nginx - written in C...

ngx http lua module

ngx http lua module

ngx http lua module

1 worker_processes 1;2 error_log logs/error.log;3 events {4 worker_connections 1024;5 }6 http {7 server {8 listen 8080;9 location /brew {

10 default_type text/html;11 content_by_lua_block {12 -- Lua code13 }14 }15 }16 }

ngx http lua module

content by lua block:1 ngx.req.read_body()2 local body = ngx.req.get_body_data()3

4 if body and (body:find("coffee") or5 body:find("beer")) then6 ngx.status = 4187 ngx.say("I’m a teapot")8 elseif body and body:find("tea") then9 ngx.status = 200

10 ngx.say("Coming right up")11 else12 ngx.status = 40013 ngx.say("Confused")14 end15

16 return ngx.exit(ngx.status)

ngx http lua module

1 http {2 upstream backend {3 server 0.0.0.1;4

5 balancer_by_lua_block {6 -- Lua code7 }8 }9

10 server {11 listen 8080;12

13 location / {14 proxy_pass http://backend/;15 }16 }17 }

ngx http lua module

balancer by lua block:

1 local redis = require "resty.redis"2 local balancer = require "ngx.balancer"3

4 local r = redis:new()5 local ok, err = r:connect("127.0.0.1", 6379)6

7 local host, err = r:get("my_upstream:host")8 local port, err = r:get("my_upstream:port")9

10 local ok, err = balancer.set_current_peer(host, port)11

12 if not ok then13 ngx.log(ngx.ERR, "failed to set peer: ", err)14 return ngx.exit(500)15 end

OpenResty

nginx + LuaJIT + async libraries

OpenResty

nginx

+ LuaJIT + async libraries

OpenResty

nginx + LuaJIT

+ async libraries

OpenResty

nginx + LuaJIT + async libraries

OpenResty

ngx http lua module ≈ OpenResty

non-trivial usage

I loading code from files: content by lua fileI LuaRocks is fair gameI testing: busted + Test::Nginx::SocketI profiler: a set of SystemTap-based scripts

non-trivial usage

I loading code from files: content by lua file

I LuaRocks is fair gameI testing: busted + Test::Nginx::SocketI profiler: a set of SystemTap-based scripts

non-trivial usage

I loading code from files: content by lua fileI LuaRocks is fair game

I testing: busted + Test::Nginx::SocketI profiler: a set of SystemTap-based scripts

non-trivial usage

I loading code from files: content by lua fileI LuaRocks is fair gameI testing: busted + Test::Nginx::Socket

I profiler: a set of SystemTap-based scripts

non-trivial usage

I loading code from files: content by lua fileI LuaRocks is fair gameI testing: busted + Test::Nginx::SocketI profiler: a set of SystemTap-based scripts

what’s not to like?

I Lua fragmentation: LuaJIT vs. PUC-Lua-currentI it’s not easy being asynchronous

I use OpenResty librariesI get your hands dirty with C

what’s not to like?

I Lua fragmentation: LuaJIT vs. PUC-Lua-current

I it’s not easy being asynchronousI use OpenResty librariesI get your hands dirty with C

what’s not to like?

I Lua fragmentation: LuaJIT vs. PUC-Lua-currentI it’s not easy being asynchronous

I use OpenResty librariesI get your hands dirty with C

what’s not to like?

I Lua fragmentation: LuaJIT vs. PUC-Lua-currentI it’s not easy being asynchronous

I use OpenResty librariesI get your hands dirty with C

Turing-complete web server configuration

Thank you!

Rate the talk!https://joind.in/talk/2de00

Questions?nikola@plejic.com — @nikolap:matrix.org

Thank you!Rate the talk!

https://joind.in/talk/2de00

Questions?nikola@plejic.com — @nikolap:matrix.org

Thank you!Rate the talk!

https://joind.in/talk/2de00

Questions?nikola@plejic.com — @nikolap:matrix.org