Rev, Revactor, ReiaOr, the journey to Reia

Tony Arcieri@bascule

Rev, Revactor,

ReiaRedundant

Rev: Ruby event framework

http://github.com/tarcieri/rev

Cool.io

More on this later...

I love network programming

Application

Web Framework

Rack

Web ServerOMFG network craziness!

Ruby VMOMFG compilers and VMs break my brain!

OS KernelHere Be Dragons

There is life outside of HTTP

2003

http://github.com/tarcieri/distribustream

BitTorrent but better

•Started writing a BitTorrent library in C, FOR SPEED!

•BitTorrent sucks, I can do it better

•Spent 2 years trying to develop a new protocol in C

2005

I’m not productive!

•I’d like to write things in a scripting language instead of C

•Ruby’s cool, but it’s not FOR SPEED

•Maybe YARV will be FOR SPEED

•OMFG RAILS!!! I can get a job with that!

2006

DistribuStream inRuby

•Written in Ruby instead of Python

•Using EventMachine to handle large numbers of connections

•Use central organization and collaborative filtering to organize the peer network

Rails has Problems

•CGI sucks

•FCGI sucks (who remembers lighttpd?)

•SCGI sucks

•Apache Coyote for Ruby??

OMG MONGREL!

RubyConf 2006

RubyConf 2006

•Slept through Evan Phoenix’s keynote

•Waited for Matz’s announcement that YARV would be released on Christmas

•Where’s YARV on Christmas?

YARV

NO YARV FOR YOU!

:(

That made me a sad panda...

but there was an open bar :D

Ask Matz a question!

ME: WHERE’S YARV?!#$!$#!$

I met this guy...

Earlier that day...

“I’m Zed Shaw and I wrote Mongrel”

THUNDEROUS APPLAUSE

Zed and I

•OMG why don’t people care about that sort of thing?

•CGI Multipart EOF Bug OMG!

•Zed: EventMachine sucks. You should work on Ruby/Event

Meanwhile...

Denver Accord

•Matz, ko1, _why, et al talked for 4 hours on the Sunday night after RubyConf 2006

•Create NEW repo for 1.9

•Prohibit commits to MATZ

•Merge YARV into NEW

2007

EventMachine

•I became a committer

•I tried to fix as many problems as I could

•I eventually succumbed to too many OMFGs/sec

Rev

•Started by Franki Rai and Zed

•A Ruby wrapper for the libev library

•Both of them soon left, leaving me as the principal author

"The best way to complain is to make things." - James Murphy

Zed wanted a new blog theme...

so I introduced him to our designer

Zed’s exit

Cool.io: Ruby event framework

http://github.com/tarcieri/cool.io

Use Cases

• Large numbers of connections• Large amounts of I/O• Shared state between connections

Example Applications

• Web Spiders• HTTP Push Servers• Chat Servers

Supported Platforms

• Ruby MRI (1.8.6+)• Ruby YARV (1.9.0+)• Rubinius HEAD

Evented Programming Crash Course

Instead of:

response = request.get(params)

do:

request.start(params)

def on_success(response); ...; enddef on_failure(response); ... end

Block Form

start_request(params) do |response|if response.success?...YAY...

else...BOO...

endend

Cool.io: like EventMachine but with Ruby I/O primitives

Cool.io concepts

•Cool.io::Loop is the core event loop and the only thing that blocks EVAR

•Attach event listeners to it

Echo Server Example

require 'rubygems'require 'cool.io'

ADDR = '127.0.0.1'PORT = 4321

class EchoServerConnection < Cool.io::TCPSocket  def on_connect    puts "#{remote_addr}:#{remote_port} connected"  end

  def on_close    puts "#{remote_addr}:#{remote_port} disconnected"  end

  def on_read(data)    write data  endend

event_loop = Cool.io::Loop.defaultCool.io::TCPServer.new(ADDR, PORT, EchoServerConnection).attach(event_loop)

puts "Echo server listening on #{ADDR}:#{PORT}"event_loop.run

HTTP Client Example

require 'rubygems'require 'cool.io'

class MyHttpClient < Cool.io::HttpClient  def on_connect    super    STDERR.puts "Connected to #{remote_host}:#{remote_port}"  end

  def on_connect_failed    super    STDERR.puts "Connection failed"  end

  def on_response_header(header)    STDERR.puts "Response: #{header.http_version} #{header.status} #{header.http_reason}"  end

  def on_body_data(data)    STDOUT.write data    STDOUT.flush  end

  def on_request_complete    STDERR.puts "Request complete!"  end

  def on_error(reason)    STDERR.puts "Error: #{reason}"  endend

loop = Cool.io::Loop.defaultclient = MyHttpClient.connect("www.google.com", 80).attach(l)client.request('GET', '/search', :query => { :q => 'foobar' })loop.run

It's got filesystem monitoring!

Watchr uses it

Ry wrote a web server called Ebb

•Based on libev

•Ry showed me better ways Rev could work with Ruby 1.8’s scheduler

•Ry would later adapt Ebb into Node.js

Next steps: - async file I/O (libeio, like Node) https://github.com/tarcieri/cool.io/tree/libeio

- libevent support ($50 bounty!)

OMG ERLANG!

Erlang seemed REALLY awesome

Please cover your ears if you are sensitive to

loud noises

My “dream” model

•Evented I/O loop

•One scheduler thread per CPU core

•Lightweight userspace processes that talked using messages

Erlang did this OVER 9000 times better than I ever

could have

I took the ideas of Erlang and started

working on Revactoron Ruby 1.9 Fibers

Christmas:YARV released!

I’m a happy panda

2008

January 2008: Revactor released

Revactor: Actors for 1.9

http://github.com/tarcieri/revactor

My boss said I couldn't use Erlang :(

Fuck it, do it in Ruby!

Combine Rev Cool.io + Fibers to get callback-free async

programming!

Later seen in NeverBlock and em-synchrony

Evented Programming Crash Course

Instead of:

response = request.get(params)

do:

request.start(params)

def on_success(response); ...; enddef on_failure(response); ... end

THIS SUCKS

Block Form

start_request(params) do |response|if response.success?...YAY...

else...BOO...

endend

THIS SUCKS

Evented Programming smells like...

•Callback soup

•Callback spaghetti

•Callback HELL

Revactor was the first and still the most comprehensive solution for Fibers+events

•NeverBlock and em-synchrony merely wrap events in synchronous APIs

•Revactor uses messages to let fibers communicate

Echo Server Example

# An example echo server, written using Revactor::TCP# This implementation creates a new actor for each# incoming connection.

require 'revactor'

HOST = 'localhost'PORT = 4321

# Create a new listener socket on the given host and portlistener = Revactor::TCP.listen(HOST, PORT)puts "Listening on #{HOST}:#{PORT}"

# Begin receiving connectionsloop do # Accept an incoming connection and start a new Actor # to handle it Actor.spawn(listener.accept) do |sock| puts "#{sock.remote_addr}:#{sock.remote_port} connected"

# Begin echoing received data loop do begin # Write everything we read sock.write sock.read rescue EOFError puts "#{sock.remote_addr}:#{sock.remote_port} disconnected"

# Break (and exit the current actor) if the connection # is closed, just like with a normal Ruby socket break end end endend

Async web apps with Rainbows!Unicorn for asynchronous backends

http://rainbows.rubyforge.org/

Rainbows! do  use :Revend

Rainbows! do  use :Revactorend

(That’s all I got, sorry)

Acting the Actor Model

Want to know more about Actors?

http://revactor.org/

I became disenchanted with fake synchronous

I/O on top of Fibers

Why?

Evented programming in Ruby is not POLS

Threads are

If you want evented programming, try Node.js

Erlang fizzled

People keep asking me, “What will happen to Erlang? Will it be a popular language?” I don’t

know. I think it’s already been influential. It might end up like Smalltalk. I think Smalltalk’s been

very, very influential and loved by an enthusiastic band of people but never really very widely

adopted. And I think Erlang might be like that.

-- Joe Armstrong, Coders at Work

“It might need Microsoft to take some of its ideas and put curly braces here and there”

Or it might just need Ruby syntax... :)

Zed got me interested in making my own languageand Erlang had horrible syntax

http://github.com/tarcieri/reia

I started Reia in 2008

Erlang’s strengths:•Nonstop systems (99.999over9000%

uptime OMFG!)

•Realtime systems

•Fault-tolerant systems

•Hot code swapping

•Concurrent systems

•Distributed systems

Erlang with Ruby syntax

Immutable State

OBLIGATORY GAGA

2009

Reia @ Erlang Factory

•Mostly working implementation!

•Concurrent object model based on Erlang/OTP gen_servers

•Web framework: Ryan

I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages, so messaging came at the very beginning. It took a while to see how to do messaging in a programming language efficiently enough to be useful.

-- Alan Kay, creator of Smalltalk

Meanwhile...

Node.js got people interested in event-driven programming

Node and Cool.io share technologies

Cool.io: libev+Ruby(+libeio)Node: libev+libeio+V8

Late 2009

•REIA ZOMG TOO SLOW!!!

•Everything’s fucked! I didn’t know what I was doing

•If you fuck up a language project and don’t rewrite it, you get PHP

2010

Reia rewrite

•ZOMG OVER 9000x FASTER!

•Clean Erlang codebase which isn’t impossible to workwith

•Use concepts from the Erlang compiler

•Everything-is-an-object (although no concurrent objetcts yet, coming soon!)

Let’s look at Reia!

Ilya Grigorik created em-synchrony

People are now actually interested in fake

synchronous programming on an

evented backend

I think em-synchrony is aimed at

ROFLSCALING

BUT... I dusted off Cool.io and Revactor

just in case you’re interested

I still like Reia better :)

Reia has had horrible

documentation...

New web site!http://reia-lang.org/

Inspired by CoffeeScript

Reia PEG Grammar

•https://github.com/graeme-defty/reia

•Built on Neotoma: Treetop for Erlang

•Written by Graeme Defty and Kim Shrier

•It has TESTS OMG!

Why PEG is better

•Whitespace handling

•Nested interpolated strings e.g. (“foo #{“bar #{“baz”}”}”)

•Regex literals ala /.../

Reia next steps:

•Singleton classes

•Concurrent objects

•Namespaces

•Default and keyword arguments

•Metaprogramming

One more thing...

COMPILERS AREN’T HARD

Create Your Own Freaking Awesome

Programming Language

If I can do it you can to...

Links

• Cool.io: http://github.com/tarcieri/cool.io• Revactor: http://github.com/tarcieri/revactor• Reia: http://github.com/tarcieri/reia• Twitter: http://twitter.com/bascule• Blog: http://unlimitednovelty.com