get queued

DON’T MAKE ME WAIT

Let’s get queued up

Chad Windnagle - @drmmr763

INTRODUCTION• Recovering Joomla Addict

(Clean 2yrs)• Joomla GSoC Admin (2012-

2015)• Joomla Resource Team Editor• Joomla!Day, JWC, & JAB

Speaker• JWC 2014 Keynote Speaker

Chad Windnagle - @drmmr763

JOOMLA WAS MY PLAYGROUND

Chad Windnagle - @drmmr763

JOOMLA INSPIRED ME

AUDIENCE INTRODUCTION

Chad Windnagle - @drmmr763

WHAT ARE QUEUES ANYWAY?

Chad Windnagle - @drmmr763

SYNCHRONOUS FLOWUser Sends

RequestServer Sends

ResponseResponseRendered

Chad Windnagle - @drmmr763

A-SYNCHRONOUS FLOW

User SendsRequest

Server SendsResponse

ResponseRendered

Background TaskInitialized

Background TaskExecuted

Notify UserTask Complete

Chad Windnagle - @drmmr763

WHAT SHOULD WE QUEUE?•Notifications (Email /

SMS)•Statistical Calculations

•Data Matching Algorithms

•Regularly Scheduled Tasks

• Image / Video Processing

•Data Indexing

•Traditional Page Requests

•User File Uploads•Login / Logout Functions

•File Downloads•Add to Cart Functions

•REST API Endpoints

Use Queues: Don’t Use Queues:

COMPONENTS OF A QUEUE SYSTEM

Chad Windnagle - @drmmr763

THE MESSAGE

A message is string-based content which informs the application which task to

perform, and encapsulates any data needed to perform that task.

Chad Windnagle - @drmmr763

{ "messageType" : "sendEmail", "email" : { "to" : "speakers@jdayflorida.com", "from" : "chad@chadwindnagle.com", "subject": "Hello speakers!", "body" : "Can't wait to see you all." }}

MESSAGE EXAMPLE

Chad Windnagle - @drmmr763

THE QUEUE / TUBE

The message queue, or tube, contains a collection of messages that need to be

processed by the queue system.

Chad Windnagle - @drmmr763

Newest OldestTypical Processing Order

SERVICE WORKER

The service worker is code that is responsible for executing the tasks on the queue.

Chad Windnagle - @drmmr763

Newest OldestTypical Processing Order

EXECUTION CYCLE

Chad Windnagle - @drmmr763

User RequestsQueueable Task

Message with DataGenerated

Message AddedTo Queue

Server BootsService Worker

Service WorkerPerforms Task

Message RemovedFrom Queue

QUEUE SYSTEM VENDORS

VARIOUS VENDORS

Beanstalkd

Laravel

Redis

Gearman

RabbitMQ

AmazonSQS

IronMQ

Symfony

Chad Windnagle - @drmmr763

Beanstalkd

BEANSTALKD BENEFITS

Chad Windnagle - @drmmr763

RAM Based Data Storage - High Performance!

HTTP Protocol - Allows Multi-Server Set Up

PHP Composer Library Available - Fast DevelopmentCommunity Resources (Blogs, Documentation, Etc.)

Chad Windnagle - @drmmr763

STACK OVERVIEWPHP Application

(Joomla!)

PHP Library (Pheanstalk)

Beanstalkd Service

Queue Jobs

Queue / Execute Job

Chad Windnagle - @drmmr763

apt-get install beanstalkd

INSTALLING BEANSTALKD

Chad Windnagle - @drmmr763

php composer.phar require pda/pheanstalk

INSTALL PHEANSTALK

RUNNING BEANSTALKD

Chad Windnagle - @drmmr763

./beanstalkd -l 127.0.0.1 -p 11300

Options:

-l Specify the address to listen on-p specify the port to listen on

CURRENT STATUS?

Chad Windnagle - @drmmr763

Beanstalkd is running:

- It can add jobs to a queue- Returns job details to service worker- Jobs can be removed from the queue

ADD MESSAGE TO QUEUE

Chad Windnagle - @drmmr763

<?php

use Pheanstalk\Pheanstalk;

// connect to Beanstalkd Service$pheanstalk = new Pheanstalk(‘127.0.0.1');

// add the job to the queue$pheanstalk->put(‘a special message’);

GET MESSAGE FROM QUEUE

Chad Windnagle - @drmmr763

<?php

use Pheanstalk\Pheanstalk;

// connect to Beanstalkd Service$pheanstalk = new Pheanstalk(‘127.0.0.1');

// get next available job from queue$job = $pheanstalk ->watch(‘default’) // specific tube ->reserve(); // job in progress

EXECUTE A TASK

Chad Windnagle - @drmmr763

<?php

// get the message from the queued job$jobData = $job->getData();

// prints “a special message”print $jobData;

// remove job from queue$job->delete();

MORE USEFUL EXAMPLE

Chad Windnagle - @drmmr763

$message = [ ‘messageType => ‘sendEmail’, ‘email’ => [ ‘to’ => ‘speakers@jdayflorida.com’, ‘from’ => ‘chad@chadwindnagle.com’, ‘subject => ‘Hello speakers!’, ‘body’ => ‘Please come to my session!’]];

MORE USEFUL EXAMPLE

Chad Windnagle - @drmmr763

<?php

// add the job to the queue$pheanstalk->put(json_encode($message));

Application code would add tasks to the queue

MORE USEFUL EXAMPLE

Chad Windnagle - @drmmr763

// get the message from the queued job$jobData = json_decode($job->getData());

$messageType = $jobData[‘messageType’];

if ($messageType == ‘sendEmail’ { // “to”, “from”, “subject”, & “body”$this->sendEmail($jobData[‘email’]);}

Service worker executes based on message type

TRIGGERING SERVICE WORKER

Chad Windnagle - @drmmr763

php ./service-worker.php

Chad Windnagle - @drmmr763

github.com/drmmr763/jdayfloridaPHP ProjectComposer BasedUses PheanstalkUses BeanstalkdCommand Line

UtilitiesPHP Class Message

Types

EXAMPLE GITHUB CODE

ANY QUESTIONS?

Chad Windnagle - @drmmr763

ICON CREDITS

Kirby WuSchmidt Sergey

IconSphere

Carl Holdnerss

ProSymbols

Above icons licensed via Creative Commons, sourced from TheNounProject.com

GregorCresna

rOliviaStoian

RudolfHoraczek

ProSymbols icon 54

ChangHoon

BaekDaniel Baker

Chad Windnagle - @drmmr763

THANK YOU!Code: github.com/drmmr763/jdayflorida

Follow Me: @drmmr763

Slides available on slideshare