gearman, supervisor and php - job management with sanity!

Supervisor, Gearman and PHP - Job management with sanity!

ABU ASHRAF MASNUN

What is it all about?

PHP GearmanSupervisor Job Management

Let’s get REAL!Deploy Ninja – A case study

Deployment Solution (SVN to s/FTP) Trigger URL – Web Hooks Deploy Now – Deploy the latest right now Full Deploy – Upload the entire repo again

PROPOSED FEATURES

How to train our Ninja?What did we have to do?

Tasks Involved: Fetch the repo Upload to the server

Instant processing Process separation for the three deployment types

REQUIREMENTS

And cometh the tragedy!The day my life sucked !^@#

Instant processing requires “Daemons” PHP was not built for long running processes! Premature termination & Zombie processes. Monitoring the processes. Restarting the processes when terminated.

ARCHITECTURAL CHALLENGES

Gearman, the super hero!He came and saved my a$$

Distributed workload management Client – Worker Architecture Multiple workers Multi-lingual workers

GEARMAN FEATURES

Gearman: How it works?Understanding Gearman

Gearman: InstallationAre you ready?

Install Gearman:

Install Libgearman:

Install the PHP Extension:

QR Code

sudo apt-get install gearman

sudo apt-get install libgearman-dev

sudo pecl install channel://pecl.php.net/gearman-0.8.0

Gearman: Daemonizing!Starting Gearman QR Code

Running Gearman Daemon:

Debugging Gearman (Verbose mode) :

sudo gearman –d –u root

sudo gearman –d -vvv –u root

Gearman: WorkersPHP Worker

QR Code

Gearman: WorkersBreaking it down!

QR Code

Gearman: WorkersBreaking it down!

QR Code

Gearman: ClientsRequesting a job!

QR Code

Gearman: Alternatives!Who else is there?

RabbitMQ ZeroMQ Apache MQ Dropr Microsoft MQ

Whew! Almost done!Life is getting better

Instant processing requires “Daemons” PHP was not built for long running processes! Premature termination & Zombie processes. Monitoring the processes. Restarting the processes when terminated.

CHALLENGES STATUS

Meet Supervisor!Here comes the tough guy!

Simple & Centralized Efficient & Extensible Compatible & Proven

It monitors your processes and restarts as instructed!

SUPERVISOR FEATURES

Supervisor: Installation!Let’s go! QR Code

sudo apt-get install python-setuptools

sudo easy_install supervisor

Supervisor: Getting Started!Config, debug and run! QR Code

sudo echo_supervisord_conf > /etc/supervisord.conf

sudo supervisord -n

sudo supervisord

Hurray! We’re done!Life is so good :D

Instant processing requires “Daemons” PHP was not built for long running processes! Premature termination & Zombie processes. Monitoring the processes. Restarting the processes when terminated.

CHALLENGES STATUS

What’s your Story?You can do it too!

Image Processing? Cache Regeneration? Mass emails?

USE CASES

That was Simple! Eh? Got anything to ask?

The Leevian!Who am I?

ABU ASHRAF MASNUN

Software Engineer @ Leeviohttp://masnun.com

masnun@leevio.com