gearman, supervisor and php - job management with sanity!

22
Supervisor, Gearman and PHP - Job management with sanity! ABU ASHRAF MASNUN

Upload: abu-ashraf-masnun

Post on 15-Jan-2015

8.897 views

Category:

Technology


3 download

DESCRIPTION

my talk in the phpXperts seminar 2011, Dhaka.

TRANSCRIPT

Page 1: Gearman, Supervisor and PHP - Job Management with Sanity!

Supervisor, Gearman and PHP - Job management with sanity!

ABU ASHRAF MASNUN

Page 2: Gearman, Supervisor and PHP - Job Management with Sanity!

What is it all about?

PHP GearmanSupervisor Job Management

Page 3: Gearman, Supervisor and PHP - Job Management with Sanity!

Let’s get REAL!Deploy Ninja – A case study

Deployment Solution (SVN to s/FTP) Trigger URL – Web Hooks Deploy Now – Deploy the latest right now Full Deploy – Upload the entire repo again

PROPOSED FEATURES

Page 4: Gearman, Supervisor and PHP - Job Management with Sanity!

How to train our Ninja?What did we have to do?

Tasks Involved: Fetch the repo Upload to the server

Instant processing Process separation for the three deployment types

REQUIREMENTS

Page 5: Gearman, Supervisor and PHP - Job Management with Sanity!

And cometh the tragedy!The day my life sucked !^@#

Instant processing requires “Daemons” PHP was not built for long running processes! Premature termination & Zombie processes. Monitoring the processes. Restarting the processes when terminated.

ARCHITECTURAL CHALLENGES

Page 6: Gearman, Supervisor and PHP - Job Management with Sanity!

Gearman, the super hero!He came and saved my a$$

Distributed workload management Client – Worker Architecture Multiple workers Multi-lingual workers

GEARMAN FEATURES

Page 7: Gearman, Supervisor and PHP - Job Management with Sanity!

Gearman: How it works?Understanding Gearman

Page 8: Gearman, Supervisor and PHP - Job Management with Sanity!

Gearman: InstallationAre you ready?

Install Gearman:

Install Libgearman:

Install the PHP Extension:

QR Code

sudo apt-get install gearman

sudo apt-get install libgearman-dev

sudo pecl install channel://pecl.php.net/gearman-0.8.0

Page 9: Gearman, Supervisor and PHP - Job Management with Sanity!

Gearman: Daemonizing!Starting Gearman QR Code

Running Gearman Daemon:

Debugging Gearman (Verbose mode) :

sudo gearman –d –u root

sudo gearman –d -vvv –u root

Page 10: Gearman, Supervisor and PHP - Job Management with Sanity!

Gearman: WorkersPHP Worker

QR Code

Page 11: Gearman, Supervisor and PHP - Job Management with Sanity!

Gearman: WorkersBreaking it down!

QR Code

Page 12: Gearman, Supervisor and PHP - Job Management with Sanity!

Gearman: WorkersBreaking it down!

QR Code

Page 13: Gearman, Supervisor and PHP - Job Management with Sanity!

Gearman: ClientsRequesting a job!

QR Code

Page 14: Gearman, Supervisor and PHP - Job Management with Sanity!

Gearman: Alternatives!Who else is there?

RabbitMQ ZeroMQ Apache MQ Dropr Microsoft MQ

Page 15: Gearman, Supervisor and PHP - Job Management with Sanity!

Whew! Almost done!Life is getting better

Instant processing requires “Daemons” PHP was not built for long running processes! Premature termination & Zombie processes. Monitoring the processes. Restarting the processes when terminated.

CHALLENGES STATUS

Page 16: Gearman, Supervisor and PHP - Job Management with Sanity!

Meet Supervisor!Here comes the tough guy!

Simple & Centralized Efficient & Extensible Compatible & Proven

It monitors your processes and restarts as instructed!

SUPERVISOR FEATURES

Page 17: Gearman, Supervisor and PHP - Job Management with Sanity!

Supervisor: Installation!Let’s go! QR Code

sudo apt-get install python-setuptools

sudo easy_install supervisor

Page 18: Gearman, Supervisor and PHP - Job Management with Sanity!

Supervisor: Getting Started!Config, debug and run! QR Code

sudo echo_supervisord_conf > /etc/supervisord.conf

sudo supervisord -n

sudo supervisord

Page 19: Gearman, Supervisor and PHP - Job Management with Sanity!

Hurray! We’re done!Life is so good :D

Instant processing requires “Daemons” PHP was not built for long running processes! Premature termination & Zombie processes. Monitoring the processes. Restarting the processes when terminated.

CHALLENGES STATUS

Page 20: Gearman, Supervisor and PHP - Job Management with Sanity!

What’s your Story?You can do it too!

Image Processing? Cache Regeneration? Mass emails?

USE CASES

Page 21: Gearman, Supervisor and PHP - Job Management with Sanity!

That was Simple! Eh? Got anything to ask?

Page 22: Gearman, Supervisor and PHP - Job Management with Sanity!

The Leevian!Who am I?

ABU ASHRAF MASNUN

Software Engineer @ Leeviohttp://masnun.com

[email protected]