joomla, 40 million page views, and younyphp.org/...mtv-quizilla-high-traffic-scaling-php.pdf ·...

38
Joomla, 40 Million Page Views, and You Mitch Pirtle NYPHP, January 2009

Upload: others

Post on 21-Jun-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Joomla, 40 Million Page Views, and YouMitch PirtleNYPHP, January 2009

Page 2: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Summary Overview

MTV brought Mitch Pirtle in to recommend a platform for a very high-traffic website - he chose an open source platform and then led development to a successful

release. Problems solved: lowering costs, fostering a can-do philosophy, promotion of the free software approach of

sharing with others, unique development approaches learned from working on large FOSS projects

Page 3: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Obligatory ‘Me’ Slide

American, married Swiss

Lived in US and EU

Currently in NYC

Web-only since 1995

VP at KickApps

High traffic sites

Pragmatic, not dogmatic

Joomla! co-founder

Former Mambo core

Booster of many FOSS

Co-founded NYCJUG

Too many projects to list

Page 4: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Let’s Get This Thing Started

Page 5: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Audience Breakdown

Page 6: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

TechnologyAre you a technologies person?

Page 7: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

BusinessAre you a business person?

Page 8: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Not Technology or BusinessAre you neither?

Page 9: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Drum Roll, Please

Page 10: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Quizillawww.quizilla.com

Page 11: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

The Challenge

www.quizilla.com

Acquired in 2006

Peaks of 40,000,000 unique pageviews in a week

Mix of two codebases (perl & php)

Difficult to maintain

Difficult to augment/evolve

Page 12: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

The Objectives: New World

High performance

Reliable, stable

Scale to extremes

Easy to maintain

Easy to extend

Page 13: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

The Shopping List

Open Source

Framework

MVC

Strong Community

Active Development

Common Language

Available Resources

Modular Architecture

Flexible, Customizable

Page 14: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Winner: Joomla! Frameworkhttp://docs.joomla.org/Category:Framework

Page 15: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Ok, we could have used CakePHP, Symfony, Zend Framework, Django, Ruby on Rails, Merb....

However PHP was desired, and I was available.

Psssssst! Hey buddy!

Page 16: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Why Joomla! Framework?

Meets all objectives

PHP, MySQL

FOSS

Flexible

Framework

MVC...

Page 17: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Result: Success

Only needed 3 webservers

Only needed 2 database servers

Becoming standard base platform for future development

Page 18: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Impact: Development

Atomic MVC - many developers, few conflicts

Source code - Tight reliance on revision control systems

Issue Tracker - Daily SCRUM-style planning meet

Change management - Build process with controls

Page 19: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Atomic MVC?

Ok, I made that up. Sounds cool though.

Page 20: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

What MVC Looks Like

Page 21: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

This is Atomic MVC

Page 22: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Plain MVC

Page 23: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Atomic MVC

Page 24: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Impact: Philosophy

Anything developed that wasn’t a proprietary feature or function was contributed back to the project

All code was packaged for ease of reuse on other projects within the enterprise

Page 25: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

So What Did We Learn?

Page 26: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

One Size Fits AllOk, almost.

Page 27: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Modular FrameworkNot fully decoupled from CMS (yet)

Page 28: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Database ClassStill very specific to one instance - no tiers or pools

Page 29: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

So How Does It End?No biggie. We didn’t fall ALL the way down.

Page 30: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

One Size Fits All: Classes!

Joomla is a stack framework

No need for language support

No need for dynamic navigation

No need for third party API support

JDate? A date class? Are you freaking kidding me?

Page 31: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Modular Framework: Extend

Extended existing classes for specific needs

Introduced helper classes to simplify complex tasks

Page 32: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Taming JDatabase

We cheated. And we’re proud of it.

Extended jConf with a constructor for poor-man’s query pool

Extended JDatabase mysqli driver to have one connection for reads, one for writes

Page 33: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

JSession Ate My Brain

Changes too numerous to mention :-)

Fixes to storage driver

Fixes to cookie mechanism

Auto-remember

Auto-resume

Page 34: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Input, Output

Hacked the output buffers so modules could share information with components

Page 35: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Special Note on Caching

JCache needs help. Serious help.

Used memcached driver

Chose our own key namespace strategy

Switched to a write-through cache approach

Dropped output caching entirely

Added per-item expiration TTL

Page 36: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Current Status

Codebase is now part of the standard offering for MTV websites that would like to take advantage of the platform

Page 37: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering

Questions?