agile adoption at google · – experiment with new practices and techniques (e.g. distributed...

Agile Adoption at Google

Potential and challenges of a true bottom-up organization

Mark Striebeck

August 2007

Google & Google Culture

• Encourage a startup culture

• Engineering-centric / bottom-up

• 20% time

• OKR’s (Objectives and Key Results)

• Tech Talks

• Consensus-driven culture

and…

Google & Google Culture

• Also: T-Shirt driven culture!

Grouplets

• Instead of central “process excellence” departments

• Accessibility, Agile, Documentation, Readability, Testing…

• 100% volunteering groups (this is the 20% project of members)

• Have budget and other support

• Mostly engineers, but also participants from other groups

“Engineers making other engineers' lives better”

Grouplets

• … of course the grouplet of grouplets has t-shirts!

Fix-It!

• Mostly driven by grouplets (doc fixit, testing fixit…)

• Every engineer is asked to participate rather then perform their regular work

• Not too often (1-2 per quarter)

“Imagine what 5000 engineers can accomplish in 1 day!”

Google & Development practices

• Small teams

• Very few standards on development process and practices

• Lightweight design process that is focused on early feedback from peers and senior engineers

• Management by feedback – project teams have complete responsibility and ownership

• Collaborative and egalitarian environment

• Release early, release often

“Google is pretty much on the left hand side of the Agile Manifesto”

Agile@Google

Quote from one of the first AdWords engineers:

“We worked in small groups (usually 2-3 engineers per project) and moved very quickly. We'd spend enough time designing and talking with PMs to have a general idea of what we wanted to build, and then we'd start coding.In some cases, we didn't even have input from UI designers, and we'd just take our best guess at a decent UI.…I remember keeping a big tasklist on a whiteboard. Our PM probably tracked things more closely, but I can't recall ever trying to estimate schedules more than 4-6 weeks out.”

Agile@Google

• Many individual projects

• Some complete programs

• Pockets in almost all areas and departments

• Very few “pure” Scrum or XP projects

• NO agile standard

Limits of Agile at Google

• YAGNI

– Your application will be used by several 100,000 users and mightneed to get extended heavily quickly

• Infrastructure projects = Research projects (no PM – engineering team decides what and how to build)

• Input and guidance from senior Googlers

– Upfront lightweight design sent to mailing list for feedback

Agile@Google

• Adoption

– Projects usually don’t ask for help – individuals sometimes do

– Very different in different project – do what seems right

• Tools

– Googlers always want tools!

– Have some lightweight tools – some self-written

Agile@Google

• Challenges

– Distributed teams

– Multi-project (program) management

– Pairing not particularly favored

– General concern of too much process

– People want to produce great things and are willing to work crazy hours and take risks

The Agile Grouplet

• Weekly meetings

– Exchange ideas

– Planning of activities

“Discover and promote lightweight methods that help teams produce better software faster. “

The Agile Grouplet - Programs

• Tech Talks

– Internal (Experience reports)

– External (Mike Cohn, Mary Poppendieck, Ken Schwaber… -whoever is in the area!)

• Learn-It!

– Instead of Fix-It!

– Lots of activities to get to know agile

– Offer help for agile projects

• BayXP

– Sponsor monthly events

– Videotape and publish event if a “celebrity” is attending

The Agile Grouplet - Programs

• Agile Safari (“Agile projects in the wild”)

– Agile projects sign-up for people to attend their activities

– People sign-up

– Grouplet coordinates and communicates

• Office Hours

– Weekly

– Completely open agenda

The Agile Grouplet - Programs

• Coaching / Mentoring– Individual mentoring– Project coaching (sometimes tough with 20% time)

• Retrospective Facilitator Network – Exchange ideas and experiences– Experiment with new practices and techniques (e.g. distributed

retrospectives)– Facilitate retrospectives

• Training1. External (sometimes multiple)2. Select and license material3. Google’ify it and deliver by Google engineers4. Ultimately becomes part of standard course offerings

Internal Push Back

• No top-down decret means people can disagree

– Who knows Steve Yegge?

• Project Management Steering Council

– Sees agile as just one method to develop software

– Pushes for more generic approach

The Testing Grouplet

• Why a separate grouplet?

– Agile Grouplet: Focus on development process – good way to develop software

– Testing Grouplet: Focus on developer testing – necessary way to develop software

“To drive adoption of developer testing.”

The Testing Grouplet - Programs

• Weekly meetings

• Tech talks

• Office hours

• Weekly lunches

• Parties! (with cookies!)

The Testing Grouplet - Programs

Testing On The Toilet

“We write flyers about everything from dependency injection to code coverage, and then regularly plaster the bathrooms all over Google with each episode, almost 500 stalls worldwide... “

"... it reminds me of the first two sentences of the now famous founders' letter Page and Brindistributed to prospective Google shareholders before the company's 2004 IPO: "Google is not a conventional company. We do not intend to become one." Mission accomplished."

The Testing Grouplet - Programs

Testing On The Toilet

• Great program to get attention for testing

– Internal and external discussions

– Added to public Google testing blog (http://googletesting.blogspot.com)

– Very funny fakes

– Reported and quoted in many places (Washington Post, Forbes, Blogs…)

"... where one cannot escape one's work even in the bathroom, strikes me as bordering on obsessive / compulsive, and seems almost Orwellian."

"the post is on a blog site, not on Google, April 1st a little early...."

The Testing Grouplet - Programs

Fix-It!logo

The Testing Grouplet - Programs

• Testing Documentation (e.g. Design4Testability guide)

• Training

• Noogler training and noogler labs

Test Certified

• Tried for a long time to tell teams “Get better at testing!” – somehow that didn’t work

• Instead: Small steps to get better at testing

– Level 1: Setup tools – get familiar with testing

– Level 2: Learn techniques and technologies that enable team to write tests for all new code

– Level 3: Improve test suite and get all code under test

– Level 4: …

“Horrible name – great results!”

Test Certified

Rewards:

Level 1 Level 2 Level 3

Test Mercenaries

• Full timer engineers to help projects advance on Test Certified ladder

• Very googley approach:

– Program was born and planned and started by the Testing Grouplet

– Sponsored by executive managers

– Then given back to Grouplet members to start and execute

Test Mercenaries

• Few days pre-assessment to evaluate state of code and readiness of team to make changes

• Activities:

– Introduce new technologies that make code more testable

– Refactor code

– Train individuals on testing technologies and techniques

– Help team to integrate testing in development process

• Engagements are 3 months max

Testapalooza

• All-day conference

• Bringing all disciplines who do any kind of testing together:

Engineering, I18N, RelEng, SecOps, SRE*, SysOps, Agile Grouplet, Testing Grouplet, Test Engineering, UE

• Presenters and attendants from all over the world

* Site Reliability Engineering

Testapalooza

• … and there were T-shirts – of course!

Testing @ Google

• Continuous Build over all code and all projects

• Use Google production cluster to run tests

• Method-level parallelization and dependency analysis

• Run tests across all Google code

• Lots of ideas, tools and applications from projects – some get “productized”

Executive Support

• Memos

• Engineering Reviews

• Program sponsors