wrangling large scale frontend web applications

@ryan_roemer | surge2015.formidablelabs.com

WRANGLINGLARGE SCALEFRONTEND WEB APPLICATIONS

The web ismassively moving

to the frontend

Users want richand seamlessexperiences

Product ownerswant fast andnimble apps

Browser apps arenow business

critical

And, yes, even forthe enterprise

... which means

LOTS OF JAVASCRIPTIN THE BROWSERWRITTEN BY LARGE TEAMS

Let’s dig into some largefrontends at a high-traffic,top-five e-commerce site

The Numbers$10 billion FY 2014, $13 billion FY 2015 (est)1.5 billion page views for Thanksgiving -Cyber Monday 2014

The Code50+ JS applications650K lines, 2500 files of JavaScript sourceMore JavaScript lines & files than Java

The Technologies

The Dev TeamA 2+ year website rewrite50+ core frontend engineers14 vertical teams / "tracks"... and many external teams / partners

A TOUR THROUGHTHE TRENCHES

My wrangling adventures

as the JavaScript lead for

the website & dev teams

A few battle-tested

tips from the field...

... with a focus on

four personas

ARCHITECTS

GUIDES

GATEKEEPERS

LIFEGUARDS

ARCHITECTS

Plan & build a strongfoundation

ArchitectsA real buildCode organization

THE FUNDAMENTAL CHALLENGE:JAVASCRIPT IS THEWILD, WILD WEST

Architecture ChallengesBrowser is a single execution environmentCode size / duplication is criticalHard to manage, easy to do wrong"Cowboy" legacy vs. large scale organization

Let's look at the

architectural complexities

of just one page...

THE HOMEPAGE

The HomepageMultiple, independent JS appsCode from 8 different teamsExemplary "in transition" page, somewherebetween raw JavaScript & a SPA

JavaScript

ON PAGE

Direct page scripts10 remote scripts18 inline scripts

LAZY LOADED

Injected script tags6 app entry points

On Page

2 application code2 CDN infrastructure2 internal ads2 Google ads

1 fonts1 IE-conditional polyfill18 inline scripts

Lazy (Code)window._entry(function() { require(["header/header"], function () { require(["header/header-deferred"]); }); // ... require(["homepage/homepage"]); // ...});

Lazy (App)Asynchronously loaded1 shared library4 primary entry points2 deferred entry points

THE TAKEAWAY?IT'S COMPLICATED.

A "REAL" BUILD

Build ChallengesModern JS apps are complicated(compression, polyfills, transpiling, etc.)Multiple JS apps on the same page is evenmore complexSupporting development & production / CDN

Use a Build ToolChoose an paradigm: AMD, CommonJSChoose a build tool / loader: RequireJS,Browserify, WebpackTake time to learn & evaluate the tradeoffs

Our Build ToolsLegacy: AMD + RequireJSModern: CommonJS + WebpackTransition: AMD & CommonJS + Webpack

Keep prod & dev buildsblazingly fast

Make development

identical to production

Build ComplexitiesSharing / caching common code (lib.js)Varying repository structuresSharing frontend & backend codeCombining application, OSS, & 3rd party code

CODE ORGANIZATION

Plan your repositorystructure

1 → 14 → many repos

Have a bias for small &flexible

GUIDES

Coordinate the chaos,

level up the developmentteams

Guides & GuidanceThe Meta teamEducateReview everything

Guidance ChallengesThinking of the project as a wholeOnboarding unfamiliar / junior developersHelping teams be consistent /complementary

THE META TEAM

Code from multiple teams

all combined on one page

These folks represent thewhole page

The Meta TeamSet conventions & best practicesLead code reviews for consistency / DRYDevelop common utilities & the deploymentinfrastructure

YOUR META TEAM TASK:

HAVE ONE.

Our Meta JS Team1.5 dedicated developers6 "loaned" track developers

Our Meta JS Duties~1 day of code review / weekChat channel participationCross-track JS initiatives (i18n, multi-tenancy,PCI, etc.)

Meta JS BenefitsProject-wide consistency & supportHelp tracks consider other teamsRepresent track interests in the coreMore engineers to support the whole project

EDUCATION

Continually write livingdocumentation,

close to the code

Take a "hands on approach"

Invest in rising developers &

spread knowledge back toteams

CODE REVIEW

All code gets substantive &

meta review

Including all third party &

internal to the org vendorcode

GATEKEEPERS

Protect the frontend through

process & architecture

Gates & GatekeepersAutomate qualityMinimize risks / exposureRequire performance

GatekeepingChallenges

Feature pressurePoor quality codeUnknown included code

AUTOMATE QUALITY

Static checking

(eslint, jshint, jscs, etc.)

Clientside unit tests

Integration / E2E tests

Code coverage

Continuous integration (CI)

MINIMIZE RISK

Learn / identify your biggestrisk areas

Protect yourself whereverpossible

Architecture risks:

Injected HTML/CSS/JS

Code pattern risks:

Defer & pray

// I'm guessing...var HOPEFULLY_ENOUGH_TIME = 2000;

// Wait until ready for next step.setTimeout(function () { theNextStep();}, HOPEFULLY_ENOUGH_TIME);

Deployment / process risks:

"Hotfeatures"

REQUIRE PERFORMANCE

Frontend code must be small& fast

Teams are "feature-driven"

unless performance isenforced & required

Enforce with audits

"Web App PerformanceMeasurement, Monitoring and

Resiliency"

www.webpagetest.org

Enforce with process

Enforce with automation

"Automating WebPerformance Measurement"

Build your own tools wherenecessary

LIFEGUARDS

Create escape hatches &

lifelines in production

Lifeguards & LifeSavers

Logging, MonitoringDebugging helpers

LifeguardingChallenges

Code executes remotely on differentbrowsersFrontend errors are costly & often invisible

LOGGING & MONITORING

Your code runs & fails

on a variety of browsers

out in the wild

Log & capture everything

Get errors & messages

to a remote store

And then aggregate,

report, & alert

ProvidersRollbar Loggly

Sentry Airbrake

DEBUGGING SUPPORT

Give developers life lines

when things go wrong

You ship this:

!function(){var e="Hello Surge!",a=$("<h1 />");a.text(e),$("body").append(a)}();

Your devs want this:

(function () { var message = "Hello Surge!"; var $heading = $("<h1 />");

$heading.text(message);

$("body").append($heading);}());

Source Maps

/* ... LOTS MORE CODE ... */[],window._entry=c}();//# sourceMappingURL=http://dev.walmart.com:9873/js-dist-frontend/core-bundle.js.map

Publish in VPN ondeployment

Hidden from end users

Available to developers

All Together Now

A "real" buildOrganized codeThe Meta teamEducationCode Review

Automate qualityMinimize risksRequire performanceLogging, MonitoringDebugging helpers

Some partingthoughts on the

future

Embrace change

Reevaluate & refactor your

infrastructure & organization

Have a transition strategy

HAPPYWRANGLING

surge2015.formidablelabs.com

THANKS!