wrangling large scale frontend web applications
TRANSCRIPT
@ryan_roemer | surge2015.formidablelabs.com
WRANGLINGLARGE SCALEFRONTEND WEB APPLICATIONS
The web ismassively moving
to the frontend
Users want richand seamlessexperiences
Product ownerswant fast andnimble apps
Browser apps arenow business
critical
And, yes, even forthe enterprise
... which means
LOTS OF JAVASCRIPTIN THE BROWSERWRITTEN BY LARGE TEAMS
Let’s dig into some largefrontends at a high-traffic,top-five e-commerce site
The Numbers$10 billion FY 2014, $13 billion FY 2015 (est)1.5 billion page views for Thanksgiving -Cyber Monday 2014
The Code50+ JS applications650K lines, 2500 files of JavaScript sourceMore JavaScript lines & files than Java
The Dev TeamA 2+ year website rewrite50+ core frontend engineers14 vertical teams / "tracks"... and many external teams / partners
A TOUR THROUGHTHE TRENCHES
My wrangling adventures
as the JavaScript lead for
the website & dev teams
A few battle-tested
tips from the field...
... with a focus on
four personas
ARCHITECTS
GUIDES
GATEKEEPERS
LIFEGUARDS
ARCHITECTS
Plan & build a strongfoundation
ArchitectsA real buildCode organization
THE FUNDAMENTAL CHALLENGE:JAVASCRIPT IS THEWILD, WILD WEST
Architecture ChallengesBrowser is a single execution environmentCode size / duplication is criticalHard to manage, easy to do wrong"Cowboy" legacy vs. large scale organization
Let's look at the
architectural complexities
of just one page...
THE HOMEPAGE
The HomepageMultiple, independent JS appsCode from 8 different teamsExemplary "in transition" page, somewherebetween raw JavaScript & a SPA
JavaScript
ON PAGE
Direct page scripts10 remote scripts18 inline scripts
LAZY LOADED
Injected script tags6 app entry points
On Page
2 application code2 CDN infrastructure2 internal ads2 Google ads
1 fonts1 IE-conditional polyfill18 inline scripts
Lazy (Code)window._entry(function() { require(["header/header"], function () { require(["header/header-deferred"]); }); // ... require(["homepage/homepage"]); // ...});
Lazy (App)Asynchronously loaded1 shared library4 primary entry points2 deferred entry points
THE TAKEAWAY?IT'S COMPLICATED.
A "REAL" BUILD
Build ChallengesModern JS apps are complicated(compression, polyfills, transpiling, etc.)Multiple JS apps on the same page is evenmore complexSupporting development & production / CDN
Use a Build ToolChoose an paradigm: AMD, CommonJSChoose a build tool / loader: RequireJS,Browserify, WebpackTake time to learn & evaluate the tradeoffs
Our Build ToolsLegacy: AMD + RequireJSModern: CommonJS + WebpackTransition: AMD & CommonJS + Webpack
Keep prod & dev buildsblazingly fast
Make development
identical to production
Build ComplexitiesSharing / caching common code (lib.js)Varying repository structuresSharing frontend & backend codeCombining application, OSS, & 3rd party code
CODE ORGANIZATION
Plan your repositorystructure
1 → 14 → many repos
Have a bias for small &flexible
GUIDES
Coordinate the chaos,
level up the developmentteams
Guides & GuidanceThe Meta teamEducateReview everything
Guidance ChallengesThinking of the project as a wholeOnboarding unfamiliar / junior developersHelping teams be consistent /complementary
THE META TEAM
Code from multiple teams
all combined on one page
These folks represent thewhole page
The Meta TeamSet conventions & best practicesLead code reviews for consistency / DRYDevelop common utilities & the deploymentinfrastructure
YOUR META TEAM TASK:
HAVE ONE.
Our Meta JS Team1.5 dedicated developers6 "loaned" track developers
Our Meta JS Duties~1 day of code review / weekChat channel participationCross-track JS initiatives (i18n, multi-tenancy,PCI, etc.)
Meta JS BenefitsProject-wide consistency & supportHelp tracks consider other teamsRepresent track interests in the coreMore engineers to support the whole project
EDUCATION
Continually write livingdocumentation,
close to the code
Take a "hands on approach"
Invest in rising developers &
spread knowledge back toteams
CODE REVIEW
All code gets substantive &
meta review
Including all third party &
internal to the org vendorcode
GATEKEEPERS
Protect the frontend through
process & architecture
Gates & GatekeepersAutomate qualityMinimize risks / exposureRequire performance
GatekeepingChallenges
Feature pressurePoor quality codeUnknown included code
AUTOMATE QUALITY
Static checking
(eslint, jshint, jscs, etc.)
Clientside unit tests
Integration / E2E tests
Code coverage
Continuous integration (CI)
MINIMIZE RISK
Learn / identify your biggestrisk areas
Protect yourself whereverpossible
Architecture risks:
Injected HTML/CSS/JS
Code pattern risks:
Defer & pray
// I'm guessing...var HOPEFULLY_ENOUGH_TIME = 2000;
// Wait until ready for next step.setTimeout(function () { theNextStep();}, HOPEFULLY_ENOUGH_TIME);
Deployment / process risks:
"Hotfeatures"
REQUIRE PERFORMANCE
Frontend code must be small& fast
Teams are "feature-driven"
unless performance isenforced & required
Enforce with audits
"Web App PerformanceMeasurement, Monitoring and
Resiliency"
www.webpagetest.org
Enforce with process
Enforce with automation
"Automating WebPerformance Measurement"
Build your own tools wherenecessary
LIFEGUARDS
Create escape hatches &
lifelines in production
Lifeguards & LifeSavers
Logging, MonitoringDebugging helpers
LifeguardingChallenges
Code executes remotely on differentbrowsersFrontend errors are costly & often invisible
LOGGING & MONITORING
Your code runs & fails
on a variety of browsers
out in the wild
Log & capture everything
Get errors & messages
to a remote store
And then aggregate,
report, & alert
ProvidersRollbar Loggly
Sentry Airbrake
DEBUGGING SUPPORT
Give developers life lines
when things go wrong
You ship this:
!function(){var e="Hello Surge!",a=$("<h1 />");a.text(e),$("body").append(a)}();
Your devs want this:
(function () { var message = "Hello Surge!"; var $heading = $("<h1 />");
$heading.text(message);
$("body").append($heading);}());
Source Maps
/* ... LOTS MORE CODE ... */[],window._entry=c}();//# sourceMappingURL=http://dev.walmart.com:9873/js-dist-frontend/core-bundle.js.map
Publish in VPN ondeployment
Hidden from end users
Available to developers
All Together Now
A "real" buildOrganized codeThe Meta teamEducationCode Review
Automate qualityMinimize risksRequire performanceLogging, MonitoringDebugging helpers
Some partingthoughts on the
future
Embrace change
Reevaluate & refactor your
infrastructure & organization
Have a transition strategy
HAPPYWRANGLING
