javascript dependency management
DESCRIPTION
The presentation first makes the case for modularity in modern JavaScript systems and the resulting need for a transitive dependency management solution. Later it covers the state of dependency management in JavaScript. Finally it describes the open-source Jingo JavaScript dependency manager (http://jingo.googlecode.com) and its approach to solving the dependency management problem.TRANSCRIPT
+
JavaScript Dependency Management Breaking Down the Barriers to Modular JavaScript Systems
Sean M. Duncan
J S
D M
0 1 0 0 1 0 1 0
0 1 0 1 0 0 1 1
0 1 0 0 0 1 0 0
0 1 0 0 1 1 0 1
+
JS
MD
Sean M. Duncan
Senior Software Consultant with Solution Design Group
9 years of experience working with Web applications
Primarily Enterprise Java space with a recent focus on dynamic languages Carol.com (Groovy/Grails)
Giftag.com (Python/Django/Google App Engine)
Giftag Firefox Addon (JavaScript)
+
JS
MD
My Reintroduction to JavaScript
I’m a fairly new believer in the potential for elegant and maintainable large-scale JavaScript systems
While working on the Giftag Firefox addon I started to see a side of JavaScript that had escaped me before
Douglas Crockford would call it “The Good Parts”. I call it “Professional-Grade JavaScript”
+
JS
MD
JavaScript’s Sordid Past Small tasks in limited contexts
Writing systems that behaved consistently across popular browsers was difficult
JavaScript’s scoping confused developers due to inconsistency with other languages that share a C-like syntax
Debugging support for JavaScript was severely lacking
Testing and verification tools were either not available or still in an early state of evolution
+
JS
MD
JavaScript’s Sorted Present Rich Internet Applications Demand More
Cross-browser consistency is now much easier to achieve
Quality of JavaScript literature has drastically improved JavaScript: The Good Parts by Douglas Crockford
Pro JavaScript Techniques by John Resig
High-quality, general-purpose frameworks are now widely available
Browser vendors and extension developers have made and continue to make strides in the area of debugging tools
Quality testing and verification tools are now available
+
JS
MD
Professional-Grade JavaScript Modern JavaScript requires the same discipline and organization we employ with other platforms
Focus on the parts of the language that exhibit consistency and steer clear of those that don’t (see JSLint)
Practice Unobtrusive JavaScript by separating server-side data, client-side logic and content markup
Understand functional scoping and closure
Strive to have minimal impact on the global namespace
Verify the proper function of your code (see jqUnit/jqMock)
Decompose your system into loosely coupled, purpose-focused modules
Organize your project source in a manner that caters to intuition
+
JS
MD
The Case for Modular JavaScript
Proven approach for other platforms that scale well to large bodies of code Java, Python, Ruby and Groovy
Loosely coupled systems are easier to enhance and maintain
Purpose-focused modules are easier to verify and reuse
Lends itself well to both object-oriented and functional solutions (see Python)
Enables us to divide and conquer larger problems through layers of abstraction
+
JS
MD
The Elements of Modularity
Logical Modularity: The problem domain concepts are captured in discrete software
artifacts
These software artifacts must have minimal knowledge of other components within the system
Components work together at different layers of abstraction to accomplish the software’s goal
Physical Modularity: File system resources are organized parallel to the concepts
within the problem domain
Easiest to accomplish when file system structures mirror logical structures
+
JS
MD
JavaScript Makes Modularity Difficult JavaScript as a language facilitates modularity; JavaScript as a platform stands in the way
Developers must manually enter script tags to load modules
Direct and transitive dependencies must be explicitly declared
Tags must be arranged in transitive dependency order
Pages become tightly coupled to modules they should need no knowledge of
Manual dependency management is initially tedious, and maintenance is worse
+
JS
MD
Transitive Dependency Management
Modules declare their direct dependencies and the module loader/manager takes it from there
We couldn’t imagine Java, Python, Groovy or Ruby without it
Why not JavaScript? See JavaScript’s Past (small tasks)
An important tool that should be in our Professional-Grade JavaScript toolbox
+
JS
MD
First-Generation Approaches
Script Tag Append 1. A dependent module declares a dependency with something
like module.require(‘foo’);
2. The dependency manager adds a <script src=“foo.js”/> element to the Document Object Model (DOM)
Ajax and Eval 1. A dependent module declares a dependency with something
like module.require(‘foo’);
2. The dependency manager makes an Ajax request for the module with xhr.open(‘GET’, ‘foo.js’, false); send(null); …
3. The dependency manager loads the response text by calling window.eval(xhr.responseText);
+
JS
MD
Script Tag Append
Pros: Simple to implement
Standard caching behavior
Source code is properly attributed to its file resource
Cons: The timing of the response to the script element introduction is
inconsistent from one browser to the next
Dependent modules may be interpreted before their dependencies
+
JS
MD
Ajax and Eval
Pros: Still relatively simple to implement
Standard caching behavior
Source can be loaded synchronously, making transitive dependency order easy to guarantee
Cons: The use of window.eval(xhr.responseText) causes source to be
attributed to the module manager rather than the module’s file resource
Some browsers have been known to ignore the asynch flag in the xhr.open(method, url, asynch) call
+
JS
MD
Dojo Toolkit’s Module Loader
Ajax and Eval with Global Callback
Translates module names into file resources in a manner similar to Java packages
Registered ’onLoad’ callbacks are invoked when all pending modules are loaded
Originally designed to streamline the distribution of the Dojo Toolkit rather than to be a general solution
+
JS
MD
Dojo Toolkit Code Examples
Publish a module:dojo.provide(‘my.module’);
Create a module:dojo.declare(‘my.module’, null, { // module body});
Declare a dependency: dojo.require(‘your.module’);
Run some dependent code: dojo.addOnLoad(function() { // code that depends on ‘your.module’});
+
JS
MD
YUI Loader
Script Tag Append with Availability Polling and Callbacks
Feature-rich but biased toward visual components Can be used for CSS as well as JavaScript
Rollup feature loads bundles of related modules in one file
Availability determination is complex and inconsistent across browsers “Nothing can be done … except to pause and hope for the best.”
Availability Polling: YUI loader polls for a variable that non-YUI scripts publish
when they are interpreted
+
JS
MD
YUI Loader Code Examples
Publish a module to the loader: loader.addModule({ name: ‘colorpicker’, type: ‘js’, varName: ‘colorPickerLoaded’, // availability polling fullPath: ‘http://www.example.com/js/module.js’, requires: [‘module.a’, ‘module.b’]});
Configure the loader and pull in some dependencies: var loader = new YAHOO.util.YUILoader({ require: ["colorpicker", "treeview"], onSuccess: function() { // code that depends on colorpicker and treeview }, combine: true // allow “rollup” of required modules});loader.insert();
+
JS
MD
jQuery & Prototype
Ajax and Eval / Basic Script GET
Neither really trying to be a player in the dependency management space
jQuery:$.getScript(‘scripts/foo/bar/baz.js’, function() { // code that requires ‘foo.bar.baz’});
Prototype:var getScript = function(url, callback) { new Ajax.Request(url, { method: ‘get’, onSuccess: function(transport) { window.eval(transport); callback(); } });};
+
Jingo JavaScript Dependency Manager jingo.googlecode.com
Breaking Down the Barriers to Modular JavaScript Systems
John Bailey Sean M. Duncan
I N
G OJ
+
JS
MD
Jingo
Created to solve the problem of JavaScript dependency management in a lightweight, framework-agnostic package
Avoids technical limitations and framework bias of common approaches to JavaScript dependency management
Streamlines both the development and maintenance of modular JavaScript systems
+
JS
MD
How Jingo Works
Works within the constraints of browser processes by extending the script lifecycle
Combines Script Tag Append and Module Body Callbacks to make modules available in transitive dependency order
Module Resolution advances modules through a progression of three distinct states
+
JS
MD
The Module Lifecycle: Loading
Translates module names into file resources in a manner similar to Java packages
Script tag with its ‘src’ attribute set to the translated resource path is appended to the DOM
+
JS
MD
The Module Lifecycle: Declared
Browser loads the source for a given module, interprets the file, and executes the jingo.declare call
Jingo registers the module’s body callback and ensures all direct dependencies are transitioned into loading state
+
JS
MD
The Module Lifecycle: Resolved
Modules with no dependencies are immediately resolved
Modules are resolved by calling the registered body callback
Jingo continues to scan for declared modules ready for resolution
Process repeats until all declared modules and their dependencies have been resolved
Module Body Callbacks are guaranteed to be invoked in transitive dependency order
+
JS
MD
Visualizing the Module Lifecycle
<script> jingo.anonymous({ require: [‘B’, ‘C’], exec: function() { // module A } }); </script>
<script src=“jingo.js”/>
- Dependency Tree - - Page Source -
+
JS
MD
Visualizing the Module Lifecycle
<script>...</script> <script src=“jingo.js”/>
<script src=“B.js”/> <script src=“C.js”/>
- Dependency Tree - - Page Source -
+
JS
MD
Visualizing the Module Lifecycle
<script>...</script> <script src=“jingo.js”/>
<script src=“B.js”/> <script src=“C.js”/> <script src=“D.js”/> <script src=“E.js”/>
- Dependency Tree - - Page Source -
+
JS
MD
Visualizing the Module Lifecycle
<script>...</script> <script src=“jingo.js”/>
<script src=“B.js”/> <script src=“C.js”/> <script src=“D.js”/> <script src=“E.js”/> <script src=“F.js”/>
- Dependency Tree - - Page Source -
+
JS
MD
Visualizing the Module Lifecycle
<script>...</script> <script src=“jingo.js”/>
<script src=“B.js”/> <script src=“C.js”/> <script src=“D.js”/> <script src=“E.js”/> <script src=“F.js”/> <script src=“G.js”/> <script src=“H.js”/>
- Dependency Tree - - Page Source -
+
JS
MD
Getting Started with Jingo Initializing Jingo
Jingo’s main repository defaults to ‘scripts’, its logging verbosity defaults to ‘warn’ and its script loading timeout defaults to 30 seconds
If the defaults work for your project, initialization is unnecessary
Override the default main repository and logging verbosity: jingo.init({ repos: { main: ‘../scripts’ }, verbosity: ’debug’, timeout: 10000});
+
JS
MD
Getting Started with Jingo A Simple Reusable Module
Declare a single module that doesn’t require any other modules to do its work: jingo.declare({ name: 'Greeter’, as: function() { Greeter = function() { this.welcome = function(name) { alert('Hello ' + name + ', how are you?'); }; }; }});
+
JS
MD
Getting Started with Jingo A Namespaced Reusable Module
Unfortunately, the previous example pollutes the global namespace
Introduce a namespace called hallmart: jingo.declare({ name: ’hallmart.Greeter’, as: function() { hallmart.Greeter = function() { this.welcome = function(name) { alert('Hello ' + name + ', how are you?'); }; }; }});
Jingo ensures the existence of the enclosing namespace for us
+
JS
MD
Getting Started with Jingo A Reusable Module with Nested Namespacing
This pattern works for arbitrarily deep namespaces as well
Create an iteration module nested within a utilities package: jingo.declare({ name: ‘hallmart.util.iterators’, as: function() { hallmart.util.iterators = { each: function(data, closure) { for(key in data) { if(data.hasOwnProperty(key) { closure(key, data[key]); } } }; }; }});
+
JS
MD
Getting Started with Jingo A Reusable Module with Dependencies
Modules only have to declare their direct dependencies
Declare a dependent module: jingo.declare({ require: [ ‘hallmart.Greeter’ ], name: ‘hallmart.Store’, as: function() { hallmart.Store = function() { var greeter = new hallmart.Greeter(); this.admit = function(customer) { greeter.welcome(customer.name); }; }; }});
+
JS
MD
Getting Started with Jingo Declaring an Anonymous Module
<html>...<script type=“text/javascript” src=“scripts/jingo.js”></script><script type=“text/javascript”> jingo.anonymous({ require: [‘hallmart.util.iterators’, ‘hallmart.Store’], exec: function() { var each = hallmart.util.iterators.each; var customers = [{name: ‘Bil’}, {name: ‘Ted’}]; var store = new hallmart.Store(); each(customers, function(index, customer) { store.admit(customer); }; } });</script>...</html>
+
JS
MD
Getting Started with Jingo Rollup Module for Unobtrusive JavaScript
jingo.declare({ require: [‘hallmart.util.iterators’, ‘hallmart.Store’], name: ‘hallmart.controller.foo’, as: function() { var each = hallmart.util.iterators.each; var customers = [{name: ‘Bil’}, {name: ‘Ted’}]; var store = new hallmart.Store(); each(customers, function(index, customer) { store.admit(customer); }; }});
...<script type=“text/javascript” src=“scripts/jingo.js”/><script type=“text/javascript”> jingo.anonymous({ require: [‘hallmart.controller.foo’] });</script>...
+
JS
MD
Getting Started with Jingo Cross-Project Reuse via Multiple Module Repositories
Modularity creates opportunities for reuse beyond the confines of a single project: jingo.init({ repos: { mib: ‘http://mib.security.com/scripts’ }});
Require modules from multiple repositories: jingo.declare({ require: [ ‘mib:security.guard’, ‘hallmart.Greeter’ ], name: ‘hallmart.Store’, as: function() { // module body }});
+
JS
MD
Distinguishing Features
Cross-browser consistency Firefox, Internet Explorer, Safari, Chrome, Opera
Small footprint Download size < 6KB
Increased module load efficiency may lead to net decrease in overall network traffic
Standard caching Jingo module source is cached in the same manner as source
loaded via manual script tag entry
Debugging support Source code is attributed to the appropriate file resource
+
JS
MD
Distinguishing Features, continued
Framework-agnostic Focused, unbiased, standalone JavaScript dependency solution
Scope assurance Module Body Callbacks guarantee the presence of a non-global
enclosing scope in the body of every Jingo module
Namespace assurance Jingo automatically ensures the existence of enclosing
namespaces for namespaced modules
Intuitive project structure File system organization mirrors the logical module
structure
+
JS
MD
Ideas for the Future
Central script repository jingo.init({ repos: { jingo: ‘http://repos.jingo-js.com/scripts’ }});
jingo.declare({ require: [ ‘jingo:foo.bar.baz’ ], ...});
Jingo Server-Side Same syntax as Jingo but server-side process builds complete
dependency graph rollup and returns it as one file Server-side caching of assembled script rollups
Any suggestions?
+
JS
MD
Thanks Everyone
Particularly: Refactr (hosting)
Solution Design Group (pizza!)
+
JS
MD
Live Code Demo with John
jingo.googlecode.com
Sean Duncan [email protected]
John Bailey [email protected]
+
JS
MD
General-Purpose Frameworks
jQuery (www.jquery.com)
Prototype (www.prototypejs.com)
Dojo Toolkit (www.dojotoolkit.com)
ExtJS (www.extjs.com)
YUI (developer.yahoo.com/yui/)
Archetype (www.archetypejs.org)
…
+
JS
MD
Testing and Verification Tools
Unit Testing jqUnit (jqunit.googlecode.com)
JsUnit (www.jsunit.net)
Mock Objects jqMock (jqmock.googlecode.com)
JSMock (jsmock.sourceforge.net)
JSCoverage (siliconforks.com/jscoverage/)
JSLint (www.jslint.com)
Selenium (seleniumhq.org)
+
JS
MD
Other Valuable Tools
Compaction JSMin (www.crockford.com/javascript/jsmin.html)
Packer (dean.edwards.name/packer/)
Dojo ShrinkSafe (dojotoolkit.org/docs/shrinksafe)
YUI Compressor (developer.yahoo.com/yui/compressor/)
Embedded Documentation JSDoc Toolkit (code.google.com/p/jsdoc-toolkit/)