groovy & grails exchange 2012 - building an e-commerce business with gr8 technologies in latin...

Building an e-commerce businesswith gr8 technologies in Latin America

Groovy & Grails eXchange London14th December 2012

Domingo Suarez Torres@domix

lunes, 29 de abril de 13

Agenda

• About the Business

• Version 1

• Version 2

• Lessons learned


About me

• Java & Grails developer from México City

• grails.org.mx Mexican G&G community Founder

• My team won the second place in the Grails48 hackaton.

• TrollBoard. Kanban Board for GitHub Issues

• http://trollboard.rs.af.cm


clickOnero

• Started in 2010 as a Daily Deals website.

• German/Swiss investors

• Late 2011 acquired by Mexican investors

• Today it is a 100% Mexican company with a pure ecommerce model


clickOnero 1.0Grails, Terracotta, RabbitMQ & mySQL


clickOnero 1.0

• CodeName: CoolDeals

• Typical Grails application

• MySql, Tomcat, ehcache (Terracotta), RabbitMQ

• Built in 2 months by 4 devs from Germany, UK, Greece and Mexico


Caching

• Prevent excessive database access

• ehcache, backed with Terracotta

• Stored in cache

• Domain classes

• HTML pages & fragments

• JSON, XML documents


Long running code

• RabbitMQ

• Messaging with AMQP

• Asynchronous processing

• Mainly to use STMP services. Millions emails sent daily

• Financial reports (long complex queries)


CoolDeals Deployment

http://www.springstar.comlunes, 29 de abril de 13

http://www.springstar.com

http://www.springstar.com

Dev Team @ Mexico

• Very hard to find Grails developers

• Java developers -> Grails developers (4)

• It’s easy to do!

• New Grails developers excited about new possibilities and simplicity of the language and frameworks


Very Bad stuff• Test cases only for critical pieces

• checkout, user registration, mailing list subscription

• Very low code coverage.

• No Continuos integration

• Internal backend was a mess, a lot of code in controllers

• Bad GORM queries


Enters JavaMelody

• The goal of JavaMelody is to monitor Java or Java EE application servers in QA and production environments.

• It is a tool to measure and calculate statistics on real operation of an application depending on the usage of the application by users.

• ¡Grails Plugin!


JDBC Queries


JavaMelody

• It helps us improve our app!


More Plugins• RabbitMQ

• Spring Security

• Rendering

• Quartz

• Feeds

• Export

• Recaptcha

• AWS

• Joda Time

• Java Melody


Hardware


Boxes

• 4 WebServers

• DBServer

• Physical Firewall

• LoadBalancer


Numbers

• 200k-300k visits daily

• 1.5M > page views daily

• 80K concurrent users

• 20K-50K new users daily

• 1000-3000 purchases daily

• 2.5M registered users


Logstalgia-httpd log viz


“Ecommerce Company of the year in Mexico”


Lessons learned

• Cache is critical to scale

• The content in our app is ‘static’, few changes, sometimes no changes.

• If the content is ‘static’, why do we always execute code to render that content, even if the content is cached?


clickOnero 2.0Single Page Application (chaplin, backbone)


Motivation

• December 2011 clickOnero was acquired by Mexican investors

• The company started to change.

• The new CEO asked to develop a brand new store to sell different items like: clothes, shoes, trips, services (hair cuts, massages, cinema tickets, etc)


New markets• clickOnero acquired other e-commerce startups

to increase the market

• Miplan.com: was a e-commerce company specialized in offering national and international trips. 500k users

• PezUrbano is a daily deals website from Brazil, merging the two platforms is on the works. 1.5M users

• Very close to 5M users


Improvements

• Split the complete system in several components

• API: Grails app for exchange information

• Admin. Grails application for internal use

• Hipstore. A totally static HTML application built with Chaplin. Not a Grails app. The store for the users


Grails Apps

• API

• Typical Grails application, with no GSPs.

• Speaks only JSON

• Used by partners (remote services) and HipStore

• Admin

• Typical Grails application


Development enhancements

• Increased test cases in both Grails apps

• Spock

• Jasmine for JavaScript code

• Introduced Jenkins

• Jenkins Jobs to deploy automatically to QA & Production environments

• Bash shell scripts


HipStore

• Static HTML application built with Chaplin

• Chaplin is an architecture for JavaScript applications using the Backbone.js library


HipStore• Developed in CoffeeScript

• Uses PushState

• RequireJS (AMD Support)

• HandleBars (Template Engine)

• JQuery

• Underscore

• Twitter Bootstrap

• Build and packaged with Jake


HipStore

• Single Page Application

• Chaplin consumes JSON from the API to render the store items.

• Uses PushState to update the URL in the browser. Very useful for bookmarking and social media sharing, even for SEO.


Sample URLs

• http://{server}/store/

• http://{server}/store/{category}

• http://{server}/store/{category}/{campaign}

• http://{server}/store/{category}/{campaign}/{item}

• Above urls valid for the Chaplin router, but invalid for the browser, because them doesn’t exist. If you try to use it, you’ll get a 404


HashBang

• http://{server}/store/#!/{category}

• Not very friendly with search engines.

• Google will try something like:

• http://{server}/store?_escaped_fragment_=

• Please read: https://developers.google.com/webmasters/ajax-crawling/


https://developers.google.com/webmasters/ajax-crawling/




Our approach

• Write to disk all the possible links in HipStore. Crazy?

• We use ZombieJS to navigate the website and then write to disk the generated HTML

• Put those static files (HTML) in the webserver document root

• The best cache ever


Why do this?

• When the user visit our website, the webserver will respond with completely static HTML, CSS, JavaScript files

• Very fast

• The user never hits the Tomcats, we reduce the load in the app servers.

• The webserver always responds very fast


ZombieJS


HTML harvesting

• The Node app receives the JSON message

• Navigates to HipStore with ZombieJS

• Executes the JavaScript (Chaplin app)

• Extracts the generated HTML with jQuery

• Saves to disk in the web server document root


Things to consider

• When an item in the Store is modified, we need to regenerate the appropriate HTML file only once.

• Then all the users will receive the same file

• The user only hits the Tomcat when really need it (CheckOut, Registration)

• When the user click in some action in the app, the interaction is handled by Chaplin controller if the browser supports JavaScript

• Remember in our website the content is almost static?


Results

• The load in our Web Servers was reduce a lot.

• The load in the database reduced a lot.

• The users can share the links.

• Store becomes very search engine friendly


Some stats


Thanks!

• Q&A


groovy & grails exchange 2012 - building an e-commerce business with gr8 technologies in latin...

Technology

grails plugin

new grails developers

java grails developer

chaplin chaplin

grails apps spock jasmine

new users

new markets clickonero

daily deals website