drupal mongodb integration benefits mobile media site with speed and scalability

CIGNEX Datamatics Confidential www.cignex.com

Scalable Mobile Media Site using Drupal & MongoDB

Presented By:

Yash Badiani, Big Data Practice Lead, CIGNEX Datamatics

Gaurav Khambhala, Technical Lead, CIGNEX Datamatics

30th July, 2013


About CIGNEX Datamatics

2


What Does CIGNEX Datamatics Do?

3

Since 2000, delivering Open Source

solutions for the enterprise through

adoption and integration to:

• Address business goals

• Increase business velocity

• Lower the cost of doing business

• Reduce TCO

• Gain competitive advantage

Portal Solutions Content

Solutions

Big Data Analytics Solutions

400+ Open Source Solutions

450+ Open Source Experts

200+ Open Source Integrations

12+ Books on Open Source

5000+ Community Contributions

Offices : America | India | UK | Europe | Singapore | Australia


Where We Can Help You

4

SOLUTIONS

Managed Cloud Services Develop • Deploy • Manage

VAR/Annual Product Subscription Liferay • Alfresco • Cloudera Hadoop • MongoDB

Extended Development Center Center of Excellence

UI • Development • Integration • Customization • Migration • Testing • Training • Support (24*7)

User eXperience Platform

Portals Liferay, Drupal, JBoss,

ZK, HTML5,

MuleSoft

• Intranet • Extranet • EAI • SOA

• Social Collaboration

• Big Data Portal • Mobile Portal

Enterprise Content Management

Content Alfresco, Adobe CQ,

Drupal, Magento,

JBoss, Moodle, Ephesoft,

Liferay

• WCM • DM • RM • CMS • DAM

• e-Commerce • e-Learning • ERP • Imaging Solutions

SERVICES

Making Data Work

Big Data Analytics

Hadoop Ecosystem,

MongoDB, Ne04j, Pentaho,

Talend, Solr, Jaspersoft

• Data Integration

• Information Delivery

• Data Analysis

• Enterprise Search

Enterprise Mobility Strategy • Mobile UX • App Development • MEAP/MDM


About the presenters

• Yash Badiani is the Big Data Practice Lead at CIGNEX Datamatics and focuses on Big Data Technologies including MongoDB & Hadoop. He has worked extensively on large Data warehousing & Business Intelligence projects with tools such as Business Objects, Microsoft SQL Server, Microstrategy, IBM Cognos.

• Gaurav Khambhala works at CIGNEX Datamatics as Technical Lead. He is the senior member of the PHP Practice at CIGNEX Datamatics and is involved on various technology initiatives like Big Data where he focuses on integration of PHP with NoSQL sources like MongoDB. He has a wide industry experience in software development & management in Open Source technologies such as Drupal, Moodle & Wordpress.

5


Agenda

• The Mobile Media Use Case

• Requirements and Challenges

• Solution : Mobile Media site using Drupal & MongoDB

• Why Drupal and MongoDB?

• Demo and Solution Features

• Benefits

• Summary

6


The Mobile Explosion!

7

By 2015, at least 60% of information workers will

interact with their content applications via a mobile

device

People use digital assets (videos, images) longer on

Tablets and Mobiles compared to desktops

Based on a report by a leading IT advisory firm

By 2017, 82% of all phones purchased will be

smartphones.


Mobile Media Use Case

• Mobile Media site includes the following features: – Store a variety of Images & associated metadata

– Massively Scalable to store billions of images

– Access through Mobile

– Create /Edit Albums

– Add Images to the Albums

– Add / Edit Metadata of Images

– Search Images / Albums by date, metadata, albums, etc

– Social Media features – Likes, comments

8


Requirements of Mobile Media sites

9

• Fast performance • Large user base • Concurrent CRUD • Access through various channels

Velocity

• Millions of digital assets • Variety of content • Complexity of data

Volume

• Rich UI features • Social features • Mobile access • Fast search

User experience

• Elastic scaling • Cost effectiveness • Centralized storage • Ease of Maintenance

Scalability

• HIGH availability • Automatic failover • User management

Security & Availability

• Easy integration • Shorter dev cycle • Faster deployment • Ease of schema design

Flexibility & Agility


Standard Three Layered Data Architecture

10

File System

Metadata in RDBMS

Search

Standard Three Layered Storage

Application layer


Introduction to NoSQL

• They are schema less

• Designed to support huge data volumes – Facebook 135 billion messages/month; Twitter 7TB data/day

• Scalable replication and distribution mechanism – Thousands of machines distributed around the world

• Massive write performance with asynchronous inserts and updates

• Designed to give high query performance

• Runs on commodity hardware

• Most NoSQL databases are Open Source

11

Not SQL Only


NoSQL – Data Models

12

Column Families

Usage: Read/Write Intensive

Popular databases: Hbase, Cassandra

Document Store

Usage: Working with Occasionally

changing/consistent data

Popular databases: CouchDB, MongoDB

Graph Database

Usage: Spatial Data storage, Social Application

Popular databases: Neo4j

Key Value / Tulip Store

Usage: Briskly changing data and high availability

Popular databases: Riak, Redis, Azure Table storage

NoSQL Databases

• 4 broad data models • 120+ variants available in the market


Requirements of Mobile Media sites - Recap

13

• Fast performance • Large user base • Concurrent CRUD • Access through various channels

• Millions of digital assets • Variety of content • Complexity of data

• Rich UI features • Social features • Mobile access • Fast search

• Elastic scaling • Cost effectiveness • Centralized storage • Ease of maintenance

• HIGH availability • Automatic failover • User management

Velocity Volume

User experience

Scalability

Security & Availability

• Easy integration • Shorter dev cycle • Faster deployment • Ease of schema design

Flexibility & Agility

Mobile Media Site


Drupal with MongoDB Solution

14

Themes

Core Modules

Nodes Taxonomy User Roles

Forms & Menu

Custom Modules

Workflow Forums Comments & Ratings

Tagging

PHP

Web Services

3rd party & Internal

Applications

MongoDB Driver

Mongos Routing Process

Replica Set

MongoDB MongoDB

Replica Set

MongoDB MongoDB


Why Drupal?

15

Pluggable Architecture

Data Abstraction Layer

Easy to Upgrade

Secure

Active Community Widely Adopted

Scalable

3rd Party Tools Integration

User Management & Permissions

HTML5 & CSS Support

Drupal Open Source CMS


Websites using Drupal

16

Website: Whitehouse.gov Website: Data.gov.uk

Website: mtv.co.uk

Website: research.yahoo.com

Website: pdx.edu Website: EndPoverty2015.org


Why MongoDB?

17

Agile and Scalable

Full Index Support

Document Oriented Storage

Replication Querying

Atomic Updates

Data Processing and aggregation

High Availability

Open Source NoSQL Solution


Customers using MongoDB

• Centralized data management platform

• 2 billion+ documents

• 20 TB of photo metadata

• TV episodes and series

• Risk solutions auditing data

18

Source: http://www.10gen.com/customers


Demo

19

• Media site on mobile simulator • Like & comment on an image on mobile simulator • Mobile site on web browser • Verify ‘Like’ & comment of the same image on web browser • Search images & access control


Solution Features

Architecture and Design

20


Architecture

21

User Metadata In

de

xes

Albums

Image Metadata

GridFS

Form API

Drupal API

Custom Module

Browser/Mobile Theme

MongoDB PHP Driver

Menu API

User Mobile Device

Image Metadata

GridFS


Add Album

Flow

22

Add Image

View Album

View Individual Images

Like Image

Comment Image

Add Tags to Images

View Counter

Search Images By Tags

User Metadata Albums GridFS Image Metadata

DBRef

DBRef DBRef

DBRef

MongoDB Collections

Use

r A

ctio

ns


Schema Design

23

User Metadata GridFS

Albums Image Metadata

• User ID • DBRef (Album) • Tags • Thumbnail • Likes • View Counter • Comments • Permission

• FS.Files • FS.Chunks

• User ID • Tags • Title

• Make • Model • Date Time • Aperture • Exposure • DBRef (GridFS)


Schema Design

24

Image Metadata Albums

User Metadata


MongoDB Monitoring Service (MMS)

25

• DB Storage

• Cursors

• Replica Sets

• Network Connections

• Non Mapped Virtual Memory

• Opcounters


Benefits

Drupal MongoDB

Most advanced content management solutions

Scalability – billions of content items, millions of users

Highly customized websites Performance – FAST writes through sharding, reads through indexes

Most search friendly CMS Safety - Replication

Less coding, high on automation Centralized Data Storage - Single System

Powered by 7000+ plugins and extensions

Monitoring & Administration - MMS

Active community, real time assistance

Enterprise Support - 10gen

26


Summary and Key Takeaways

• MongoDB provides the RIGHT fit for CMS applications with flexibility, scale & speed

• Drupal’s advanced & automated CMS features and tight integration with MongoDB makes it the right choice for building agile websites

• Both Drupal & MongoDB are feature rich and being Open Source, provide significant cost benefits

27


To know how Open Source can be adopted and integrated into your enterprise

Sign Up for Free Assessment Workshop http://www.cignex.com/freeassessment

Thank You. Any Questions ?

Making Open Source WorkTM

Yash Badiani Big Data Practice Lead

[email protected]

Gaurav Khambhala Technical Lead

[email protected]

[email protected]

http://www.cignex.com/freeassessment

http://www.cignex.com/freeassessment

mailto:[email protected]

mailto:[email protected]

drupal mongodb integration benefits mobile media site with speed and scalability

Technology