© copyright 2014 marklogic corporation. all rights reserved. marklogic overview of key features

© COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

MarkLogicOverview of Key Features

Geospatial Support

Full-textSearch

Flexible Indexes

Native JSONStore

Native XML Store

Real-time Alerting

Native RDF Triple Store

Bitemporal

Tiered Storage

Fully Transactional

Server-side JavaScript

Hadoop and HDFS

CloudReady (AWS)

SQL Support

Scalable and Elastic

MarkLogic Content Pump

REST API

Samplestack

Ad-hoc Queries

Schema Agnostic

XA Transactions

24/7EngineeringSupport

LDAP and KerberosSecurity

Security Certifications

ConfigurationManagement

Monitoring and Management

Performance at scale

Customizable Failover

Customizable Backup

Atomic Forests

Point-in-time Recovery

ACIDTransactions

Index Across Data Types

Flexible Replication

Semantic Inference

Multi-OS Support

POWERFUL AGILE TRUSTED

MarkLogic / Enterprise NoSQL Database Platform

MarkLogic 7

Semantics FoundationNext-gen Infrastructure Support ElasticityTiered StorageContinue Hadoop RoadmapRun on HDFS

MarkLogic 6

AccessibilitySQL/BIJava/REST/JSONUDFs/AnalyticsmlcpHadoop DistributionsHDFS Tech Preview

MarkLogic Server 3.1

Advanced Search FeaturesWildcard queriesDirectoriesForward CompatibilitySupport for Sun SolarisXCC

MarkLogic 8

JSON StorageServer-side JavaScriptSemanticsBitemporalSamplestackJava Client APINode.js Client APIManagement APIIncremental BackupFlexible ReplicationEnhanced HTTP Server

MarkLogic Server 4.2

ReplicationFailoverDatabase RollbackCompartment SecuritySearch OptimizationsSearch APIInformation StudioApplication Builder

MarkLogic Server 4

AlertingEntity EnrichmentGeospatialAnalytics (co-occur., value lexicons, bucketing)Modular documentsSecurity auditingHA: forest-level failover

MarkLogic Server 5

Complete Enterprise RoadmapDatabase ReplicationMulti-statement and distributed transactionsPoint-in-time recoveryStart Hadoop RoadmapHadoop Connector

MarkLogic Server 2

ClusteringRole-based security w/BASIC authenticationDocument CollectionsEnhanced Search (stemming, thesaurus, wildcard)WebDAV supportDocument lockingEnhanced XDBC support

MarkLogic Server 3

Advanced Search FeaturesContent processing (including PDF, Word, Excel, PPT)HTTP callsFailoverSupport for Linux, Windows Server, .NET

Cerisent XQE Server 1

ACID TransactionsText Based SearchBackup and RestoreLinux SupportWeb-based ProtocolsHTTP and XDBC XQuery

2003 2004 2005 2006 2010 2011 20122008 2013 2014

Continuous Innovation

Enterprise NoSQL Database Platform

Flexible Data ModelStore and manage JSON, XML, RDF, and Geospatial data with a document-centric, schema-agnostic database

Scalability and Elasticity

ACID Transactions

Search andQuery

Semantics CertifiedSecurity

Hadoop Integration

Scale to petabytes of data without over-provisioning or over-spending

Avoid data loss, data corruption, and stale reads—even at speed and scale

Lightning fast, sophisticated, sub-second search and query across all of your data

Store and query linked data as RDF and SPARQL

Make your Hadoop better by connecting it to MarkLogic

Government-grade, granular, role-based security

© COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 5

Flexible Data Model

JSON, XML, RDF, Geospatial data, and also large binaries—all stored and managed on a single unified platform

Document-centric and schema-agnostic for agility, reducing lost fidelity and functionality from data conversion and brittle ETL

Use the data format that makes the most sense, keeping the data in its most readable form

Store and manage JSON, XML, RDF, and Geospatial data with a document-centric, schema-agnostic database


Flexible Data ModelSchema-agnostic, structure-aware

<report>

<title> Suspicious vehicle…Suspicious vehicle near airport

<date>

<type>

<threat>

2012-11-12Z

observation/surveillance

<type>suspicious activity

<category>suspicious vehicle

<location>

<lat>37.497075

<long>-122.363319

<subject>IRIID

<subject>IRIID

<predicate>

<predicate>

isa

value

<triple>

<triple>

<object>license-plate

<object>ABC 123

<description>A blue van…A blue van with license plate ABC 123 was observed parked behind the airport sign…

</title>

</date>

</type>

</type>

</category>

</threat>

</lat>

</long>

</location>

</subject>

</subject>

</predicate>

</predicate>

</object>

</object></description>

</triple>

</triple>

Unstructured full-text

Geospatial

Values

Semantic

(RDF) Triples


Search and QueryBuilt-in search to find answers in documents, relationships, and metadata

In MarkLogic, a search is a query, and a query is search Ingest your data as-is and rely on over 30 sophisticated

indexes to get better answers from today’s data Lightning fast, sub-second search across hundreds of

terabytes of data and billions of documents Powerful, agile development providing complex query

capability across heterogeneous data Full-featured UX with full-text search, type-ahead

suggestions, facets, snippeting, highlighted search terms, proximity boosting, relevance ranking, and language support

JavaScript XQuery SPARQL

Rich Query Capability

In-databaseMapReduce

Full-text Search

Semantic Search

Geospatial Search


Universal Index

“blue” 123, 125, 129, 130, 152, 344, …

“van” 123, 125, 126, 129, 130, 152, …

“observed” 125, 152, 516, 522, 765, 890, …

“blue van” 123, 125, 129, 130, 152, 486, …

STEM “observe” 125, 152, 516, 522, 765, 890, …

<report> …

<report>/<location> …

<threat>/<category> …

<type>suspicious activity</type> …

<date>2012-11-12Z</date> …

Collection:Vetted …

Role:Analyst + Action:Read …

… …

… …

… …

Term Term List

MarkLogic indexes… Words

Phrases

Stemmed words and phrases

Structure

Words and phrases in the context of structure

Values

Collections

Security Permissions

Document References

125, 516, 890, …

Which vetted reports contain the phrase blue van?


Range Index

“blue” 123, 125, 129, 130, 152, 344, …

“van” 123, 125, 126, 129, 130, 152, …

“observed” 125, 152, 516, 522, 765, 890, …

“blue van” 123, 125, 129, 130, 152, 486, …

STEM “observe” 125, 152, 516, 522, 765, 890, …

<report> …




<date>2012-11-12Z</date> …



… …

… …

… …

Term Term List Document References

125, 516, 890, …

Which vetted reports containing the phrase blue van were submitted before 2014?

Range indexes map document IDs to values, and vice-versa in a compact in-memory representation.


Which vetted reports about a blue van from before 2013 with this location refer to partial plate ABC?Which vetted reports about a blue van from before 2014 refer to a location near the airport?Triple IndexGeospatial Index

“blue” 123, 125, 129, 130, 152, 344, …

“van” 123, 125, 126, 129, 130, 152, …

“observed” 125, 152, 516, 522, 765, 890, …

“blue van” 123, 125, 129, 130, 152, 486, …

STEM “observe” 125, 152, 516, 522, 765, 890, …

<report> …




<date>2012-11-12Z</date> …



… …

… …

… …


125, 516, 890, …

The Geospatial index is like a 2D range index, with built-in query support for point, box, circle, and complex polygons.


Which vetted reports about a blue van from before 2013 with this location refer to partial plate ABC?

The Triple index is an index of “facts” expressed as Semantic triples. It can efficiently query and join billions of “linked data” triples.

All indexes fully

composable

for sub-second

response to

very complex queries

Triple Index

“blue” 123, 125, 129, 130, 152, 344, …

“van” 123, 125, 126, 129, 130, 152, …

“observed” 125, 152, 516, 522, 765, 890, …

“blue van” 123, 125, 129, 130, 152, 486, …

STEM “observe” 125, 152, 516, 522, 765, 890, …

<report> …




<date>2012-11-12Z</date> …



… …

… …

… …


125, 516, 890, …


SemanticsEnterprise triple store, document store, and database combined

Store and query billions of facts and relationships; infer new facts

Facts and relationships provide context for better search Flexible data modeling—integrate and link data from

different sources Standards-based for ease of use and integration

– RDF, SPARQL, and standard REST interfaces


Scalability and ElasticityMassive enterprise scalability and elasticity

Scale horizontally in clusters on commodity hardware to hundreds of nodes, petabytes of data, and billions of documents

Process thousands of transactions per second with distributed XA transactions

Start small and scale up or down to meet capacity and performance demands without over-provisioning or over-spending

Even better with Tiered Storage

D-NODE D-NODE

E-NODE E-NODE

D-NODE

Add nodes

to scale out

Automated

failover


ACID TransactionsDon’t settle for a BASE-ic database

Reads and writes are durably logged to disk, and strongly isolated from other transactions

Prevents data corruption, stale reads, and inconsistent data—common problems with databases that settle for eventual consistency—and all of which are unacceptable

No performance drop-offs at scale. Production applications run tens of thousands of very complex transactions per second for tens of thousands of users

Accomplished using MVCC (multi-version concurrency control)


/articles/doc1.xml

∞ ∞423 628

Creation Timestamp Deletion Timestamp

Year

∞ ∞

Document

Title Author

Section

Section Section Section Section Section

FirstLast

Metadata

628

/articles/doc1.xml

Document

Title Author

Section

Section Section Section Section Section

FirstLast

Metadata

MVCC Benefits: ACID transactions

Zero-latency search indexing

High throughput

Lock-free reads

Serial writes

Point-in-time query

Fast database rollback

ACID TransactionsImplemented Using MVCC


Government-Grade SecurityCertified, granular security for modern data governance

Certified security – Higher security certifications than any other NoSQL database, carrying a Common Criteria Security Certification and being certified to run in classified government systems

Granular Security – Role Based Access Control (RBAC) at the document level, and can also employ other models for cell-level security

Privacy

Provenance

Compliance

Continuity

Retention

Security

Data Governance With MarkLogic


MarkLogic and HadoopMake Hadoop better by connecting it to MarkLogic

Handle both operational and analytical workloads

Run MarkLogic directly on HDFS on data staged in HDFS

Leverage the economics of HDFS by using it as a storage tier for archival data

Connect MarkLogic to Hadoop to run large-scale MapReduce jobs for ETL, analytics, or enrichment

MORE FEATURES


Tiered StorageMove data to different tiers based on cost and performance trade-offs

Use a fluid mix of flash storage, traditional local or shared disk storage, HDFS, or Amazon cloud storage

Migrate data automatically between storage tiers without any ETL, additional software, or expensive infrastructure changes

Optimize data availability while reducing the costs of storage

Manage data across the information lifecycle


High Availability and Disaster RecoveryKeep your data continuously safeguarded and available

HA using shared-disk or local-disk failover

– Shared-nothing architecture so there is no single point of failure

– ACID transactions ensure full redundancy and consistency

DR using customizable database replication

– Point-in-time recovery with journal archiving

– Incremental Backup consumes less storage and can be completed quickly

Database ReplicationFull database replication with journal frames that enable point-in-time disaster recovery


Cloud DeploymentGet started quickly on AWS for only 99 cents an hour

Build a single cluster in minutes on Amazon Web Services using pre-packaged AMIs

Easily and quickly scale up or back down

Blend on-premise, virtualized, and AWS nodes in a single cluster, and scale out without downtime

Flexibility to move licenses across your environment as changes occur

MARKLOGIC 8 FEATURES

Developer Experience

JSONUnified indexing and query for today’s web and SOA data

Server-side JavaScriptJavaScript runtime inside MarkLogic using Google’s V8

Node.js Client APIEnterprise NoSQL database for Node.js applications

Java Client APINoSQL agility in a pure Java interface

MarkLogic 8 is more powerful than ever, but remarkably easy to use

Enterprise triple store, document store, database combined

Semantics

BitemporalTrack information along two dimensions of time

MarkLogic 8 / More Powerful, Easier to Use


JSONUnified indexing and query for today’s web and SOA data

Speed up development with powerful built-in search, transformation, and alerting capabilities designed for JSON

Reduce lost fidelity and functionality from data model translations and brittle ETL

Simplify architecture with data, metadata, and relationships managed consistently and securely together

Ease modern, end-to-end JavaScript development

123456789101112131415

{ "_id": 1, "name": { "MarkLogic" }, "supports" : [ { "datatype": "XML", "year": 2003 }, { "datatype": "JSON", "year": 2014 } ] }


Node.js Client APIEnterprise NoSQL database for Node.js applications

Focus on application features rather than plumbing with out-of-the-box search, transactions, aggregates, alerting, geospatial, and more

Move faster to production with proven reliability at scale

Maximize performance and flexibility—bringing code to the data

Enable modern end-to-end JavaScript development

Always open source on GitHub

Participate.Contribute.Fork it.


Java Client APINoSQL agility in a pure Java interface

Faster development and less custom code with out-of-the-box data management, search, and alerting

Pure Java query builder and conveniences for POJOs, JSON, XML, and binary I/O

Built-in extensibility for moving performance-critical code to the database

Always open source and developed on GitHub




Front End

Middle Tier

Database Layer

Run code near the data for unparalleled power, efficiency

Build applications faster from a growing pool of skills, tools

Reduce risk with proven performance and reliability

Decrease brittle ETL and lost fidelity and functionality from JSON data conversions

Pair with Node.js to ease full-stack JavaScript development

+

JavaScript runtime inside MarkLogic using Google’s V8


An end-to-end three-tiered application in Java and Node.js

Front End

Middle Tier

Database Layer

Encapsulates best practices and introduces key MarkLogic concepts

Use sample code as a model for building applications more quickly

Modern technology stack shows where MarkLogic fits in your environment


Samplestack


SemanticsEnterprise triple store, document store, database combined

Store and query billions of facts and relationships; infer new facts

Facts and relationships provide context for better search

Flexible data modeling—integrate and link data from different sources

Standards-based for ease of use and integration

– RDF, SPARQL, and standard REST interfaces Even better with Built-in Search and Bitemporal

– Triples, documents, and data combined


BitemporalTiming is everything

Va

lid

Tim

e

System Time

Valid Time – Real-world time, information “as it actually was”

EVENT 1

EVENT 3

EVENT 2

EVENT 2

System Time – Time it was recorded to the database

Rewind the information “as it actually was” in combination with “as it was recorded” at some point in time

Provides increased insight into your business and mission

Capture evolving schema as the shape of the data changes with changing time, a capability that has prevented relational bitemporal offerings from being widely adopted

Critical for anyone in regulated industries Even better because of Tiered Storage and

Semantics


Management APIREST-based API to manage all MarkLogic capabilities

Increase efficiency and agility by automating time-consuming repetitive tasks across production, testing and development

Reduce setup time and admin error by orchestrating multi-step configurations and deployments

Fit more seamlessly into IT environments by using REST interfaces unlike CLI or proprietary APIs

Perform automated testing and monitor performance using market tools that support REST

Even better with Client REST API, Elasticity


Incremental Backup

SUNDAY MONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY SATURDAY SUNDAY

FULL FULL

INCREMENTAL BACKUP (differential)

Store only changes since the previous full or incremental backup Consume less storage for backup copies Reduce backup window Improve availability with multiple daily backups Work with Log Archiving to enable fine-grained point-in-time recovery

Faster backups while using less storage


Flexible ReplicationCustomizable information sharing between systems

Enable content collaboration across numerous systems

Support directly connected or mobile users

Provide data that users need using simple configurable parameters or queries

Ensure data consistency and security with simple workflows

Even better with Bitemporal and Management API


Enhanced HTTP Server

Use a single interface when employing the REST API, custom HTTP, XCC/XDBC to connect to any database

Delivers ease-of-use by not having to create extra ports

Simplifies the out-of-the-box interaction and can improve the performance of client/server

Provides an improved and more efficient developer experience with MarkLogic

Simple and fast client-server interactions out-of-the-box

APPENDIX

MarkLogic / Enterprise NoSQL Database Platform

Better answers from today’s dataMarkLogic is built to find answers in

documents, relationships, and metadata

Adaptive to every environmentMarkLogic runs well everywhere, while

preserving the option to change

hardware, data, and scale later

Simpler data integrationMarkLogic accelerates and simplifies

data sharing across silos, cutting down

on ETL and making agile development

possible

Hardened, proven platformMarkLogic has a proven track record of

performance under all enterprise

conditions

Uncompromised data resiliencyMarkLogic will keep your data safe and

whole—no matter what happens in your

application or at your data center

POWERFUL AGILE TRUSTED

The intelligent data layerAn intelligent data layer powers intelligent

applications—and makes them faster and

more flexible than any alternative

Store and manage data natively as XML documents, a hierarchical self-describing data type that is ideal for a wide variety of applications.

Geospatial Support

Full-textSearch

Flexible Indexes

Native JSONStore

Native XML Store

Real-time Alerting

Native RDF Triple Store

Bitemporal

Tiered Storage

Fully Transactional


Semantic Inference

// POWERFUL / Deliver more value, build better apps

Store geospatial data such as GML, KML, and GeoRSS and do complex queries on the data or in combination with other data types. Also integrate with ESRI ArcGIS and Google Maps for visualization.

Store RDF triples and query them using SPARQL—providing context to your data and better search with a database that can handle a combination of documents, data, and triples.

Store and manage data natively as JSON documents, speeding up development and reducing data transformation with a simplified architecture for end-to-end JavaScript development.

Create an unlimited number of real-time alerts by email or text using the alerting API and reverse indexes. Whenever a document is loaded that matches a specific query, you’ll know.

Run complex distributed transactions across multiple documents and collections with no performance drop-offs at scale. Production applications run tens of thousands of transactions per second for tens of thousands of users.

Work with new data that didn’t exist before. For example, if John lives in London, and London is in England, then MarkLogic can infer that “John lives in England” and then add that new fact to your semantic search.

Store and manage data in different tiers based on cost and performance trade-offs, and easily migrate between tiers without any ETL, additional software, or expensive infrastructure changes.

Live in JavaScript. Run JavaScript near the data for unparalleled power and efficiency with a high performance JavaScript runtime inside MarkLogic using Google’s V8.

Built-in, lightning fast search and query capabilities across hundreds of billions of documents. And, full-featured UX with type-ahead suggestions, facets, snippeting, relevance ranking, and language support.

Rely on over 30 sophisticated, composable indexes including a universal index, range index, geospatial index, and triple index—all designed so that developers can ask harder questions and get faster responses.

Handle historical data along two different timelines, making it possible to rewind the information “as it actually was” in combination with “as it was recorded” at some point in time.

Use MarkLogic’s cloud templates to get up and running quickly on AWS or other cloud environments, starting with a three node cluster or a large cluster with over a hundred nodes.

REST API

SQL Support

Multi-OS Support

Scalable and Elastic

CloudReady (AWS)

Samplestack

Hadoop and HDFS

Schema Agnostic

XA Transactions

Index Across Data Types

Ad-hoc Queries

MarkLogic Content Pump

// AGILE / Prepare for and respond to change

Configure and administer MarkLogic with a single REST-based API. This provides more programmatic control than ever before—giving DBAs the power and flexibility necessary to run a modern data center.

Make Hadoop better by connecting it to MarkLogic and using it as part of an infrastructure to handle both operational and analytic workloads.

Handle petabytes of data without over-provisioning, over-spending, or experiencing downtime, inconsistency, or risk of data loss.

Get going fast on MarkLogic with Samplestack, an end-to-end three tiered sample application designed to show developers how to implement a reference architecture using key MarkLogic concepts and sample code.

Use multiple indexes in concert across multiple data types—giving you the power to search and query all of your data.

MLCP makes it easy to quickly import or export documents and metadata from MarkLogic, or to copy from one database to another using a command-line tool.

Run distribute transactions across a cluster using the XA (eXtended Architecture) standard, which ensures ACID properties for global transaction processing.

Don’t plan your queries in advance of ingesting your data. MarkLogic is designed for search and discovery so that you can run any query at any-time and get real-time results.

Use a relational SQL data model within MarkLogic, connecting to SQL-based tools using the ODBC driver, or execute SQL commands against relational databases using the MLSAM open-source XQuery library.

Run MarkLogic on Windows, Linux, Solaris, OS X. MarkLogic runs easily and is easy to setup in your environment, whether in the cloud, virtualized, or on premises.

Only use schema when you need it. Ingest all your data as-is, whether structured or unstructured, using the NoSQL document model rather than being forced to use a predefined schema.

Use third party authentication from LDAP or Kerberos, making the most secure NoSQL database easier to manage.

Monitoring and Management

ConfigurationManagement

24/7EngineeringSupport

Performance at scale

LDAP and KerberosSecurity

Flexible Replication

Security Certifications

ACIDTransactions

Customizable Failover

Atomic Forests

Point-in-time Recovery

Customizable Backup

// TRUSTED / Enterprise-ready for mission-critical uses

Use the Management API for cluster management, process automation, access controls, database cloning, audit trails, and connections to third-party interfaces.

Secure your data with government-grade security. MarkLogic has certified, granular security for modern data governance and to handle the increased complexity of today’s cyber threats.

Scales easily to handle hundreds of terabytes using shared-nothing architecture in which data partitions are completely independent of each other and can act independently.

Enable customizable information sharing between systems, allowing for the easy and secure distribution of portions of data even across disconnected, intermittent, and latent networks.

Manage data in collections of documents similar to partitions, called forests, that exist independently and enable scalability and elasticity, rebalancing, efficient operations, and easier data governance.

Restore the database quickly with minimal downtime, relying on full and consistent backups, hot configuration changes, and automatic index optimization without shutting down the system.

Have confidence that your data is always available, reducing risk and avoiding interruptions with automated local- or shared-disk failover made possible with shared-nothing architecture.

Rollback to a specified point in time by replaying journal archives, an additional feature to ensure disaster recovery and easy of management.

View and manage the configuration settings for MarkLogic databases, forests, application servers, groups or hosts—and easily propagate changes across the entire cluster.

Rely on support from the 24/7, all-engineer support staff to ensure you get answers fast, or just want some friendly tips on saving a few milliseconds on performance.

Don’t settle for a BASE-ic database. Use ACID transactions to ensure you don’t run the risk of encountering data corruption, stale reads, and inconsistent data—all of which are unacceptable.

© copyright 2014 marklogic corporation. all rights reserved. marklogic overview of key features

Documents