and oltp use cases with apache ignite accelerate mysql for … · 2017-04-27 · accelerate mysql...

Accelerate MySQL for Demanding OLAP and OLTP Use Cases with Apache Ignite

Peter Zaitsev, Denis Magda Santa Clara, California | April 25th, 2017

About the Presentation

Problems

Existing Solutions

Denis Magda will show the power of Apache Ignite

My Conviction

There is no silver bullet in technology!

Why ?

All design decisions comes with their own benefits and drawbacks

Technologies not Technology

Large Scale applications tend to use more than one technology on

data layer

Works especially well with Open Source!

Additional Components do not require hefty license fees

Easy to prototype and test out

Open Source Community is good at building bridges

Balance is Needed

•Use as many technologies as you need, but no more

MySQL

•MySQL is no Exception. It is not Great for Everything.

Some of the Problems

Hot Data

Highly Volatile Data

Large Data Volume

Analytical Processing

Full Text Search

Hot Data

For example “Cache”

Large volume of simple requests

High overhead due to SQL

No good Memory focused Engine

Not Designed for very high Concurrency

Solutions

•MySQL•MemcacheD interface•Thread Pool

•External•MemcacheD•Redis

Highly Volatile Data

Lots of updates, especially to a single row

Design around full Transactional ACID semantics

Disk Log based durability

Pessimistic Logging

Solutions

•MySQL•Data Design•Configuration Tuning•Parallel Replication

•External•MemcacheD•Redis

Large Data Volume

MySQL is designed as single node system

Limited in CPU, Memory

Manual “Sharding” solutions are painful

Especially with complex queries

Solutions

•MySQL•Manual Sharding•Vitess•ProxySQL

•External•Shading for MemcacheD and Redis•MongoDB•Cassandra

Analytics (OLAP)

MySQL does not support column based storage

MySQL optimizer is limited for complex queries

MySQL does not do parallel query execution

MySQL does not do distributed query execution

Solutions

•MySQL•Configuration and Schema Design (Limited)

•External•Hadoop & Spark•Vertica•ClickHouse

Full Text Search

Can handle basic Full Text Search

Does not scale well with data volume

No parallel processing

Limited search relevance options

Hard To do GIS searches; Facets

No language processing

Solutions

•MySQL•Small Scale search applications only•Supported with Innodb tables since MySQL 5.6

•External•Elastic •Solr•Sphinx

New Solutions constantly appear

•Always be on lookout for a better solutions!

DENIS MAGDAProduct Manager, Apache PMC Chair

Apache Ignite - In-Memory Data FabricBeyond the Data Grid

www.ignite.apache.org #apacheignite

• In-Memory Data Fabric• Advanced Clustering• In-Memory Compute Grid• In-Memory Data Grid• In-Memory SQL Grid• In-Memory Service Grid• In-Memory Streaming & CEP• Plug-n-Play Hadoop Accelerator

IN-MEMORY DATA FABRICSTRATEGIC APPROACH TO IMC

IN-MEMORY COMPUTING WHY NOW?

ADVANCED CLUSTERING & DEPLOYMENT

Ease of Getting StartedAutomatic Discovery

Any EnvironmentPublic CloudPrivate CloudHybrid CloudLocal Laptop

Zero-DeploymentAuto-Deploy Code

Full Cluster Management

Pluggable Design

IN-MEMORY COMPUTE GRID

Direct API for MapReduce

Zero Deployment

Cron-like Task Scheduling

State Checkpoints

Load Balancing

Automatic Failover

Full Cluster Management

Pluggable SPI Design

IN-MEMORY DATA GRID

Distributed In-Memory Key-Value StoreReplicated and Partitioned dataTBs of data, of any typeOn-Heap and Off-Heap StorageHighly Available In-Memory ReplicasAutomatic Failover Distributed ACID Transactions SQL queries and JDBC driverCollocation of Compute and Data

IN-MEMORY SQL GRID

ANSI-99 SQL CompliantAggregations, group by, sortingCross-cache joins, unions, etc.

DML and DDL

JDBC, ODBC, Native APIs

DistributedAlways consistentFault tolerant

IN-MEMORY SERVICE GRID

Distribute Any Data StructureAvailable Anywhere on the GridAutomatic Remote Access via Proxies

Controlled DeploymentSupport for Cluster SingletonSupport for Node SingletonSupport for Custom TopologyLoad Balanced

Guaranteed AvailabilityAuto Redeployment in Case of Failures

IN-MEMORY STREAMING AND CEP

Streaming Data Never Ends

Branching Pipelines

Pluggable Routing

Sliding Windows

Real Time Analysis

IN-MEMORY HADOOP ACCELERATOR

Plug and Play installation

2x to Nx Acceleration

In-Memory Native MapReduce

In-Process Data Colocation

IgniteFS In-Memory File System

Read-Through from HDFS

Write-Through to HDFS

Sync and Async Persistence

SPARK INTEGRATIONSHARED RDD & IN-MEMORY FILE SYSTEM

Turbocharge your SQL queries in-memory with Apache® Ignite™ 25 April - 4:50 PM - 5:15 PM @ Room 210

THANK YOU!

www.ignite.apache.org #apacheignite

Thank You to All of Our Sponsors!!