october 2013 cassandra boulder meetup.key
DESCRIPTION
TRANSCRIPT
©2013 DataStax Confidential. Do not distribute without consent.
Cassandra 2.0!!Michael Shaler!Senior Director, Applications Business
Open Source Database Pedigrees
Key Value Stores
Amazon Dynamo
Google BigTable
Document DB JSON/XML DB
Graph Databases
MemcacheDB
Azure Table Services
Redis
Tokyo Cabinet
SimpleDB
Riak
Voldemort
Cassandra
Hbase
Hypertable
CouchDB
MongoDB
Neo4J
FlockDB
* Courtesy of @GuyHarrison
UserGrid
TitanDB
Common Use Cases
�3
cassandra
• Big data OLTP and write intensive systems!• Time series data management!• High velocity device data consumption and analysis!• Healthcare systems input and analysis!• Media streaming (music, movies, etc.)!• Online Web retail (shopping carts, user transactions, etc.)!• Online gaming (real-time messaging, etc.) !• Real time data analytics
• Social media input and analysis!• Web click-stream analysis !• Buyer event and behavior analytics!• Fraud detection and analysis!• Risk analysis and management !• Supply chain analytics !
• Web product searches !• Internal document search (law firms, etc.)!• Real estate/property searches !• Social media match ups!• Web & application log management / analysis
Cassandra as Foundation
Benefit Feature
Fully Distributed: no SPOF Peer-to-peer architecture for continuous availability and operational simplicity
Multi-Datacenter Node-, rack- and DC-aware with tunable consistency
Massively Scalable Multiple customers > 10M writes/second
SSD and Cloud optimized All writes are linear, and all files are immutable
Rich Application Data Model CQL (no joins or 2PC), integration with ODBC/JDBC et al
What’s New in Cassandra 2.0• Lightweight Transactions!
-IF keyword in CQL INSERT and UPDATE statements!-SERIAL consistency level!
• Triggers!-Phase 1 support!
• CQL paging support!• Prepared statement support: Atomic BATCH guarantees!• Bind variable support!• Improved authentication via SASL!• Drop column support (ALTER TABLE DROP)!• SELECT column aliases!• Conditional DDL!• Index enhancements!• One-off prepare and execute statements!• Performance enhancements!
-Off-heap partition summary!-Eager retries support!-Compaction improvements
�5
C* 2.0 Feature Highlights
• Lightweight transactions !• Triggers (experimental)!• Improved compaction!• CQL cursors
�6
The Problem: Sometimes we need Serializablity
Session 1
SELECT * FROM usersWHERE username = ’jbellis’[empty resultset]INSERT INTO users (...)VALUES (’jbellis’, ...)
�7
Session 2
SELECT * FROM usersWHERE username = ’jbellis’[empty resultset]INSERT INTO users (...)VALUES (’jbellis’, ...)
LWT: Details
• 4 round trips vs 1 for normal updates • Paxos state is durable • Immediate consistency with no leader
election or failover • ConsistencyLevel.SERIAL
• http://www.datastax.com/dev/blog/lightweight-transactions-in- cassandra-2-0
•
�8
Paxos for LWT
�9
LWT: Use sparingly, with caution
• Great for 1% of your application • Eventual consistency is your
friend
• http://www.slideshare.net/planetcassandra/c-summit-2013-eventual-consistency- hopeful-consistency-by-christos-kalantzis
•
�10
LWT example
INSERT INTO USERS (username, email, ...)VALUES (‘jbellis’, ‘[email protected]’, ... )IF NOT EXISTS;!UPDATE USERSSET email = ’[email protected]’, ...WHERE username = ’jbellis’IF email = ’[email protected]’;
�11
Some fine print…:)
• The columns updated do NOT have to be the same as the columns in the IF clause.
• Lightweight transactions are restricted to a single partition; this is the granularity at which we keep the internal Paxos state. As a corollary, transactions in different partitions will never interrupt each other.
• If your transaction fails because the existing values did not match the one you expected, Cassandra will include the current ones so you can decide whether to retry or abort without needing to make an extra request.
• ConsistencyLevel.SERIAL has been added to allow reading the current (possibly un-committed) Paxos state without having to propose a new update. If a SERIAL read finds an uncommitted update in progress, it will commit it as part of the read.
• For details of how we deal with failures, see the comments and code. • Tickets for DC-local transactions, updating multiple rows in a transaction, and
cqlsh support for retrieving the current value from an interrupted transaction are open and will be fixed for 2.0.1.
�12
Triggers
CREATE TRIGGER <name> ON <table> USING <classname>;
!class MyTrigger implements ITrigger
{ public Collection<RowMutation> augment(ByteBuffer key, ColumnFamily update)
{ ...
} }
�13
Triggers are EXPERIMENTAL!
• Relies on internal RowMutation, ColumnFamily classes
• [partition] key is a ByteBuffer • Expect changes in 2.1
•
�14
Compaction
• Single-pass, always• LCS performs STCS in L0
�15
Healthy Leveled Compaction
�16
Healthy leveled compaction
Sad Leveled Compaction
�17
Sad leveled compaction
STCS in L0
�18
STCS in L0
Cursors (before)! CREATE TABLE timeline ( user_id uuid, tweet_id timeuuid, tweet_author uuid, tweet_body text, PRIMARY KEY (user_id, tweet_id) );!SELECT * FROM timeline WHERE (user_id = :last_key
AND tweet_id > :last_tweet) OR token(user_id) > token(:last_key)
LIMIT 100
•
�19
Cursors (before)! CREATE TABLE timeline ( user_id uuid, tweet_id timeuuid, tweet_author uuid, tweet_body text, PRIMARY KEY (user_id, tweet_id) );!SELECT * FROM timeline WHERE (user_id = :last_key
AND tweet_id > :last_tweet) OR token(user_id) > token(:last_key)
LIMIT 100
•
�20
Other Performance Improvements
• Tracking statistics on clustered columns allows eliminating unnecessary sstables from the read path
• New half-synchronous, half-asynchronous Thrift server based on LMAX Disruptor
• Faster partition index lookups and cache reads by improving performance of off-heap memory
• Faster reads of compressed data by switching from CRC32 to Adler checksums
• JEMalloc support for off-heap allocation
�21
Clean-up
• Removed compatibility with pre-1.2.5 sstables and pre-1.2.9 schema
• The potentially dangerous countPendingHints JMX call has been replaced by a Hints Created metric
• The on-heap partition cache (“row cache”) has been removed
• Vnodes are on by default
The old token range bisection code for non-vnode clusters is gone
• Removed emergency memory pressure valve logic
�22
Operational Considerations
• Java7 is now required!
• Leveled compaction level information has been moved into sstable metadata
• Kernel page cache skipping has been removed in favor of optional row preheating (preheat_kernel_page_cache)
• Streaming has been rewritten to be more transparent and robust.
• Streaming support for old-version sstables
�23
Questions?!!
Answers?
�24