welcome to apache cassandra day silicon valley 2014: keynote by chief evangelist, patrick mcfadin
DESCRIPTION
Keynote Kick-Off for Apache Cassandra Day Silicon Valley 2014TRANSCRIPT
![Page 1: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/1.jpg)
©2013 DataStax Confidential. Do not distribute without consent.
@PatrickMcFadin
Patrick McFadinChief Evangelist for Apache Cassandra
Keynote: The State of Cassandra
1
![Page 2: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/2.jpg)
Five Years of Cassandra
Jun-09 Mar-10 Jan-11 Nov-11 Sep-12 Jul-13
0.1 0.3 0.6 0.7 1.0 1.2...
2.0
DSE
Jul-08
![Page 3: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/3.jpg)
![Page 4: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/4.jpg)
•Massively scalable •High performance • Reliable/Available
Core values Cassandra HBase Redis MySQL
![Page 5: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/5.jpg)
CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int);!!CREATE INDEX ON users(state);!SELECT * FROM users WHERE state=‘Texas’ AND birth_date > 1950;
New Core Value
•Massively scalable •High performance •Reliable/Available •Productivity + ease of use
![Page 6: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/6.jpg)
CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int);
CREATE TABLE users_addresses ( user_id uuid REFERENCES users, email text);!SELECT *FROM users NATURAL JOIN users_addresses;X
Collections
![Page 7: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/7.jpg)
UPDATE usersSET email_addresses = email_addresses + {‘[email protected]’, ‘[email protected]’};
CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int, email_addresses set<text>);
Collections
![Page 8: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/8.jpg)
Cassandra Community
![Page 9: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/9.jpg)
Your community
![Page 10: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/10.jpg)
We are growing!
![Page 11: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/11.jpg)
Great projects!
•Storage backend for Graphite •Replacement of whisper •Plugs into carbon •Real users: Love it!
Cyanite
https://github.com/pyr/cyanite
![Page 12: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/12.jpg)
Great projects!
The Kiji Project
http://www.kiji.org/2014/02/11/working-on-cassandra-integration-with-datastax/
•Real-time application framework •Product and content recommendation systems •Risk analysis and fraud monitoring •Customer profile and segmentation applications
![Page 13: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/13.jpg)
Get Involved!
•Talk at a meet up •5 minute interview? •Stack overflow •Mailing list •Apache Jira
![Page 14: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/14.jpg)
Cassandra 2.0
![Page 15: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/15.jpg)
Reads in a cluster
Client Coordinator
40% busy
90% busy
30% busy
![Page 16: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/16.jpg)
A failure
Client Coordinator
40% busy
90% busy
30% busyXtimeout
![Page 17: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/17.jpg)
Rapid read protection
Client Coordinator
40% busy
90% busy
30% busyXsuccess
![Page 18: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/18.jpg)
Rapid Read Protection
NONE
![Page 19: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/19.jpg)
Latency (mid-compaction)
![Page 20: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/20.jpg)
Cold data
10,000 req/s 5,000 req/s
4,000 req/s 10 req/s
![Page 21: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/21.jpg)
Cold data
10,000 req/s 5,000 req/s
4,000 req/s 10 req/s
![Page 22: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/22.jpg)
Cold data compaction
10 req/s
10,000 req/s
![Page 23: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/23.jpg)
Cold Data Compaction•Disabled in 2.0
•Default in 2.1
•Configuration per Table:
ALTER TABLE mykeyspace.mytable WITH compaction = {'class': 'SizeTieredCompactionStrategy', 'cold_reads_to_omit': 0.05};
http://www.datastax.com/dev/blog/optimizations-around-cold-sstables
![Page 24: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/24.jpg)
Cassandra 2.1
![Page 25: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/25.jpg)
User defined typesCREATE TYPE address ( street text, city text, zip_code int, phones set<text>)!CREATE TABLE users ( id uuid PRIMARY KEY, name text, addresses map<text, address>)!SELECT id, name, addresses.city, addresses.phones FROM users;! id | name | addresses.city | addresses.phones--------------------+----------------+-------------------------- 63bf691f | jbellis | Austin | {'512-4567', '512-9999'}
![Page 26: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/26.jpg)
Collection indexingCREATE TABLE songs ( id uuid PRIMARY KEY, artist text, album text, title text, data blob, tags set<text>);!CREATE INDEX song_tags_idx ON songs(tags);!SELECT * FROM songs WHERE tags CONTAINS 'blues';! id | album | artist | tags | title----------+---------------+-------------------+-----------------------+------------------ 5027b27e | Country Blues | Lightnin' Hopkins | {'acoustic', 'blues'} | Worrying My Mind!!
![Page 27: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/27.jpg)
(UDT indexing?)
![Page 28: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/28.jpg)
Counters++•simpler implementation, no more edge cases
•possible to properly repair now
•significantly less garbage and internode traffic generated
•better performance for 99% of uses •RF>1, replicate_on_write=true
•topology changes not leading to data loss (#4071)
•commitlog now 100% safe to replay (#4417)
•Internal format overhaul still coming in 3.0 (#6506)
![Page 29: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/29.jpg)
What hasn’t changed•same API
•same average throughput
•same restrictions on mixing counter and non-counter columns
•same restrictions on mixing counter and non-counter updates
•same restrictions on counter deletes
•same retry limitations
![Page 30: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/30.jpg)
Writes (low contention)
![Page 31: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/31.jpg)
Writes (high contention)
![Page 32: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/32.jpg)
Data directories (2.0)/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-CompressionInfo.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Data.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Filter.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Index.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Statistics.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Summary.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-TOC.txt
![Page 33: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/33.jpg)
Data directories (2.1)/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-CompressionInfo.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Data.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Filter.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Index.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Statistics.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Summary.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-TOC.txt
![Page 34: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/34.jpg)
Inefficient bloom filters
+
= ?
![Page 35: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/35.jpg)
+
=
Inefficient bloom filters
![Page 36: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/36.jpg)
+
=
Inefficient bloom filters
![Page 37: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/37.jpg)
Inefficient bloom filters
![Page 38: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/38.jpg)
HyperLogLog applied
![Page 39: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/39.jpg)
HLL and compaction
![Page 40: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/40.jpg)
HLL and compaction
![Page 41: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/41.jpg)
More-efficient repair
![Page 42: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/42.jpg)
More-efficient repair
![Page 43: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/43.jpg)
More-efficient repair
![Page 44: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/44.jpg)
Implications for LCS (and STCS)
![Page 45: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/45.jpg)
Performance•Memtable memory use cut by 40% (85% by beta2)
• larger sstables, less compaction
•~50% better write performance
•Full results after beta2
![Page 46: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/46.jpg)
The new query cache
![Page 47: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/47.jpg)
The new row cacheCREATE TABLE notifications ( target_user text, notification_id timeuuid, source_id uuid, source_type text, activity text, PRIMARY KEY (target_user, notification_id))WITH CLUSTERING ORDER BY (notification_id DESC) AND caching = 'rows_only' AND rows_per_partition_to_cache = '3';!
![Page 48: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/48.jpg)
The new row cachetarget_user notification_id source_id source_type activity
nick e1bd2bcb- d972b679- photo jbellis liked
nick 321998c- d972b679- photo rbranson commented
nick ea1c5d35- 88a049d5- user mbulman created account
nick 5321998c- 64613f27- photo jbellis commented
nick 07581439- 076eab7e- user thobbs created account
rbranson 1c34467a- f04e309f- user jbellis created account
![Page 49: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/49.jpg)
The new row cachetarget_user notification_id source_id source_type activity
nick e1bd2bcb- d972b679- photo jbellis liked
nick 321998c- d972b679- photo rbranson commented
nick ea1c5d35- 88a049d5- user mbulman created account
nick 5321998c- 64613f27- photo jbellis commented
nick 07581439- 076eab7e- user thobbs created account
rbranson 1c34467a- f04e309f- user jbellis created account
![Page 50: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/50.jpg)
2.1 roadmap•Beta1 today •Beta2 beginning of April •RC end of March •Final release end of April?
![Page 51: Welcome to Apache Cassandra Day Silicon Valley 2014: Keynote by Chief Evangelist, Patrick McFadin](https://reader033.vdocument.in/reader033/viewer/2022061218/54b701114a7959943a8b45bf/html5/thumbnails/51.jpg)
Questions?