c* summit eu 2013: building a cql driver
DESCRIPTION
Speaker: Theo Hultberg, Chief Architect at Burt Video: http://www.youtube.com/watch?v=9vgKWPCztXw&list=PLqcm6qE9lgKLoYaakl3YwIWP4hmGsHm5e&index=13 I'm not a database driver expert, I'm just a Ruby developer who really likes Cassandra, but I've written a CQL driver. It wasn't hard, and it feels really good to now have an understanding of how my applications talk to Cassandra. To know what happens when I connect, prepare a statement, or how my data is encoded on its way to the database. When my CQL collections got big and I got weird results back, I could use my knowledge to quickly debug the issue. I'd like to give you an overview of the new native protocol from the driver perspective, maybe it can help you understand Cassandra the way it has helped me.TRANSCRIPT
#CASSANDRAEU CASSANDRASUMMITEU
Theo Hultberg | Chief Architect, Burt
Building a CQL driver
#CASSANDRAEU @ICONARA
Theo HultbergChief Architect at
#CASSANDRAEU @ICONARA
#CASSANDRAEU @ICONARA
I LIKE RUBYAND I LIKE CASSANDRA
#CASSANDRAEU @ICONARA
start_native_transport: true
#CASSANDRAEU @ICONARA
JUSTOPEN ASOCKET
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
0000 0001
1000 0001
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
02 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
82 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b C Q L _ V E R S I O N 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 3 . 0 . 0
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
01 00 2a 01 00 00 00 1600 01 00 0b 43 51 4c 5f56 45 52 53 49 4f 4e 0005 33 2e 30 2e 30
81 00 2a 02 00 00 00 00
#CASSANDRAEU @ICONARA
STARTUP
READY
#CASSANDRAEU @ICONARA
READY
{“CQL_VERSION”: “3.0.1”}
#CASSANDRAEU @ICONARA
OPTIONS
SUPPORTED
STARTUP
READY
#CASSANDRAEU @ICONARA
OPTIONS
{“CQL_VERSION”: [“3.0.0”, ...]}
STARTUP
READY
#CASSANDRAEU @ICONARA
OPTIONS
SUPPORTED
STARTUP
AUTHENTICATE
CREDENTIALS
READY
#CASSANDRAEU @ICONARA
QUERY
RESULT
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 73 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 75 73 65 72 5f 6e 616d 65 00 0d 00 05 65 6d 61 69 6c 00 0d 00 08 7061 73 73 77 6f 72 64 00 03 00 00 00 02 00 00 0004 70 68 69 6c 00 00 00 0d 70 68 69 6c 40 69 6e74 65 72 2e 6e 65 74 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 7375 65 00 00 00 0d 73 75 65 40 69 6e 74 65 72 2e6e 65 74 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 73 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 75 73 65 72 5f 6e 616d 65 00 0d 00 05 65 6d 61 69 6c 00 0d 00 08 7061 73 73 77 6f 72 64 00 03 00 00 00 02 00 00 0004 70 68 69 6c 00 00 00 0d 70 68 69 6c 40 69 6e74 65 72 2e 6e 65 74 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 7375 65 00 00 00 0d 73 75 65 40 69 6e 74 65 72 2e6e 65 74 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 73 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 75 73 65 72 5f 6e 616d 65 00 0d 00 05 65 6d 61 69 6c 00 0d 00 08 7061 73 73 77 6f 72 64 00 03 00 00 00 02 00 00 0004 70 68 69 6c 00 00 00 0d 70 68 69 6c 40 69 6e74 65 72 2e 6e 65 74 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 7375 65 00 00 00 0d 73 75 65 40 69 6e 74 65 72 2e6e 65 74 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 S E L E C T * F R O M U S E R S 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 75 73 65 72 5f 6e 616d 65 00 0d 00 05 65 6d 61 69 6c 00 0d 00 08 7061 73 73 77 6f 72 64 00 03 00 00 00 02 00 00 0004 70 68 69 6c 00 00 00 0d 70 68 69 6c 40 69 6e74 65 72 2e 6e 65 74 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 7375 65 00 00 00 0d 73 75 65 40 69 6e 74 65 72 2e6e 65 74 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 73 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 75 73 65 72 5f 6e 616d 65 00 0d 00 05 65 6d 61 69 6c 00 0d 00 08 7061 73 73 77 6f 72 64 00 03 00 00 00 02 00 00 0004 70 68 69 6c 00 00 00 0d 70 68 69 6c 40 69 6e74 65 72 2e 6e 65 74 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 7375 65 00 00 00 0d 73 75 65 40 69 6e 74 65 72 2e6e 65 74 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 73 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 75 73 65 72 5f 6e 616d 65 00 0d 00 05 65 6d 61 69 6c 00 0d 00 08 7061 73 73 77 6f 72 64 00 03 00 00 00 02 00 00 0004 70 68 69 6c 00 00 00 0d 70 68 69 6c 40 69 6e74 65 72 2e 6e 65 74 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 7375 65 00 00 00 0d 73 75 65 40 69 6e 74 65 72 2e6e 65 74 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 73 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 75 73 65 72 5f 6e 616d 65 00 0d 00 05 65 6d 61 69 6c 00 0d 00 08 7061 73 73 77 6f 72 64 00 03 00 00 00 02 00 00 0004 70 68 69 6c 00 00 00 0d 70 68 69 6c 40 69 6e74 65 72 2e 6e 65 74 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 7375 65 00 00 00 0d 73 75 65 40 69 6e 74 65 72 2e6e 65 74 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 73 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 u s e r _ n a m e 00 0d 00 05 e m a i l 00 0d 00 08 p a s s w o r d 00 03 00 00 00 02 00 00 0004 70 68 69 6c 00 00 00 0d 70 68 69 6c 40 69 6e74 65 72 2e 6e 65 74 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 7375 65 00 00 00 0d 73 75 65 40 69 6e 74 65 72 2e6e 65 74 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 73 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 75 73 65 72 5f 6e 616d 65 00 0d 00 05 65 6d 61 69 6c 00 0d 00 08 7061 73 73 77 6f 72 64 00 03 00 00 00 02 00 00 0004 70 68 69 6c 00 00 00 0d 70 68 69 6c 40 69 6e74 65 72 2e 6e 65 74 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 7375 65 00 00 00 0d 73 75 65 40 69 6e 74 65 72 2e6e 65 74 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 73 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 75 73 65 72 5f 6e 616d 65 00 0d 00 05 65 6d 61 69 6c 00 0d 00 08 7061 73 73 77 6f 72 64 00 03 00 00 00 02 00 00 0004 p h i l 00 00 00 0e p h i l @ i n t e r . n e t 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 s u e 00 00 00 0d s u e @ i n t e r . n e t 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
01 00 00 07 00 00 00 19 00 00 00 13 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 73 0004
81 00 00 08 00 00 00 bf 00 00 00 02 00 00 00 0100 00 00 03 00 0a 6d 79 6b 65 79 73 70 61 63 6500 05 75 73 65 72 73 00 09 75 73 65 72 5f 6e 616d 65 00 0d 00 05 65 6d 61 69 6c 00 0d 00 08 7061 73 73 77 6f 72 64 00 03 00 00 00 02 00 00 0004 70 68 69 6c 00 00 00 0d 70 68 69 6c 40 69 6e74 65 72 2e 6e 65 74 00 00 00 20 5e b6 3b bb e01e ee d0 93 cb 22 bb 8f 5a cd c3 00 00 00 03 7375 65 00 00 00 0d 73 75 65 40 69 6e 74 65 72 2e6e 65 74 00 00 00 20 32 7b 6f 07 43 58 11 23 9bc4 7e 15 44 35 32 73
#CASSANDRAEU @ICONARA
QUERY
ERROR
#CASSANDRAEU @ICONARA
PREPARE
RESULT
EXECUTE
RESULT
EXECUTE
RESULT
EXECUTE
#CASSANDRAEU @ICONARA
01 00 03 09 00 00 00 2a 00 00 00 26 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 20 5748 45 52 45 20 75 73 65 72 5f 6e 61 6d 65 20 3d20 3f
81 00 03 08 00 00 00 3e 00 00 00 04 00 10 ca 487f 1e 7a 82 d2 3c 4e 8a f3 35 51 71 a5 2f 00 0000 01 00 00 00 01 00 0a 6d 79 6b 65 79 73 70 6163 65 00 05 75 73 65 72 73 00 09 75 73 65 72 5f6e 61 6d 65 00 0d
#CASSANDRAEU @ICONARA
01 00 03 09 00 00 00 2a 00 00 00 26 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 20 5748 45 52 45 20 75 73 65 72 5f 6e 61 6d 65 20 3d20 3f
81 00 03 08 00 00 00 3e 00 00 00 04 00 10 ca 487f 1e 7a 82 d2 3c 4e 8a f3 35 51 71 a5 2f 00 0000 01 00 00 00 01 00 0a 6d 79 6b 65 79 73 70 6163 65 00 05 75 73 65 72 73 00 09 75 73 65 72 5f6e 61 6d 65 00 0d
#CASSANDRAEU @ICONARA
01 00 03 09 00 00 00 2a 00 00 00 26 S E L E C T * F R O M u s e r W H E R E u s e r _ n a m e = ?
81 00 03 08 00 00 00 3e 00 00 00 04 00 10 ca 487f 1e 7a 82 d2 3c 4e 8a f3 35 51 71 a5 2f 00 0000 01 00 00 00 01 00 0a 6d 79 6b 65 79 73 70 6163 65 00 05 75 73 65 72 73 00 09 75 73 65 72 5f6e 61 6d 65 00 0d
#CASSANDRAEU @ICONARA
01 00 03 09 00 00 00 2a 00 00 00 26 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 20 5748 45 52 45 20 75 73 65 72 5f 6e 61 6d 65 20 3d20 3f
81 00 03 08 00 00 00 3e 00 00 00 04 00 10 ca 487f 1e 7a 82 d2 3c 4e 8a f3 35 51 71 a5 2f 00 0000 01 00 00 00 01 00 0a 6d 79 6b 65 79 73 70 6163 65 00 05 75 73 65 72 73 00 09 75 73 65 72 5f6e 61 6d 65 00 0d
#CASSANDRAEU @ICONARA
01 00 03 09 00 00 00 2a 00 00 00 26 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 20 5748 45 52 45 20 75 73 65 72 5f 6e 61 6d 65 20 3d20 3f
81 00 03 08 00 00 00 3e 00 00 00 04 00 10 ca 487f 1e 7a 82 d2 3c 4e 8a f3 35 51 71 a5 2f 00 0000 01 00 00 00 01 00 0a 6d 79 6b 65 79 73 70 6163 65 00 05 75 73 65 72 73 00 09 75 73 65 72 5f6e 61 6d 65 00 0d
#CASSANDRAEU @ICONARA
01 00 03 09 00 00 00 2a 00 00 00 26 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 20 5748 45 52 45 20 75 73 65 72 5f 6e 61 6d 65 20 3d20 3f
81 00 03 08 00 00 00 3e 00 00 00 04 00 10 ca 487f 1e 7a 82 d2 3c 4e 8a f3 35 51 71 a5 2f 00 0000 01 00 00 00 01 00 0a 6d 79 6b 65 79 73 70 6163 65 00 05 75 73 65 72 73 00 09 75 73 65 72 5f6e 61 6d 65 00 0d
#CASSANDRAEU @ICONARA
01 00 03 09 00 00 00 2a 00 00 00 26 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 20 5748 45 52 45 20 75 73 65 72 5f 6e 61 6d 65 20 3d20 3f
81 00 03 08 00 00 00 3e 00 00 00 04 00 10 ca 487f 1e 7a 82 d2 3c 4e 8a f3 35 51 71 a5 2f 00 0000 01 00 00 00 01 00 0a 6d 79 6b 65 79 73 70 6163 65 00 05 75 73 65 72 73 00 09 u s e r _ n a m e 00 0d
#CASSANDRAEU @ICONARA
01 00 03 09 00 00 00 2a 00 00 00 26 53 45 4c 4543 54 20 2a 20 46 52 4f 4d 20 75 73 65 72 20 5748 45 52 45 20 75 73 65 72 5f 6e 61 6d 65 20 3d20 3f
81 00 03 08 00 00 00 3e 00 00 00 04 00 10 ca 487f 1e 7a 82 d2 3c 4e 8a f3 35 51 71 a5 2f 00 0000 01 00 00 00 01 00 0a 6d 79 6b 65 79 73 70 6163 65 00 05 75 73 65 72 73 00 09 75 73 65 72 5f6e 61 6d 65 00 0d
#CASSANDRAEU @ICONARA
STREAM 0b
STREAM 0c
STREAM 0d
STREAM 0e
STREAM 07
STREAM 08
STREAM 09
STREAM 0a
STREAM 0f
STREAM 10
STREAM 06
STREAM 04
#CASSANDRAEU @ICONARA
STREAM ff EVENT
#CASSANDRAEU @ICONARA
THE HARD PARTIS NOT THE PROTOCOL
#CASSANDRAEU @ICONARA
THE DISTRIBUTED SYSTEM INCLUDES THE CLIENT
#CASSANDRAEU @ICONARA
×
×
#CASSANDRAEU @ICONARA
I GET BUGREPORTS
#CASSANDRAEU @ICONARA
Map: a [short] n indicating the size of the map, followed by n entries. Each entry is composed of two [short bytes] representing the key and the value of the entry map.
#CASSANDRAEU @ICONARA
RUBY PERFORMANCE(WAIT FOR IT)
#CASSANDRAEU @ICONARA
GO WRITEYOUR OWNDRIVERS
#CASSANDRAEU @ICONARA
github.com/iconara/cql-rb
#CASSANDRAEU CASSANDRASUMMITEU
Theo Hultberg | Chief Architect, Burt
Building a CQL driver