la cassandra day 2015 - testing cassandra
Post on 15-Jul-2015
389 Views
Preview:
TRANSCRIPT
copy2013 DataStax Confidential Do not distribute without consent
chbatey
Christopher BateyTechnical Evangelist for Apache Cassandra
LA 2015 Testing Cassandra applications
chbatey
Who am IbullBased in LondonbullTechnical Evangelist for Apache CassandrabullWork on Stubbed CassandrabullHelp out Apache Cassandra users
bull Previous Cassandra backed apps at BSkyB
chbatey
Agendabull Cassandra failure scenariosbull Developer toolsbull Stubbed Cassandra
chbatey
Production
Application
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
replication factor 3
consistency ONE
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Read timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull wasDataReceived
chbatey
Write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Write timeout
chbatey
Retrying writesbull Cassandra does not roll back
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Who am IbullBased in LondonbullTechnical Evangelist for Apache CassandrabullWork on Stubbed CassandrabullHelp out Apache Cassandra users
bull Previous Cassandra backed apps at BSkyB
chbatey
Agendabull Cassandra failure scenariosbull Developer toolsbull Stubbed Cassandra
chbatey
Production
Application
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
replication factor 3
consistency ONE
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Read timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull wasDataReceived
chbatey
Write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Write timeout
chbatey
Retrying writesbull Cassandra does not roll back
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Agendabull Cassandra failure scenariosbull Developer toolsbull Stubbed Cassandra
chbatey
Production
Application
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
replication factor 3
consistency ONE
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Read timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull wasDataReceived
chbatey
Write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Write timeout
chbatey
Retrying writesbull Cassandra does not roll back
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Production
Application
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
replication factor 3
consistency ONE
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Read timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull wasDataReceived
chbatey
Write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Write timeout
chbatey
Retrying writesbull Cassandra does not roll back
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
replication factor 3
consistency ONE
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Read timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull wasDataReceived
chbatey
Write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Write timeout
chbatey
Retrying writesbull Cassandra does not roll back
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Read timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull wasDataReceived
chbatey
Write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Write timeout
chbatey
Retrying writesbull Cassandra does not roll back
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Read timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Read timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull wasDataReceived
chbatey
Write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Write timeout
chbatey
Retrying writesbull Cassandra does not roll back
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Read timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull wasDataReceived
chbatey
Write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Write timeout
chbatey
Retrying writesbull Cassandra does not roll back
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Write timeout
chbatey
Retrying writesbull Cassandra does not roll back
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Retrying writesbull Cassandra does not roll back
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Write timeoutbull Received acknowledgementsbull Required acknowledgementsbull Consistency levelbull CAS and Batches are more complicated- WriteType SIMPLE BATCH UNLOGGED_BATCH
BATCH_LOG CAS
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Batchesbull BATCH_LOG- Timed out waiting for batch log replicas- Retry if you likebull BATCH- Written to batch log but timed out waiting for actual replica- Will eventually be committed (serial read)bull UNLOGGED_BATCH- Unknown - retry if you like
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Idempotent writesbull All writes are idempotent with the following exceptions- Counters- lists
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Unavailablebull Alive Replicasbull Required Replicasbull Consistency
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittlebull Integration tests against an embedded Cassandra bull cassandra-unitbull Integration tests against an external Cassandra running on developer CI
machinesbull CCMbull Regular installbull Dockerbull Vagrantbull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
How do this if it was a HTTP servicebull Release it - great bookbull Wiremock - mocking HTTP
servicesbull Saboteur - adding network
latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Two sides of Scassandrabull Java-Acts as a unitintegration testing library-90 of effort on this sidebullStandalone-Runs as a separate process
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Java ClientbullEmbeds Stubbed Cassandra in a Java library- Start stop the server- Prime the server to return rows andor errors- Verify exactly the queries your application has executed
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Starting Stopping
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Activity Client
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
public ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Priming Client
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
public void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt row = ImmutableMapof( first_name Chris last_name Batey age 29) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(column(age PrimitiveTypeINT)) withRows(row) build()) when ListltPersongt names = underTestretrievePeople() then assertEquals(1 namessize()) assertEquals(ldquoChris Batey namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt configuredRetries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQueryPattern(insert into person) withVariableTypes(VARCHAR INT list(TIMESTAMP)) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29 CollectionsemptyList()))
Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Summary of featuresbull Start as many instances as you like in the same JVMbull Prime all primitive types + collectionsbull Prime prepared statementsbull Verify number of connections your application makesbull Test slow queries
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Future featuresbull Loading of schema for priming prepared statementsbull Pretending to be multiple nodes
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
How do I get itbull It is on maven central bullOr go to wwwscassandraorgbullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt060ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
I want to helpbull Server httpsgithubcomscassandrascassandra-
serverbull Client httpsgithubcomscassandrascassandra-java-
clientbull Tests httpsgithubcomscassandrascassandra-it-java-
driver-2
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
Summarybull Testing is good - I want more tools to help mebull Existing tools for Cassandra are great at happy path
scenariosbull Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
chbatey
The end - Questions
wwwscassandraorghttpchristopher-bateyblogspotcoukPing me on twitter chbatey
chbatey
How does this workbull Server implemented in Scalabull Binary port for your Cassandra application to connect tobull Admin port for the REST APIbull Thin Java wrapper to make it easy to be used in JUnit
tests
top related