taming the cloud database with apache jclouds

1. 1 Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14

2. Introductions Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 2 Zack Shoylev irc: zacksh twitter: @zackshoylev freenode #jclouds Also email!

3. Introductions Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 3 Developer Relations Group [email protected]

4. Goals Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 4 cloud jclouds Create a database in the cloud Abstractions and how to contribute

5. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 5

6. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 6 Buzzword Networked and distributed computing Variety of services Compute (servers) Storage (files) Databases Email

7. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 7

8. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 8 Public cloud (external provider) Private cloud (internal deployment) Hybrid cloud Public + Private Rackspace example

9. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 9 Your System S D K Cloud A P I VM VM VM This can be an application server Or your home machine Or belong to one of your end users Or a cloud VM Or a smartphone jclouds

10. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 10 Advantages Metered pricing (pay as you go) Project scalability (unlimited) Safer (offsite/redundant) Economies of scale Expertise Support

11. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 11 Disadvantages Less hardware control Provider-controlled downtime Virtualization (efficiency) Provider lock-in Provider-specific apis/sdks/features Expensive to switch clouds or deploy locally jclouds minimizes this disadvantage

12. OpenStack Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 12 Cloud OS Open-sourced by Rackspace and NASA (2010) Free Supported AT&T, HP, IBM, Red Hat, Rackspace, Dell, Cisco, Intel, VMware,

13. Databases Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 13

14. OpenStack Trove / Cloud Databases Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 14 Database specific advantages Optimized by provider Settings Container virtualization Automated software updates Redundant data storage Backup Migration Support

15. jclouds Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 15 Cloud SDK Apache project Java (and Clojure) Easy Portable Cloud-agnostic Community Open source

16. jclouds Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 16 HTTP requests, responses, retries Authentication and re-authentication Pagination Polling Rate limits Retries Abstractions Logging Less code

17. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 17 Cant someone else do it?

18. jclouds Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 18

19. jclouds Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 19 Services Storage Compute VM Images Load Balancers DNS Databases Providers Openstack Rackspace HP Amazon Azure

20. jclouds Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 20 Showcase database code Best practices Gotchas and workarounds Abstractions

21. Requirements Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 21 Maven 3 Java 6+ jclouds Windows or Linux [etc..] (thanks Java!)

22. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 22 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup

23. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 23 Maven Dependencies Selective subset Versioning jclouds-labs

24. POM Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 24 https://github.com/jclouds/jclouds-examples/blob/master/rackspace/pom.xml org.apache.jclouds.providerrackspace-clouddatabases-us${jclouds.version}mysqlmysql-connector-java5.1.25

25. Logging Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 25 // This module is responsible for enabling logging Iterable modules = ImmutableSet. of(new SLF4JLoggingModule()); ComputeServiceContext context = ContextBuilder.newBuilder(provider) .credentials(username, apiKey) .modules(modules) // don't forget to add the modules to your context! .buildView(ComputeServiceContext.class); logback.xml target/test-data/jclouds-wire.log%d %-5p [%c] [%thread] %m%n

27. Initialize Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 27 private final TroveApi troveApi; troveApi = ContextBuilder .newBuilder("rackspace-clouddatabases- us") .credentials(username, apiKey) .buildApi(TroveApi.class);

28. Initialize Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 28 private final FlavorApi flavorApi; flavorApi = troveApi.getFlavorApiForZone(ZONE);

29. APIs Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 29 TroveApi FlavorApi InstanceApi DatabaseApi UserApi Utils

31. Create Instance Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 31 Flavor flavor = Iterables.getFirst(flavorApi.list(), null); Instance instance = instanceApi.create(flavorId, size, name); flavor.getId() Volume size in GB

32. Create Instance Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 32 Instance updatedInstance = awaitAvailable(instance, instanceApi);

33. Create Instance Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 33 Instance updatedInstance = awaitAvailable(instance, instanceApi); Polls status, waits until ready. Will not retry by itself!

34. Actual Code Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 34 InstanceApi instanceApi = api.getInstanceApiForZone(zone); for (int retries = 0; retries < 10; retries++) { Instance instance = null; try { instance = instanceApi.create(flavorId, size, name); } catch (Exception e) { Uninterruptibles.sleepUninterruptibly(15, TimeUnit.SECONDS); logger.error(Arrays.toString(e.getStackTrace())); continue; } Instance updatedInstance = awaitAvailable(instance, instanceApi); if (updatedInstance != null) { return updatedInstance; } instanceApi.delete(instance.getId()); InstancePredicates.awaitDeleted(instanceApi).apply(instance); } return null;

35. Actual Code Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 35 Utils. getWorkingInstance( String zone, String name, String flavorId, int size)

37. Create Database Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 37 troveApi = ContextBuilder.newBuilder(PROVIDER) .credentials(username, apiKey) .buildApi(TroveApi.class); instanceApi = troveApi.getInstanceApiForZone(ZONE); databaseApi = troveApi .getDatabaseApiForZoneAndInstance(ZONE,getInstance().getId()); getSomeApiForXandYandZ fairly common in jclouds

38. Create Database Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 38 boolean result = databaseApi.create(NAME);

40. Create User Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 40 userApi = troveApi .getUserApiForZoneAndInstance(ZONE, instance.getId()); boolean result = userApi.create(UNAME, PASSWORD, DBNAME); User name User password Database name

42. Test Connection Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 42 Load balancer needed! CreateLoadBalancer createLB = CreateLoadBalancer.builder() .name(NAME) .protocol("MYSQL") .port(3306) .algorithm(LoadBalancer.Algorithm.RANDOM) .nodes(addNodes) .virtualIPType(VirtualIP.Type.PUBLIC) .build();

43. Test Connection Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 43 loadBalancer = lbApi.create(createLB);

44. Test Connection Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 44 StringBuilder connString = new StringBuilder(); connString.append("jdbc:mysql://"); connString .append(getVirtualIPv4(getLb().getVirtualIPs())); connString.append("/"); connString.append(DBNAME); connString.append("?user="); connString.append(UNAME); connString.append("&password="); connString.append(PASSWORD);

45. Test Connection Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 45 Connection conn = DriverManager.getConnection(connString.toString());

46. Test Connection Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 46 Statement stmt = conn.createStatement(); ResultSet rs = stmt.executeQuery("SELECT 3+5"); rs.first(); System.out.format(" 3+5 is %s%n", rs.getInt(1));

47. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 47 jclouds examples https://github.com/jclouds/jclouds-examples More docs http://jclouds.incubator.apache.org/documentation/ http://javadocs.jclouds.cloudbees.net/ Maven alternatives? http://jclouds.incubator.apache.org/documentation/use rguide/installation-guide/ Contribute! https://wiki.apache.org/jclouds/How%20to%20Contrib ute

48. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 48 Compile javac -classpath "lib/*:src/main/java/:src/main/resources/" src/main/java/org/jclouds/examples/rackspace/*.java Run java -classpath "lib/*:src/main/java/:src/main/resources/" org.jclouds.examples.rackspace.clouddatabases.CreateInstance username apikey

49. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 49 TroveApi Backup Extension Settings Extension Abstraction layer

50. Architecture Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 50 Cloud Databases Trove jclouds-database Increasing level of abstraction Provider settings: Endpoint, etc. Implementation code Abstractions

51. Architecture Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 51 Cloud Databases Trove jclouds-database Increasing level of abstraction Provider settings: Endpoint, etc. Implementation code Abstractions Not implemented

52. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 52 TroveApi Backup Extension Settings Extension Abstraction layer Rackspace Microsoft Amazon CouchDB Salesforce

53. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 53 TroveApi Backup Extension Settings Extension Abstraction layer Rackspace Microsoft Amazon CouchDB Salesforce

54. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 54

55. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 55 Collaborate! http://jclouds.apache.org/ http://jclouds.apache.org/community/

56. Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 56 Thank you! http://developer.rackspace.com [email protected] Zack Shoylev Software Developer zacksh #jclouds @zackshoylev

taming the cloud database with apache jclouds

Technology

apache jclouds

test connection

database increasing

java project

abstraction

actual code

create database

create instance