zookeeper big sonata
Post on 15-Jan-2015
194 Views
Preview:
DESCRIPTION
TRANSCRIPT
Introduction to Zookeeper
Anh Le @BigSonata
What is a Distributed System?
A distributed system consists of multiple computers that communicate through a
computer network and interact with each other to achieve a common goal.
-Wikipedia
Coordination in a Distributed System?
Coordination: An act that multiple nodes must perform together.Examples: Leader Election Managing group membership Managing metadata Synchronization (Semaphore, Mutex...)
Coordination in a Distributed System?
To coordinate, processes can Exchange messages through network Read/Write using shared storage Use distributed locks
Problems for exchanging messages Message delays Processor speed Clock drift
Use case for Master-Work Applications
Problems Master crashes Worker crashes Communication failures
Use case for Master-Work Applications
Problems for Master Crashes Use a backup master Recover the latest state ? Backup master may suspect the primary master has crashed ? !
> Split Brain scenario
Use case for Master-Work Applications
Problems for Worker Crashes Master must detect worker crashes Recover assigned tasks
Problems for Communication Failures Execute a same task only once
Introduction to ZooKeeper
An open source, performant coordination service for distributed applications
Was a sub project of Hadoop but is now a Apache top-level project
Exposes common services in simple interface Leader Election Naming Configuration management Locks & Synchronization Group Service
→ Don't have to write them from scratch
ZooKeeper Use cases
Distributed Cluster Management Node join/leave Node statuses in real time
Distributed synchronization Locks Barriers Queues
ZooKeeper Use cases
Apache Hbase use ZooKeeper to Elect a cluster master Keep track of available servers Keep cluster metadata
Apache Kafka use Zookeeper to Detect crashes Implement topic discovery Maintain state for topics
ZooKeeper Guarantees
Sequential Consistency: Updates are applied in order Atomicity: Updates either succeed or fail Single System Image: A client sees the same view of the service
regardless of the ZK server it connects to. Reliability: Updates persists once applied, till overwritten by some
clients. Timeliness: The clients’ view of the system is guaranteed to be up-
to-date within a certain time bound. (Eventual Consistency)
ZooKeeper Services
All machines store a copy of the data (in memory) A leader is elected on service startup Clients only connect to a single server & maintains a TCP
connection. Client can read from any server, writes go through the leader &
needs majority consensus.
ZooKeeper Data Model
ZooKeeper has a hierarchal name space. Each node is called as a ZNode. Every ZNode has data (given as byte[]) ZNode paths:
canonical, absolute, slash-separated no relative references. names can have Unicode characters
ZNode
Maintain a stat structure with version numbers for data changes, ACL changes and timestamps.
Version numbers increases with changes
Data is read and written in its entirety
ZNode types
Persistent Nodes exists till explicitly deletedEphemeral Nodes exists as long as the session is active can’t have childrenSequence Nodes (Unique Naming) append a monotonically increasing counter to the end of path applies to both persistent & ephemeral nodes
ZNode watches
Clients can set watches on znodes: NodeChildrenChanged NodeCreated NodeDataChanged NodeDeleted
Changes to a znode trigger the watch and ZooKeeper sends the client a notification.
Watches are one time triggers. Watches are always ordered. Client sees watched event before new ZNode data.
ZNode APIs
String create(path, data, acl, flags)void delete(path, expectedVersion)Stat setData(path, data, expectedVersion)(data, Stat) getData(path, watch)Stat exists(path, watch)String[] getChildren(path, watch)→ Each API has its own asynchronous version also
ZooKeeper Recipes
Recipe: Leader Election
/master
Recipe: Leader Election
Continuous watching on znodes requires reset of watches after every events / triggers
Too many watches on a single znode creates the “herd effect” - causing bursts of traffic and limiting scalability
Recipe: Leader Election (Improved)
1.All participants create an ephemeral-sequential node on the same election path.
2.The node with the smallest sequence number is the leader.
3.Each “follower” node listens to the node with the next lower seq. number
4.Upon leader removal go to election-path and find a new leader, or become the leader if it has the lowest sequence number.5.Upon session expiration check the election state
and go to election if needed
Zookeeper Programming
https://github.com/anhldbk/Zookeeper-Demo
Zookeeper Programming
Zookeeper Programming
Zookeeper Programming
Difficult to use Zookeeper APIs Connection Issues:
Initial connection: Requires a handshake before executing any operations (create(), delete()...)
Session expiration: Clients are expected to watch for this state and close and re-create the ZooKeeper instance.
Zookeeper Programming
Difficult to use Zookeeper APIs Recoverable Errors:
When creating a sequential ZNode on the server, there is the possibility that the server will successfully create the ZNode but crash prior to returning the node name to the client.
There are several recoverable exceptions thrown by the ZooKeeper client. Users are expected to catch these exceptions and retry the operation.
Zookeeper Programming
Difficult to use Zookeeper APIs Recipes:
The standard ZooKeeper "recipes" (locks, leaders, etc.) are only minimally described and subtly difficult to write correctly..
Zookeeper Programming with Curator
l Curator- The Netflix Zookeeper library
Zookeeper Programming with Curator
Zookeeper Programming with Curator
Zookeeper for Our Systems
top related