Implementation
Zookeeper based implementation
It is becoming popular because it is the default registration center for Dubbo framework.
Idea: Use Zookeeper ephemeral node to work as service registry and watch mechanism for notifications.
Cons:
When there are too many directories or too many clients connecting to Zookeeper, the performance will have a natural degradation because Zookeeper enforces strong consistency.
ββββββββββββββββββ
βService consumerβ
β β
ββββββββββββββββββ
β β²
β β
Step 3. Create a node β β
"consumer1" under β β
/service/consumer directory. β β Step 4. ZooKeeper notifies the
β β client that a new node
And watch all nodes under β β "providerN" is added under the
/service/provider β β /service/provider registration
β β
β β
βΌ β
βββββββββββββββββββββββββββββββββββββββββ
β β
β Zookeeper β
β β
β /service/provider β
β /service/provider/provider1 β
Step 1. Create a root β ... β
service path β /service/provider/providerN β
β β
/service/provider β β
/service/consumer β /service/consumer β
β /service/consumer/consumer1 β
β ... β
β /service/consumer/consumerN β
β β
βββββββββββββββββββββββββββββββββββββββββ
β²
β
β
β
β Step 2. Create a node under
β service provider
β
β /service/provider/provider1
β
β
β
β
ββββββββββββββββββ
βService providerβ
β β
ββββββββββββββββββCharacteristics
Zookeeper keeps data in memory as a tree structure.
Limitations
CP model
Zookeeper is in essence a CP model, not an AP model.
Service discovery is an AP scenario, not a CP scenario. For example, when new nodes come online, it is fine if there is a delay in discovering them.
Example
Setup:
There are five nodes deployed in three clusters
ZK1/ZK2 in Cluster 1, ZK3/ZK4 in Cluster 2, ZK5 in Cluster 3.
Suddenly there is a network partition between Cluster1/2 and Cluster3.
Within each cluster, there is an application cluster. Application service A is calling service B.
If using CP model
ZK5 is not readable or writable. All clients connected to Cluster3 will fail.
Then Service B cannot be scaled up.
Then Service A cannot be rebooted.
Should use AP model
Scalability
All writes will come to master node, then it will be replicated to the majority of slave nodes synchronously (two phase commit is used for replication).
Possible solution: Split Zookeeper cluster according to the business functionalities. Different business units will not talk to each other.
Limitation: Due to the reorg and evolution, different business units will need to talk to each other.
Storage
Since Zookeeper is based on ZAB protocol. ZAB will maintain a commit log on each node which records every write request. On a regular basis the in-memory records will be dumped to disk. This means that the entire history of write change will be recorded.
From the perspective of registration center, it does not need to access history of node changes
It only needs to access information such as epoch number, weight of different nodes
Industrial usage
HBase: Use Zookeeper for leader election
Solr: Use Zookeeper for cluster management, configuration management, leader election
Dubbo: Service discovery http://dubbo.apache.org/en-us/docs/user/references/registry/zookeeper.html
Mycat: Cluster management, configuration management
Sharding-sphere: Cluster management, configuration management
Pros and Cons
Reliable
Need to create ephemeral nodes which are not as efficient
DNS based implementation
Benefits: Low intrusion to the business logic when compared with SDK-based solution.
Put service providers under a domain
Idea: Put all service providers under a domain.
Cons:
If an IP address goes offline, then the service provider could not easily remove the node because DNS has many layers of cache.
If scaling up, then newly scaled nodes will not receive enough traffic.
βββββββββββββ βββββββββββββ βββββββββββββββββ βββββββββββββ βββββββββββββββ βββββββββββββ
βuser browseβ β β βLocal DNS cacheβ β local DNS β βRegional DNS β β authority β
β a domain ββββββββΆβ JVM cache βββββββΆβ (Host file) ββββββββΆβ server ββββββββΆβ(with cache) βββββββΆβ DNS β
β β β β β β β β β β β β
βββββββββββββ βββββββββββββ βββββββββββββββββ βββββββββββββ βββββββββββββββ βββββββββββββDNS service points to a load balancer address
Idea: Consumers connect to the virtual ip address of a load balancer, not DNS servers.
Cons:
All traffic needs to go through an additional hop, causing performance degradation.
Usually for load balancers, if you want to add or remove nodes, it needs to be done manually.
When it comes to service governance, usually a more flexible load balancing algorithm will be needed.
βββββββββββββββββββββββββββ
β β
βββββββDiscoverββββββββββββββββ DNS β
β β β
β βββββββββββββββββββββββββββ
β
β
β
βΌ
βββββββββββββββββββββββββββ βββββββββββββββββββββββββββ βββββββββββββββββββββββββββ
β β β β Load β β
β Service consumer β ββInvokeβββββββΆβ Load balancer ββββββbalancing ββββΆβ Service Provider β
β β β β and revoke β β
βββββββββββββββββββββββββββ βββββββββββββββββββββββββββ βββββββββββββββββββββββββββAli DNS implementation
Independent DNS server
Pros:
Centralized DNS server. Easy for maintenance.
Cons:
High requirement on DNS server performance.
SPOF
Filter based on DNS server: Embed a DNS server in local server. All DNS queries will first be parsed by the local DNS.
Pros:
Avoid the SPOF
Cons:
Higher maintenance cost because DNS is embedded within each service.
Ali's implementation derive from filter based DNS server.
Service A query service B's IP address
DNS-F intercept service A's request, and see whether VIPServer has the data.
Otherwise, DNS-F will query the actual DNS server.
Reference in Chinese: https://developer.aliyun.com/article/598792
Proxy based
In-App Registration
Def: Each app embed a proxy (e.g. Alibaba Dubbo / Netflix karyon / Twitter Finagle)
Pros:
No single point of failure
High performant
Cons:
Language compatibility. Requires multiple language support.
Use cases:
In mid/large sized company where language stack is consistent and unified.
Side car
Def: Run two separate applications on the same machine. One for service registration, and the other for service discovery.
Consul implementation
Consul: Registry center's server end, will store registration information and provide registration and discovery service.
Registrator: An open-source third party service management project. It will listen to services' docker instances and provide registration and unregistration.
Consul template: Regularly pull information from registry center and update load balancer configuration (such as Nginx stream module). Then service consumers could get latest info by querying Nginx.
Message bus based registration
Idea:
When service providers come online, consumers could tolerate some latency in discovering these service providers. A strong consistency model such as what Zookeeper provides is not needed.
Could have multiple registry center connectecd by message bus. Registry center could have a full cached copy of service providers and multiple registry centers could sync the data via a message bus.
How to solve the above idea's update latency problem
Client retry when the service provider is no longer there.
βββββββββββββββββββββββββββββββ
β Service provider β
β β
βββββββββββββββββββββββββββββββ
β
β
Step 1. Create a
registration
message
β ββββββββββββββββββββββββββββββββββββ
β β Message bus β
βΌ β β
ββββββββββββββββββββββββββββββ Step 3. Publish β βββββββββββββββββββββββββββββ β
β β registration β βService: Checkout β β
β β ββββmessage to ββββΆ β βAddress: 192.168.1.9:9080 β β
β β message bus β βVersion: 2019113589 β β
β β β βββββββββββββββββββββββββββββ β
β β β β
β β β βββββββββββββββββββββββββββββ β
β β β βService: addToCart β β
β β β βAddress: 192.168.1.2:9080 β β
β β β βVersion: 2019103243 β β
β Registration center β β βββββββββββββββββββββββββββββ β
β β β β
β β Step 4. β βββββββββββββββββββββββββββββ β
β β Pull/receive push β β ...... β β
β ββββnotification fromββ β β β β
β β message bus β β β β
β β β βββββββββββββββββββββββββββββ β
β β β β
β β β βββββββββββββββββββββββββββββ β
β β β βService: Checkout β β
ββββββββββββββββββββββββββββββ β βAddress: 192.168.1.9:9080 β β
β² β β βVersion: 2019113590 β β
β β β βββββββββββββββββββββββββββββ β
β β ββββββββββββββββββββββββββββββββββββ
Step 2. Consumer β
subscribe to β
registration Step 5. Receive
center change notification for service
β βprovider list
β β
β βΌ
βββββββββββββββββββββββββββββββ
β Service consumer β
β β
βββββββββββββββββββββββββββββββLast updated
Was this helpful?