My Randezvous with Tech

Loadbalancing & Service Discovery Setup details

February 21, 2017 by Ameya Advankar

---

Problem Statement:

---

Setup a Highly Available UI Portal with Load balancing and Service Discovery mechanism

---

Detailed Solution:

---

HAProxy

HAProxy can be setup on any server. Once it is installed, we need to configure the Haproxy.cfg file before starting the server. The following steps can be used to setup base HAProxy on a Ubuntu 16.04 server

apt-get install -y haproxy
echo "ENABLED=1" > /etc/default/haproxy
service haproxy start

Configure the authentication and logging information in the HAProxy.cfg file as required and then restart or reload the haproxy service. Now you will have a basic HAProxy server up and running

Consul

We will be using Consul for Service Discovery and levereging its quorum setup capability to our advantage. We will have 1 Consul agent setup on the server where we have our HAProxy setup. For setting up consul on a server, we can use a Docker image

docker run -d -e SERVICE_IGNORE \
			-p 8300:8300 \
			-p 8301:8301 \
			-p 8301:8301/udp \
			-p 8302:8302 \
			-p 8302:8302/udp \
			-p 8400:8400 \
			-p 8500:8500 \
			-p 8600:53/udp \
			-h consul --name consul progrium/consul \
			-server -advertise $HOST_IP_1 -bootstrap

Next we will have to setup consul on every server where our services are located. These consul instances will join together and form a quorum.

docker run -d -e SERVICE_IGNORE \
			-p 8300:8300 \
			-p 8301:8301 \
			-p 8301:8301/udp \
			-p 8302:8302 \
			-p 8302:8302/udp \
			-p 8400:8400 \
			-p 8500:8500 \
			-p 8600:53/udp \
			-h node1 --name node1 progrium/consul \
			-server -advertise $HOST_IP_2 -join $HOST_IP_1

Registrator

We can use Registrator in conjunction with consul to implement service discovery for our service as it provides for code-independent and a plugable approach. All Registrator requires is that the our service should be running inside a docker container. Registrator will register any docker container with each exposed port which is started to the consul instance. Since we have consul installed on each server where the service is to be started, we can bind the registrator to the local consul instance.

docker run -d \
	--name=registrator \
	--net=host \
	--volume=/var/run/docker.sock:/tmp/docker.sock \
	gliderlabs/registrator:latest \
	consul://localhost:8500

With this, we have completed the Service Discovery setup. Any service which starts up will get registered to the consul quorum through Registrator

consul-template

The consul-template daemon provides a way to perform file updates and command executions for consul data changes. We can use this feature to reload HAProxy config for changes in service availability in consul. consul-template will be setup on the server where we HAProxy. consul-template can be configured for our use case via a .json file as follows -

• source: This is the .ctmpl Template file. It has code related to iterating and generating meaningful data from consul
• destination: This is the file which needs to populated. In our case, it will be the HAProxy.cfg file.
• command: This is the command which has to be run at the end of file update. In our case, it will be the reloading haproxy service using either service haproxy reload or haproxy -f /etc/haproxy/haproxy.cfg -sf $(pidof haproxy)

Using the above config we can then start the consul-template daemon

consul-template -config /tmp/haproxy.json \
			-consul-addr 127.0.0.1:8500

This concludes our Loadbalancing setup. As and when there is a change in Consul data, consul-template will write the available services into HAProxy.cfg and reload the haproxy service. This way we will always have the loadbalancer routing setup configured for all the discovered service instances at any given time