Docker Service and Swarm Mode to Create Couchbase Cluster

containers, couchbase, ,

Docker 1.12 introduced Services. A replicated, distributed and load balanced service can be easily created using docker service create command. A “desired state” of the application, such as run 3 containers of Couchbase, is provided and the self-healing Docker engine ensures that that many containers are running in the cluster. If a container goes down, another container is started. If a node goes down, containers on that node are started on a different node.

This blog will show how to setup a Couchbase cluster using Docker Services.

Many thanks to @marcosnils, another fellow Docker Captain, to help me debug the networking!

Couchbase Cluster

A cluster of Couchbase Servers is typically deployed on commodity servers. Couchbase Server has a peer-to-peer topology where all the nodes are equal and communicate to each other on demand. There is no concept of master nodes, slave nodes, config nodes, name nodes, head nodes, etc, and all the software loaded on each node is identical. It allows the nodes to be added or removed without considering their “type”. This model works particularly well with cloud infrastructure in general.

A typical Couchbase cluster creation process looks like:
  • Start Couchbase: Start n Couchbase servers
  • Create cluster: Pick any server, and add all other servers to it to create the cluster
  • Rebalance cluster: Rebalance the cluster so that data is distributed across the cluster
In order to automate using Docker Services, the cluster creation is split into a “master” and “worker” service.
docker-service-couchbase-cluster
The master service has only one replica. This provides a single reference point to start the cluster creation. This service also exposes port 8091. It allows the Couchbase Web Console to be accessible from outside the cluster.
The worker service uses the exact same image as master service. This keeps the cluster homogenous which allows to scale the cluster easily.
Let’s get started!

Setup Swarm Mode on Ubuntu

  1. Launch an Ubuntu instance on Amazon. This blog used mx4.large size for the AMI.
  2. Install Docker:
  3. Docker Swarm mode is an optional feature and need to be explicitly enabled. Initialize Swarm mode:

Create Couchbase “master” Service

  1. Create an overlay network:
    This is required so that multiple Couchbase Docker containers in the cluster can talk to each other.
  2. Create a “master” service:
    This image is created using the Dockerfile here. This Dockerfile uses a configuration script to configure the base Couchbase Docker image. First, it uses Couchbase REST API to setup memory quota, setup index, data and query services, security credentials, and loads a sample data bucket. Then, it invokes the appropriate Couchbase CLI commands to add the Couchbase node to the cluster or add the node and rebalance the cluster. This is based upon three environment variables:
    • TYPE: Defines whether the joining pod is worker or master
    • COUCHBASE_MASTER: Name of the master service
    • AUTO_REBALANCE: Defines whether the cluster needs to be rebalanced
    For this first configuration file, the TYPE environment variable is set to MASTER and so no additional configuration is done on the Couchbase image.

    This service also uses the previously created overlay network named couchbase. It exposes the port 8091 that makes the Couchbase Web Console accessible outside the cluster. This service contains only one replica of the container.

  3. Check status of the Docker service:

    It shows that the service is running. The “desired” and “expected” number of replicas are 1, and thus are matching.

  4. Check the tasks in the service:

    This shows that the container is running.

  5. Access Couchbase Web Console using public IP address and it should look like:docker-service-couchbase-login
     The image used in the configuration file is configured with the Administrator username and password password. Enter the credentials to see the console:
    docker-service-couchbase-web-console
  6. Click on Server Nodes to see how many Couchbase nodes are part of the cluster. As expected, it shows only one node:docker-service-couchbase-one-active-server

Create Couchbase “worker” Service

  1. Create “worker” service:

    This RC also creates a single replica of Couchbase using the same arungupta/couchbase:swarm image. The key differences here are:

    • TYPE environment variable is set to WORKER. This adds a worker Couchbase node to be added to the cluster.
    • COUCHBASE_MASTER environment variable is passed the name of the master service,  couchbase-master.couchbase in our case. This uses the service discovery mechanism built into Docker for the worker and the master to communicate.
  2. Check service:
  3. Checking the Couchbase Web Console shows the updated output:
    docker-service-couchbase-one-pending-server
     It shows that one server is pending to be rebalanced.During the worker service creation, AUTO_REBALANCE environment variable could have been set to true or false to enable rebalance. This ensures that the node is only added to the cluster but the cluster itself is not rebalanced. Rebalancing the cluster requires to re-distribute the data across multiple nodes of the cluster. The recommended way is to add multiple nodes, and then manually rebalance the cluster using the Web Console.

Add Couchbase Nodes by Scaling Docker Service

  1. Scale the service: 

  2. Check the service:
    This shows that 2 replicas of worker are running.
  3. Check the Couchbase Web Console:
    docker-service-couchbase-two-pending-serversAs expected, two servers are now added in the cluster and pending rebalance.
  4. Optionally, you can rebalance the cluster by clicking on the Rebalance button. and it will show like:docker-service-couchbase-rebalancingAfter the rebalancing is complete, the Couchbase Web Console is updated to as as shown:
    docker-service-couchbase-rebalanced
  5. See all the running containers using docker ps:

In addition to creating a cluster, Couchbase Server supports a range of high availability and disaster recovery (HA/DR) strategies. Most HA/DR strategies rely on a multi-pronged approach of maximizing availability, increasing redundancy within and across data centers, and performing regular backups.

Now that your Couchbase cluster is ready, you can run your first sample application.

Learn more about Couchbase and Containers:

  • Couchbase on Containers
  • Follow us on @couchbasedev or @couchbase
  • Ask questions on Couchbase Forums

Source: http://blog.couchbase.com/2016/september/docker-service-swarm-mode-couchbase-cluster

Be Sociable, Share!
  • Tweet
Related posts:
  1. Couchbase Cluster on Docker Swarm using Docker Compose and Docker Machine
  2. Couchbase XDCR using Docker Swarm, Machine and Compose
  3. Couchbase Cluster using Docker Compose
  4. Scaling and Rebalancing a Couchbase Cluster using CLI
  5. Kubernetes Cluster on Amazon and Expose Couchbase Service

4 thoughts on “Docker Service and Swarm Mode to Create Couchbase Cluster”

  1. Every peoples connected the internet and searching many websites daily and also types data, how to delete browsing history information’s even passwords all are save into the browsing history so it will keep save in your privacy then this website is useful for you.

  2. From this one might conclude that Perception is a few styles of a trick that helps the USA to require these spectral pictures and turns them into a world we are able to associate and act with. assignment writing service – assignmentdoer.com. This clever device looks to be a creation of our intellect that allows the USA to act with one another in what seems to be a 3-dimensional reality.

  3. It shows that one server is pending to be rebalanced.During the worker service creation, AUTO_REBALANCE environment variable could have been set to true or false to enable rebalance. This ensures that the node is only added to the cluster but the cluster itself is not rebalanced. Rebalancing the cluster requires to re-distribute the data across multiple nodes of the cluster. The recommended way is to add multiple nodes, and then manually rebalance the cluster using the Web Console.

  4. This is also a very good post which I really enjoyed reading. It is not everyday that I have the possibility to see something like this.

Leave a Reply

Your email address will not be published. Required fields are marked *