Tag Archives: cluster

Couchbase Cluster on Amazon using CLI

Couchbase on Amazon Marketplace showed how to setup a single Couchbase node using EC2 Console. But typically you provision these nodes en masse, and more commonly create a cluster of them. Couchbase clusters are homogenous, scales horizontally and thus ensure that database does not become a bottleneck for your high performing application.

This blog will show how to create, scale, and rebalance a Couchbase cluster using AWS Command Line Interface (CLI).

Install AWS CLI

Install the AWS CLI provide complete details, but here is what worked on my machine:

Configure the CLI:

Enter your access key id and secret access key. These can be obtained as explained in Getting Your Access Key Id and Secret Access Key.

Create AWS Security Group

If you provisioned a server earlier using Amazon 1-click then a security group by the name Couchbase Server Community Edition-4-0-0-AutogenByAWSMP- is created for you. This security group has all the ports authorized required for creating a Couchbase cluster and can be used for creating the instance.

Alternatively, you can create a new security group and explicitly authorize ports.

Create a security group:

Authorize ports in the security group:

Create an AWS Key Pair

Read more about creating key pair. Create a key pair:

Note the key name as that will be used later.

Create Couchbase Nodes on Amazon

Create two instances using the newly created security group as:

Note, the number of instances are specified using --count 2.

AMI ID can be obtained using EC2 console: https://us-west-1.console.aws.amazon.com/ec2/v2/home?region=us-west-1#Images:visibility=public-images;search=couchbase;sort=desc:name.

Or create two instances using the pre-created security group as:

This will show the output as:

Status of the instances can be checked as:

And shows the output as:

Here the status is shown as initializing. It takes a few minutes for the instances to be provisioned. Instances that have passed all the checks can be verified as:

At first, it shows the result as:

But once all the instances pass the check, then the results look like:

Here the status is shown as passed.

Configure Couchbase Nodes

Each Couchbase node needs to be provisioned with the following details:

  • Memory
  • Services (index, data and query)
  • Auth credentials (username: Administrator, password: password)
  • Loads travel-sample bucket

This can be done using the script:

This is available at: https://github.com/arun-gupta/couchbase-amazon/blob/master/configure-instance.sh.

It can be invoked as:

And shows the output as:

This is invoking Couchbase REST API to configure each Couchbase node.

Now that each Couchbase node is configured, lets access them. Find public IP address of the instances:

It shows the output as:

Pick one of the IP addresses and access it at <public-ip-1>:8091 to see the output:

Each Couchbase node is configured with username as Administrator and password as password. Entering the credentials shows Couchbase Web Console:

Click on Server Nodes to see that only a single node is in the cluster:

Create and Rebalance Couchbase Cluster

All Couchbase server nodes are created equal. This allows the Couchbase cluster to truly scale horizontally to meet your growing application demands. Independently running Couchbase nodes can be added to a cluster by invoking the server-add CLI command.

This is typically a two step process. The first step is to add one or more nodes. The second step is to rebalance the cluster where data on the existing nodes is rebalanced across the updated cluster.

In our case, a Couchbase node is running on each AMI. Lets pick IP address of any one Couchbase node and add IP address of the other node. This can be done using the script:

This script is available at https://github.com/arun-gupta/couchbase-amazon/blob/master/create-cluster.sh and can be invoked as:

And shows the output as:

Couchbase Web Console is updated to show:

Finally, rebalance the cluster using the script:

This shows the output as:

Couchbase Web Console is now updated:


Once the cluster is up and running, try out Hello Couchbase Example.

Terminate Couchbase Nodes

Finally, killing the cluster is fairly straight forward:

This blog showed how to spin up, scale and rebalance a Couchbase cluster using AWS CLI. All scripts are available at https://github.com/arun-gupta/couchbase-amazon.

Further references …

  • Couchbase Server Developer Portal
  • Hello Couchbase Example
  • Questions on StackOverflow, Forums or Slack Channel
  • Follow us @couchbasedev
  • Couchbase 4.5 Beta

Source: http://blog.couchbase.com/2016/may/couchbase-cluster-amazon-using-cli

Couchbase Cluster using Docker Compose

Couchbase 4.0 provides lots of features that allows you to develop with agility and operate at any scale. Some of the features that allow you to operate at any scale are:

  • Elastic Scalability
  • Consistent High Performance
  • Always-On Availability
  • Multi-Data Center Deployment
  • Simple and Powerful Administration
  • Enterprise-grade Security

Learn more about these enterprise features at couchbase.com/operate-at-any-scale.

A complete overview is available in Couchbase Server 4.0 datasheet.

This blog will explain how you can easily setup a 3-node Couchbase Cluster using Docker Compose.

Docker Couchbase Cluster

The source code and latest instructions are available at github.com/arun-gupta/docker-images/tree/master/couchbase-cluster.

Create Couchbase Nodes

Couchbase cluster can be easily created using the following Docker Compose file:

This file has service definition for three Couchbase nodes. Admin ports are exposed for only one node as other nodes will talk to each other use Docker-internally assigned IP addresses.

  1. Create three directories ~couchbase/node1, ~couchbase/node2, ~couchbase/node3 – one for each node.
  2. Start three Couchbase nodes as using the docker-compose.yml shown earlier:
    This command is given on a Docker Machine.
  3. Check status of the nodes:
    Docker Compose can also show the status:
  4. Check logs of the nodes:

Configure Couchbase Cluster

Lets configure these nodes to be part of a cluster now.

  1. Find IP address of the Docker Machine:
  2. Access Couchbase Admin Console at http://<DOCKER_MACHINE_IP:8091. This is in our case. It will show the output as:

    Docker Couchbase Cluster Setup

    Click on “Setup”.

  3. Each container is given an internal IP address by Docker, and each of these IPs is visible to all other containers running on the same host. We need to use these internal IP address when adding a new node to the cluster.Find IP address of the first container:

    Use this IP address to change the Hostname field:

    Docker Couchbase Cluster Node 1

  4. Click on “Next”. Adjust the RAM if necessary. Read more about Couchbase Cluster Settings.
  5. Pick a sample bucket that you’d like to get installed, and click on Next.
  6. Change Per Node RAM Quota from 400 to 100. This is required as we’ll add other nodes later.Docker Couchbase Cluster Per Node RAM Quota
  7. Click on Next, accept T&C, and click on Next.
  8. Enter a password that you can remember as we’ll need this later to add more nodes.

Default view of the cluster looks like as shown:

Docker Couchbase Cluster Default View

Add More Couchbase Nodes

Now, lets add the other two nodes that were created earlier by Docker Compose.

  1. Click on “Server Nodes” to see the default view as:

    Docker Couchbase Cluster Server Nodes Default View

  2. Find IP address of one of the remaining nodes:

  3. Click on “Add Server”, specify the IP address:

    Docker Couchbase Cluster Add Server Node1
    and click on “Add Server”.

  4. Repeat the previous two steps with the server name couchbasecluster_couchbase2_1.

Couchbase Cluster Rebalance

A cluster needs to be rebalanced to ensured that the data is well distributed amongst the newly added or removed nodes. Read more about Couchbase Cluster Rebalance.

Clicking on “Pending Rebalance” tab shows the nodes that have been added to the cluster but are not rebalanced yet:

Docker Couchbase Cluster Pending Rebalance

Click on “Rebalance” and this will automatically rebalance the cluster:

Docker Couchbase Cluster Rebalanced

You just deployed a Couchbase cluster using Docker Compose, enjoy!

Some more references:

  • Couchbase 4 Administration Guide
  • Couchbase 4 Server Architecture
  • Couchbase 4 Cluster Setup