Monthly Archives: May 2016

Docker for Java Developers with IntelliJ IDEA Webinar Replay

Video recording from Docker for Java Developers webinar is now available:

Here are answers to the questions asked:

  • What do you think of Kubernetes compared to Docker and Docker Swarm? Kubernetes or Docker Swarm? Which way to go?

    It depends ๐Ÿ˜‰

    Docker Swarm has a much simpler learning curve as opposed to Kubernetes. Kubernetes is very opinionated but can be very powerful once you understand how it works.

    Some of the points worth considering are:

    • Easy to setup/start
    • Need to learn new concepts or aligned with Docker
    • How can it be aligned with your existing workflow
    • Single or multiple master
    • Support for multiple clouds
    • Cluster across multiple clouds
    • Dynamic scaling up/down
    • Rescheduling containers on node/container failure
    • Pluggable scheduler strategies (if you want to go fancy)
    • Service discovery
    • Persistent volumes

    If this question was asked a few months ago, then Kubernetes would have been the obvious choice. But Docker Swarm is really cranking out releases and features and has caught up well. Check out Docker Swarm Exceeds Kubernetes Performance at Scale. Hackernews has some interesting discussion on this topic as well. Also read Evaluating Container Platforms at Scale.

    In addition, you should also look at Mesos+Marathon or DC/OS.

    Here are some blogs to get started:

    • Couchbase cluster on Docker Swarm using Compose and Machine
    • Kubernetes Cluster on Google Cloud and Expose Couchbase Service
    • Kubernetes Cluster on Amazon and Expose Couchbase Service
    • Couchbase on DC/OS and Amazon
  • What is the best practice to develop against a docker WildFly container if you do not want to bundle the WAR file after each change but use an exploded archive instead in order to just repackage static resources for example?
    Docker allows mapping directories on your filesystem as volumes in the container. This can be used to develop your application against a WildFly container without the need to be bundle the WAR with each change. This is explained in more detail at Deploy to WildFly Docker from Eclipse.
  • Does IntelliJ Idea support native docker on Mac OS/Windows?
    Andrey: Not yet but it’s on the roadmap.
  • What are pros and cons of providing environment variable vs link like consul:consul?

    Docker Links are legacy and so environment variables and Docker Networking are the recommended way to build any new applications. Read more details in another answer below.

  • How do we attach a debugger to a Java process running in a container?

    Read this great blog entry by Daniel Bryant on Debugging Java Applications running in Docker.

  • What is the relation between Docker and Vagrant? Can both work together? Is docker a substitute of Vagrant?

    Docker and Vagrant solve different problems.

    Vagrant allows you to provision Virtual Machines independent of the host machine. This is done by using a hypervisor, such as VirtualBox. The definition for the VM is stored in a Vagrantfile, and defines how to setup the VM and what scripts to provision the environment. These VMs can be created on Linux, Windows or Mac.

    Docker allows you to create containers. It is a native Linux technology based upon lxc and coming to Windows Server 2016 this Summer. Docker for Mac provides development environment for Mac.

    If you need to provision a Docker Engine, then you can create a Vagrantfile that provisions Ubuntu or CentOS using Virtual Box. Then run apt get or yum install scripts to install Docker Engine. If a new version of Docker Engine is available, then you are responsible for updating the Vagrant image as well, or run the script again.

    Another option is to use Docker Machine and use VirtualBox driver to create a machine that will have Docker Engine installed. The docker-machine script has commands like upgrade that simply upgrades the Docker Engine.

    Some more interesting discussions are at StackOverflow and Quora.

    To make this whole equation further interesting, Vagrant can be backed by Docker containers instead of VMs!

  • Docker tool window is missing in my IntelliJ IDEA 15. Plugin is installed.

    Andrey: The separate tool window was added in 2016.1. Before that Docker was a part of the Application Servers tool window.

  • Is it possible to change index.jsp and see the change in the browser without building the image again? In the spirit of update resources action in IntelliJ IDEA?

    Andrey: Sounds like a great idea. Please submit it as a feature request to tracker: https://youtrack.jetbrains.com/issues/IDEA.

  • Is there a performance increase for using xhyve/Hyper-V over VirtualBox?

    Docker for Mac runs Docker Engine in Alpine Linux distribution on top of xhyve Virtual Machine for Mac OS X or on Hyper-V on Windows. xhyve is built on Hypervisor.framework in OS X 10.10+. Some of the primary advantages of using a native technology as opposed to a Type 2 hypervisor is native look-and-feel, better performance, and better volume management.

    Read more details in (3-months old blog) xhyve vs VirtualBox driver benchmarks. This area is fast evolving and keep an eye on blog.docker.com for more details.

  • In Docker Compose, how do I update only one container, does it lose the “link” do I need to redeploy everything?

    Docker Links are now legacy. It’s highly recommended to migrate your application to use Docker Networking. The main difference between Links and Networking are:

    1. Connect containers to each other across different physical or virtual hosts
    2. Containers using Networking can be easily stopped, started and restarted without disrupting the connections to other containers
    3. You donโ€™t need to create a container before you can link to it. With Networking containers be created in any order and discover each other using their container names
  • How can one expose a host port to a container running on the host?

    Ports in a container are exposed on the host using -p with docker run. Ports on the host can be accessed using <host-ip>:<port> directly.

  • Official Java Docker image (OpenJDK-based) is not TCK certified. What do you suggest in this case?
    OpenJDK is the Reference Implementation of Java SE and so is TCK compliant. Official Java Docker image is based on OpenJDK.Azul Systems provide commercial support for OpenJDK. Another option is to use OpenJDK images available from Azul systems. These are fully tested and compatibility verified builds of the latest versions of the OpenJDK 8, 7, and 6 platforms.

    Oracle publishes Dockerfile for JDK but don’t publish the images. Note that this definition uses oraclelinux:latest as the base image and you may want to use ubuntu, centos or alpine. In this case, you can create your own Dockerfile.

    Running Java on Docker? You’re breaking the law provides a good summary of why Oracle JDK-based image should not be pushed to Docker Hub.

  • How do you have a container with WildFly use a container with the db on another server?

    Docker Links are legacy and only allow containers to talk to each on the same host. The recommended way for two containers to communicate is using Docker Networks.By default, Docker Swarm cluster creates an overlay network that allows multiple containers to talk across hosts. Multi-container and multi-host applications using WildFly and Couchbase shows how to connect WildFly with Couchbase where each container is running on two separate hosts on a Docker Swarm cluster.Here is the Compose file for reference:

Here are some more references:

  • Learn all about IntelliJ and Docker at jetbrains.com/idea/docker/
  • Self-paced Docker for Java Developers workshop
  • Docker for Java developers Introductory slides
  • Script for Docker and IntelliJ demo shown in the video
  • Docker Docs
  • Couchbase on Containers

Source: http://blog.couchbase.com/2016/may/docker-java-intellij-idea-webinar-replay

Couchbase XDCR using Docker Swarm, Machine and Compose

Cross Datacenter Replication (XDCR) in Couchbase provides an easy way to replicate data from one cluster to another. The clusters are typically set in geographically diverse data centers. This enables for disaster recovery or to bring data closer to users for faster data access.

This blog will show:

  • Setup two data centers using Docker Swarm
  • Run Couchbase containers on each node of Docker Swarm
  • Setup a Couchbase cluster on each Docker Swarm cluster
  • Configure one-way XDCR between two Couchbase clusters

For the purpose of this blog, the two data centers will be setup on a local machine using Docker Machine.

swarm-xdcr-couchbase-overview

Complete code used in this blog is available at: github.com/arun-gupta/couchbase-xdcr-docker.

Create Consul Discovery Service

Each node in Docker Swarm needs to be registered with a discovery service. This blog will use Consul for that purpose. And even Consul will be running on a Docker Machine.

Typically, you’ll run a cluster of Consul but for simplicity a single instance is started in our case.

Create a Docker Machine and start Consul using this script:

Create Docker Swarm cluster

Docker Swarm allows multiple Docker hosts to be viewed as a single unit. This allows your multi-container applications to easily run on multiple hosts. Docker Swarm serves the same Remote API as served by a single host. This allows your existing tools to target a single host or a cluster of hosts.

Both the Docker Swarm clusters will be registered with a single discovery service. This is achieved by using the following value for --swarm-discovery:

Create a Docker Swarm cluster using Docker Machine using this script:

The script needs to be invoked as:

This will create two Docker Swarm clusters with one “master” and two “worker” as shown below:

Consul is running on Docker Machine with IP address 192.168.99.101. And so Consul UI is accessible at http://192.168.99.101:8500:

swarm-xdcr-couchbase

It shows two Docker Swarm clusters that have been registered.

Exact list of nodes for each cluster can also be seen. Nodes in clusterA are shown:

swarm-xdcr-consul-clusterA

Nodes in clusterB are shown:

swarm-xdcr-consul-clusterB

Run Couchbase containers

Run Couchbase container on each node of Docker Swarm cluster using this Compose file.

Configure Docker CLI for the first cluster and run 3 containers:

Check the running containers:

Configure Docker CLI for the second cluster and run 3 containers:

Check the running containers:

Create/Rebalance Couchbase cluster

Scaling and Rebalancing Couchbase Cluster using CLI explains how to create a cluster of Couchbase nodes and rebalance an existing cluster using Couchbase CLI.

Create Couchbase cluster on each Swarm cluster using this script.

The script needs to be invoked as:

And now rebalance this cluster using this script:

This script is invoked as:

Couchbase Web Console for any node in the cluster will show the output:

swarm-xdcr-couchbase-clusterA

Invoke this script to create the second Couchbase cluster as:

Rebalance this cluster as:

Couchbase Web Console for any node in the second cluster will show the output:

swarm-xdcr-couchbase-clusterB

Setup XDCR

Cross datacenter replication can be setup to be uni-directional, bi-directional or multi-directional. Uni-directional allows data to replicated from source cluster to destination cluster, bi-directional allows replication both ways, multi-directional allows to configure in any direction.

We’ll create a simple uni-directional replication using this script:

This script is invoked as:

A bi-directional replication can be easily created by executing the commands again but reversing the source and destination cluster.

Couchbase Web Console for the source cluster will show:

swarm-xdcr-couchbase-clusterA-xdcr

Couchbase Web Console for the destination cluster will show: swarm-xdcr-couchbase-clusterB-xdcr

Enjoy!

This blog shows how you can simplify your complex deployments using Docker Machine, Docker Swarm, and Docker Compose.

Source: http://blog.couchbase.com/2016/may/couchbase-xdcr-docker-swarm-machine-compose

Scaling and Rebalancing a Couchbase Cluster using CLI

Couchbase provides high availability and disaster recovery in several ways:

  • Homogenous cluster
  • Replication
    • Intra-cluster replication
    • Cross-data center replication (XDCR)
  • Backup and restore
  • Rack zone awareness

This blog will show how to create a Couchbase cluster using Couchbase Command Line Interface (CLI). In addition, these steps can also be performed using the Couchbase REST API and the Couchbase Web Console.

couchbase-cluster-amazon

Couchbase nodes in a cluster are homogeneous. Applications can write to any node and read from any node. There is no write-to-master and read-from-slave architecture that inherently gives you scalability problems. This allows the Couchbase cluster to truly scale horizontally to meet your growing application demands.

Creating a Couchbase cluster involves the following steps:

  • Provision Couchbase: Provision 1 or more Couchbase nodes, say on Amazon Web Services
  • Initialize master: Initialize any node to be the “master” of the cluster by calling cluster-init CLI command
  • Create cluster: For all other nodes, create a cluster by invoking the server-add CLI command
  • Rebalance cluster: Finally, rebalance the cluster by calling the rebalance CLI command

Provision Couchbase

Provision a Couchbase node on Amazon:

Security group name is explained in Couchbase Cluster on Amazon using CLI.

Let’s call this node as “master”.

Initialize Couchbase “master”

Configure the “master” instance and initialize the cluster:

Create another instance, lets call it “worker”. Note, this is not a master/slave architecture. Couchbase cluster is homogenous where any node in the cluster can be “master”.

Create Couchbase Cluster

Add this newly created “worker” instance to the cluster:

Typically, you’ll create and add multiple nodes to the cluster before rebalancing.

Rebalance Couchbase Cluster

Rebalance the cluster:

Now, you can create as many instances and easy include them in the cluster.

Adding a single node and rebalancing the cluster can be easily done as a single step:

Now, your cluster is accessible at http://<master-ip>:8091 or http://<worker-ip>:8091.

Further references …

  • Couchbase Server Developer Portal
  • Hello Couchbase Example
  • Couchbase CLI
  • Questions on StackOverflow, Forums or Slack Channel
  • Follow us @couchbasedev
  • Couchbase 4.5 Beta

Enjoy!

Source: http://blog.couchbase.com/2016/scaling-rebalancing-couchbase-cluster-cli

Couchbase Cluster on Amazon using CLI

Couchbase on Amazon Marketplace showed how to setup a single Couchbase node using EC2 Console. But typically you provision these nodes en masse, and more commonly create a cluster of them. Couchbase clusters are homogenous, scales horizontally and thus ensure that database does not become a bottleneck for your high performing application.

This blog will show how to create, scale, and rebalance a Couchbase cluster using AWS Command Line Interface (CLI).

Install AWS CLI

Install the AWS CLI provide complete details, but here is what worked on my machine:

Configure the CLI:

Enter your access key id and secret access key. These can be obtained as explained in Getting Your Access Key Id and Secret Access Key.

Create AWS Security Group

If you provisioned a server earlier using Amazon 1-click then a security group by the name Couchbase Server Community Edition-4-0-0-AutogenByAWSMP- is created for you. This security group has all the ports authorized required for creating a Couchbase cluster and can be used for creating the instance.

Alternatively, you can create a new security group and explicitly authorize ports.

Create a security group:

Authorize ports in the security group:

Create an AWS Key Pair

Read more about creating key pair. Create a key pair:

Note the key name as that will be used later.

Create Couchbase Nodes on Amazon

Create two instances using the newly created security group as:

Note, the number of instances are specified using --count 2.

AMI ID can be obtained using EC2 console: https://us-west-1.console.aws.amazon.com/ec2/v2/home?region=us-west-1#Images:visibility=public-images;search=couchbase;sort=desc:name.

Or create two instances using the pre-created security group as:

This will show the output as:

Status of the instances can be checked as:

And shows the output as:

Here the status is shown as initializing. It takes a few minutes for the instances to be provisioned. Instances that have passed all the checks can be verified as:

At first, it shows the result as:

But once all the instances pass the check, then the results look like:

Here the status is shown as passed.

Configure Couchbase Nodes

Each Couchbase node needs to be provisioned with the following details:

  • Memory
  • Services (index, data and query)
  • Auth credentials (username: Administrator, password: password)
  • Loads travel-sample bucket

This can be done using the script:

This is available at: https://github.com/arun-gupta/couchbase-amazon/blob/master/configure-instance.sh.

It can be invoked as:

And shows the output as:

This is invoking Couchbase REST API to configure each Couchbase node.

Now that each Couchbase node is configured, lets access them. Find public IP address of the instances:

It shows the output as:

Pick one of the IP addresses and access it at <public-ip-1>:8091 to see the output:
couchbase-aws-cli-1

Each Couchbase node is configured with username as Administrator and password as password. Entering the credentials shows Couchbase Web Console:
couchbase-aws-cli-2

Click on Server Nodes to see that only a single node is in the cluster:
couchbase-aws-cli-3

Create and Rebalance Couchbase Cluster

All Couchbase server nodes are created equal. This allows the Couchbase cluster to truly scale horizontally to meet your growing application demands. Independently running Couchbase nodes can be added to a cluster by invoking the server-add CLI command.

This is typically a two step process. The first step is to add one or more nodes. The second step is to rebalance the cluster where data on the existing nodes is rebalanced across the updated cluster.

In our case, a Couchbase node is running on each AMI. Lets pick IP address of any one Couchbase node and add IP address of the other node. This can be done using the script:

This script is available at https://github.com/arun-gupta/couchbase-amazon/blob/master/create-cluster.sh and can be invoked as:

And shows the output as:

Couchbase Web Console is updated to show:
couchbase-aws-cli-4

Finally, rebalance the cluster using the script:

This shows the output as:

Couchbase Web Console is now updated:

couchbase-aws-cli-5

Once the cluster is up and running, try out Hello Couchbase Example.

Terminate Couchbase Nodes

Finally, killing the cluster is fairly straight forward:

This blog showed how to spin up, scale and rebalance a Couchbase cluster using AWS CLI. All scripts are available at https://github.com/arun-gupta/couchbase-amazon.

Further references …

  • Couchbase Server Developer Portal
  • Hello Couchbase Example
  • Questions on StackOverflow, Forums or Slack Channel
  • Follow us @couchbasedev
  • Couchbase 4.5 Beta

Source: http://blog.couchbase.com/2016/may/couchbase-cluster-amazon-using-cli

Couchbase on Amazon Marketplace

Couchbase Server can be easily downloaded and installed on your local machine. However a common way to run it is on Amazon. This blog will explain how you can run Couchbase on Amazon.

Couchbase Server can be easily launched on Amazon Web Services using Couchbase on AWS Marketplace. Click on Continue and review the settings in 1-Click Launch as shown below:

couchbase-amazon-oneclick-1

This is using Couchbase Server 4.0 Community Edition and an m3.large instance.

couchbase-amazon-oneclick-2

A new security group with all the relevant ports exposed will be automatically created. This group’s name is Couchbase Server Community Edition-4-0-0-AutogenByAWSMP-. This group is created once and can be repurposed if you launch another instance.

You may have to create a new key pair as explained here.

Click on “Accept Software Terms and Launch with 1-click”. The following confirmation window will be shown:

couchbase-amazon-oneclick-3

Wait for a few minutes for the image to be provisioned. AWS EC2 Console should show the status as  “2/2 checks passed” as shown:

couchbase-amazon-oneclick-4

Get the public DNS address and access Couchbase at http://<public-dns-address>:8091. In our case, this will be http://ec2-54-153-6-241.us-west-1.compute.amazonaws.com:8091/index.html. Note, that this instance is now shutdown and so is not available ๐Ÿ˜‰

Couchbase on Amazon Usage Instructions provide more details about how to use Couchbase on Amazon.

<public-ip-address>:8091 shows a login screen. A default username is Administrator and the password is your instance id:

couchbase-amazon-oneclick-5

And clicking on Sign In shows the Couchbase Web Console:

couchbase-amazon-oneclick-6

Further references …

  • Couchbase Server Developer Portal
  • Couchbase Downloads
  • Couchbase Server Developer Portal
  • Questions on StackOverflow, Forums or Slack Channel
  • Follow us @couchbasedev
  • Couchbase 4.5 Beta

Enjoy!

Run #Couchbase on #Amazon Market Place using 1-click Click To Tweet

Source: https://blog.couchbase.com/2016/may/couchbase-amazon-marketplace

Couchbase Cluster on Docker Swarm using Docker Compose and Docker Machine

This blog post will explain how to create and scale a Couchbase Cluster using full armor of Docker – Docker Machine, Docker Swarm and Docker Compose.

Here is what we’ll do:

  • Create a 3-node Docker Swarm Cluster using Docker Machine
  • Run a Couchbase instance on two nodes
  • Create a cluster
  • Rebalance the cluster
  • Scale and rebalance the cluster again

couchbase-docker-swarm-0

Docker Swarm Cluster using Consul

Create a three-node Docker Swarm cluster using Docker Machine:

Provision a Swarm cluster with Docker Machine provide more details about why and what’s done in this script. Here is a summary:

  • Create a Docker Machine and run Consul for service discovery
  • Create three Docker Machines – one for Master and two for Worker nodes.Each machine is configured to be part of a Swarm cluster using --swarm. It also uses the Consul service discovery specified using --swarm-discovery.

Couchbase Nodes on Docker Swarm

Create two instances of Couchbase using Docker Compose:

arungupta/couchbase image is used here. This image is defined at Couchbase Docker Image. It uses the Official Couchbase Docker Image add configures it as explained:

  1. Setups memory for Index and Data
  2. Configures the Couchbase server with Index, Data, and Query service
  3. Sets up username and password credentials
  4. Loads the travel-sample bucket

Compose file uses host network. This is equivalent to using --net=host on docker run CLI. It allows the container to use the host networking stack. It also limits only a single Couchbase container to run on a single Docker Machine. So this means that our Couchbase cluster can scale based upon the number of Docker Machines – 3 in our case.

The exact command to use this Compose file is:

There are three nodes in the Docker Swarm cluster. The default scheduler strategy is spread and so the containers will be spread on different hosts.

This is evident by docker ps:

Note, one Couchbase server is running on swarm-node-01 and another on swarm-node-02. Each server is configured with an administrator username Administrator and password password.

Find out IP address of the Docker Machine:

If you have jq installed then IP address can be conveniently found as:

Couchbase Cluster on Docker Swarm

All Couchbase server nodes are created equal. This allows the Couchbase cluster to truly scale horizontally to meet your growing application demands. Independently running Couchbase nodes can be added to a cluster by invoking the server-add CLI command. This is typically a two step process. The first step is adding one or more nodes. The second step then rebalances the cluster where the data on the existing nodes is rebalanced across the updated cluster.

In our case, a single Couchbase container is running on each Docker Machine. Lets pick IP address of any one Couchbase node, and add IP address of the other node:

Couchbase Web Console for both the nodes will show a similar output:

couchbase-docker-swarm-1

couchbase-docker-swarm-2

This shows that the two nodes now form a cluster, and needs to be rebalanced.

Rebalance Couchbase Cluster

Now, lets rebalance the cluster:

Couchbase Web Console will be updated to show that rebalance is happening:

couchbase-docker-swarm-3

And finally you’ll see a rebalanced cluster:

couchbase-docker-swarm-4

Scale and Rebalance Couchbase Cluster

Scale the Couchbase cluster:

Check that the container is running on a different Docker Machine:

As mentioned earlier, scaling a Couchbase cluster is a two-step process. This is so because typically you’ll add multiple servers and then rebalance the cluster. However, in cases where you only need to add a single Couchbase node and then rebalance, the rebalance command can be used achieve that.

In our case, this is done as shown:

The rebalanced cluster now looks like:

couchbase-docker-swarm-5

This blog showed how you can easily create and scale a Couchbase Cluster using Docker Swarm, Machine and Compose.

Enjoy!

Further reading …

  • Couchbase Server Developer Portal
  • Couchbase Server Concepts
  • Couchbase Cluster Setup
  • Questions on StackOverflow, Forums or Slack Channel
  • Follow us @couchbasedev
  • Couchbase 4.5 Beta

Source: http://blog.couchbase.com/2016/may/couchbase-cluster-docker-swarm-compose-machine

Create and scale your #Couchbase cluster using #Docker Machine, Swarm and Compose Click To Tweet

Couchbase Weekly, May 11, 2016

Learn what’s latest in the Couchbase Community.
Couchbase Developer Community

Let us know if we missed anything at @couchbasedev, @couchbase or #Couchbase.

Couchbase Weekly

Highlights

Events

  • May 11: Apps World North America, San Francisco (Speaking)
  • May 11 – 12: GeeCON, Krakow (Speaking)
  • May 16 – 19: OSCON, Austin
  • May 18: Couchbase Day โ€” Cromwell

Couchbase Weekly

News

Media Mentions

  • DZone: Document Database Granularity Level for the Relational Minded in 2 Minutes
  • Business Wire: Aviatrix Names Steven Mih as Chief Executive Officer
  • arstechnica: Power tools: Sorting through the crowded specialized database toolbox
  • Venture Beat: Redis launches modules to add extensibility to the open source database
  • Business Wire: Black Duck and North Bridge Release Results from 10th Annual Future of Open Source Survey
  • Business Wire: Centrify Honored as Bronze Stevieยฎ Award Winner in 2016 American Business Awardsโ„ 
  • SDTimes: Why developers are sitting pretty for IoT
  • Fierce Enterprise Communications: Aviatrix beefs up security and support for OpenVPN in latest Cloud Native Networking release
  • DZone: Polyglot Persistence in NoSQL Space

Blogs

  • Monitoring Couchbase Server with SignalFx – Cihan Biyikoglu
  • Couchbase will be at Apps World North America 2016 โ€” Nic Raboy
  • Couchbase with Windows and .NET – Part 2 โ€” Matthew Groves
  • High Performance NOSQL Query Consistency with Couchbase 4.5 โ€” Todd Greenstein
  • Faster Indexing and Query: Introducing Memory Optimized Global Secondary Indexes – PART II โ€” Cihan Biyikoglu
  • Faster Indexing & Queries: Introducing Global Secondary Indexes (GSI) – PART I โ€” Cihan Biyikoglu
  • Making most of your Arrays.. with Covering Array Indexes and more.. โ€” Prasad Varakur
  • Docker container using Apache Mesos and Marathon โ€” Arun Gupta
  • Introducing New Advanced Features to the Couchbase Shell – CBQ โ€” Nic Raboy
  • Couchbase 4.5 Application Contest – Chance to win $500 Amazon gift card! โ€” Arun Gupta
  • Couchbase with Windows and .NET – Part 1 – Setup โ€” Matthew Groves
  • Introducing Couchbase Server 4.5 Beta โ€” Don Pinto
  • Couchbase 4.5 Beta is Now Available – Faster, Simpler, More Secure โ€” Arun Gupta
  • A CRUD REST API with Couchbase, RxJava and Ratpack โ€” Laurent Doguin
  • Ratpack, Couchbase and RxJava โ€” Laurent Doguin
  • Getting Started with Ratpack and Couchbase โ€” Laurent Doguin
  • Setup Docker on Windows Server 2016 โ€” Arun Gupta
  • Windows Server 2016 using VirtualBox for Docker Containers โ€” Arun Gupta
  • Couchbase is at Mobile Web DevCon 2016 โ€” Nic Raboy
  • Hello, Couchbase, it’s me, Matthew Groves! โ€” Matthew Groves
  • Annoucing PHP SDK 2.1 beta โ€” Sergey Avseyev

Whitepapers

  • Couchbase Server: An Architectural Overview

Couchbase Weekly 3

Upcoming Events

  • May 11: Apps World North America, San Francisco (Speaking)
  • May 11 – 12: GeeCON, Krakow (Speaking)
  • May 16 – 19: OSCON, Austin
  • May 23 – 26: Informatica World 2016, San Francisco
  • May 24 – 26: XLDB, Menlo Park
  • May 18: Couchbase Day โ€” Cromwell
  • May 24: Couchbase Day โ€” Houston 
  • May 24: Couchbase Day โ€” Las Vegas
  • May 26: Couchbase Day โ€” Austin

Webinars

  • April 27: Future of Open Source Survey 2016

twitter-logo

#Couchbase Server 4.5 Beta introduces new security features: x.509 certificates, role-based access control, and moreโ€ฆ Click To Tweet Get professional advice from the #Couchbase services team on everything from #JSON modeling to using #SQLfor #NoSQLโ€ฆ Click To Tweet Canโ€™t decide what db to use for your #mobile apps? Hereโ€™s 9 reasons to choose #Couchbase: ow.ly/4mXdgc Click To Tweet Enterprises are adopting modern databases like #Couchbase โ€œciting grow flexibility as the top benefitโ€ Learn more:โ€ฆ Click To Tweet Get lightning fast data access: discover #Couchbase XDCR for high availability & better disaster recovery:โ€ฆ Click To Tweet

Source: http://blog.couchbase.com/2016/may/couchbase-weekly-may-11-2016

Couchbase on Mesos using DC/OS and Amazon

Couchbase Docker container can easily run on a variety of orchestration platforms:

Docker container using Apache Mesos and Marathon explained how to setup Mesos, Marathon, and run a simple Docker image. The setup was quite involving and a bit flaky.

It required to download and Install Mesos Master and Slave, ZooKeeper, Docker Engine, and Marathon. In some cases, the correct repo need to be added first. These components need to talk to each and so must be configured accordingly. Even if you get past that setup, how do you monitor the entire infrastructure as one entity?

Meet DC/OS – Datacenter Operating System. Its a distributed operating system using Apache Mesos as its kernel.

dcos-kernel

DC/OS can installed in a variety of ways:

  • Local using Vagrant
  • Cloud using Amazon/CloudFormation, Microsoft Azure and Packet
  • On-prem using CentOS or CoreOS

This blog will show how to setup a DC/OS cluster using CloudFormation templates on Amazon and run a Couchbase Docker container.

Launch DC/OS cluster

Launch DC/OS cluster:
dcos-couchbase-1

Take the defaults:
dcos-couchbase-2

Give the template a name, select a previously create KeyPair, change the number of slaves:
dcos-couchbase-3

Take the defaults:
dcos-couchbase-4

Verify the configuration:
dcos-couchbase-5

Click on “I acknowledge that …” and on Create to start the template creation.

CloudFormation Stack Status  page comes up:
dcos-couchbase-6

Make sure to choose the appropriate region.

After ~10-15 mins, the status changes:
dcos-couchbase-7

Wait for the status to change from CREATE_IN_PROGRESS to CREATE_COMPLETE.

Download and Configure DC/OS CLI

DC/OS CLI can be used to manage your cluster nodes, install DC/OS packages, inspect the cluster state, and administer the DC/OS service subcommands.

Install DC/OS CLI on your local machine.

On your CloudFormation Stack Status  page, select the created stack, Outputs tab, and copy the address of Mesos master.
dcos-couchbase-8

Configure the DC/OS CLI to use this cluster:

Authenticate:

Enter the URL in the browser, proceed to the unsafe URL:
dcos-couchbase-9

Copy the token to your clipboard:
dcos-couchbase-10

Paste the authentication token in the terminal window:

Mesos and Marathon UI

Mesos UI is available using the address of the Mesos master:
dcos-couchbase-11

Click on Services to see Marathon service already installed:
dcos-couchbase-12

Click on marathon to see the list of tasks:
dcos-couchbase-13

As expected for a freshly created cluster, no tasks have been assigned yet.

Click on Nodes to see the nodes:
dcos-couchbase-14

Install and Configure Marathon Load Balancer

DC/OS slave nodes are not directly exposed on the Internet. An “external” load balancer can be configured to expose the tasks running on the slaves.

Marathon-lb, short for Marathon Load Balancer, is a load balancer available as a Mesos service. It is based on HAProxy that provides proxying and load balancing for TCP and HTTP based applications, with features such as SSL support, HTTP compression, health checking and more. Marathon-lb subscribes to Marathonโ€™s event bus and updates the HAProxy configuration in real time.

Marathon service UI will show the LB task running:
dcos-couchbase-15

AWS Load Balancer allows port 80 and 443 by default. We’ll run a Couchbase server that will be exposed at port 8091.

In CloudFormation Stack Status  page, copy the value from Values column of PublicSlaveDnsAddress:
dcos-couchbase-16

In AWS Console, select Load Balancers, add a new firewall rule to allow port 8091 on TCP:
dcos-couchbase-17

Run Couchbase Server Docker container on DC/OS

Run Couchbase Server Docker container on DC/OS using the following configuration file:

This configuration file uses arungupta/couchbase image that configures the Couchbase Server using pre-defined Couchbase REST API. This image is Couchbase Docker Image.

cpus and mem attributes define the processing memory needed to run this task.

Give the command to run Couchbase in DC/OS cluster:

Use the previously copied for PublicSlaveDnsAddress and access the Couchbase Web Console at http://<URI>:8091. In our case, the URL is: http://couchbase-publicsl-vjzmwpa38k6d-429093455.us-west-1.elb.amazonaws.com:8091/index.html.

This shows up the login page as:
dcos-couchbase-18

Enter the login credentials as Administrator and password:dcos-couchbase-19

Click on Sign In to see:
dcos-couchbase-20

Learn more about Couchbase Web Console.

Marathon UI is updated to show all the running services:
dcos-couchbase-21

Couchbase Docker image log can be seen in Log Viewer:
dcos-couchbase-22

And the standard output view:
dcos-couchbase-23

Mesos dashboard is updated to show the resources that are consumed:
dcos-couchbase-24

Finally, the complete stack can be deleted from StackFormation template page:
dcos-couchbase-25

Further reading:

  • Latest DC/OS docs
  • DC/OS Installation Guide
  • Get Started with DC/OS
  • Manage your DC/OS Cluster
  • Service discovery and load balancing with DC/OS
  • DC/OS Slack Channel
  • Get Started with Couchbase

Enjoy!

Now, you’ve seen Couchbase on Docker Swarm, Couchbase on Kubernetes, Couchbase on OpenShift 3. This blog showed how to run a Couchbase Docker image on Mesos and DC/OS.

Where else would you like Couchbase container to run?

Source: http://blog.couchbase.com/2016/may/couchbase-mesos-dcos-amazon

Docker container using Apache Mesos and Marathon

apache-mesos-logo apache-mesos-marathon-logo Docker Logo

Apache Mesos is an open source cluster manager developed at UC Berkeley. It provides resource isolation and sharing across distributed applications.

The figure shows the main components of Mesos. Mesos consists of a master daemon that manages slave daemons running on each cluster node. Mesos frameworks are applications that run on Mesos and run tasks on these slaves. Slaves are either physical or virtual machines, typically from the same provider.

mesos-architecture

Mesos uses a two-level scheduling mechanism where resource offers are made to frameworks. The Mesos master node decides how many resources to offer each framework, while each framework determines the resources it accepts and what application to execute on those resources.

Marathon is a container orchestration platform running on Mesos. Multiple container formats are supported and Docker is certainly the most common one!

This blog will show how to setup Mesos, Marathon, and run a simple Docker image. This setup is only for the brave of heart. I’m always interested in looking under the hood and that’s what motivated this post. But a future post will show a more seamless install.

Let’s get started!

Configure CentOS VM

Download CentOS and configure the VM as shown:

centos-7.1-install

Install Components

Install the different components required for this setup.

  1. Configure Mesos repo:
  2. Install Mesos and Marathon: