Category Archives: javaee

Service Discovery with Java and Database application in Kubernetes

containers, couchbase, javaee, wildfly, , ,

This blog will show how a simple Java application can talk to a database using service discovery in Kubernetes.

 Kubernetes Logo WildFly Logo

Service Discovery with Java and Database application in DC/OS explains why service discovery is an important aspect for a multi-container application. That blog also explained how this can be done for DC/OS.

Let’s see how this can be accomplished in Kubernetes with a single instance of application server and database server. This blog will use WildFly for application server and Couchbase for database.

This blog will use the following main steps:

  • Start Kubernetes one-node cluster
  • Kubernetes application definition
  • Deploy the application
  • Access the application

Start Kubernetes Cluster

Minikube is the easiest way to start a one-node Kubernetes cluster in a VM on your laptop. The binary needs to be downloaded first and then installed.

Complete installation instructions are available at github.com/kubernetes/minikube.

The latest release can be installed on OSX as as:

It also requires kubectl to be installed. Installing and Setting up kubectl provide detailed instructions on how to setup kubectl. On OSX, it can be installed as:

Now, start the cluster as:

The kubectl version command shows more details about the kubectl client and minikube server version:

More details about the cluster can be obtained using the kubectl cluster-info command:

Kubernetes Application Definition

Application definition is defined at github.com/arun-gupta/kubernetes-java-sample/blob/master/service-discovery.yml. It consists of:

  • A Couchbase service
  • Couchbase replica set with a single pod
  • A WildFly replica set with a single pod
The key part is where the value of the COUCHBASE_URI environment variable is name of the Couchbase service. This allows the application deployed in WildFly to dynamically discovery the service and communicate with the database.

arungupta/couchbase:travel Docker image is created using github.com/arun-gupta/couchbase-javaee/blob/master/couchbase/Dockerfile.

arungupta/wildfly-couchbase-javaee:travel Docker image is created using github.com/arun-gupta/couchbase-javaee/blob/master/Dockerfile.

Java EE application waits for database initialization to be complete before it starts querying the database. This can be seen at github.com/arun-gupta/couchbase-javaee/blob/master/src/main/java/org/couchbase/sample/javaee/Database.java#L25.

Deploy Application

This application can be deployed as:

The list of service and replica set can be shown using the command kubectl get svc,rs:

Logs for the single replica of Couchbase can be obtained using the command kubectl logs rs/couchbase-rs:

Logs for the WildFly replica set can be seen using the command kubectl logs rs/wildfly-rs:

Access Application

The kubectl proxy command starts a proxy to the Kubernetes API server. Let’s start a Kubernetes proxy to access our application:

Expose the WildFly replica set as a service using:

The list of services can be seen again using kubectl get svc command:

Now, the application is accessible at:

A formatted output looks like:

Now, new pods may be added as part of Couchbase service by scaling the replica set. Existing pods may be terminated or get rescheduled. But the Java EE application will continue to access the database service using the logical name.

This blog showed how a simple Java application can talk to a database using service discovery in Kubernetes.

For further information check out:

  • Kubernetes Docs
  • Couchbase on Containers
  • Couchbase Developer Portal
  • Ask questions on Couchbase Forums or Stack Overflow
  • Download Couchbase

Docker Services, Stack and Distributed Application Bundle

containers, couchbase, javaee, wildfly, , ,

docker-1.12

First Release Candidate of Docker 1.12 was announced over two weeks ago. Several new features are planned for this release.

This blog will show how to create a Distributed Application Bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode. Many thanks to @friism to help me understand these concepts.

Let’s look at the features first:

  • Built-in orchestration: A typical application is defined using a Docker Compose file. This definition consists of multiple containers and deployed on multiple hosts. This avoids Single Point of Failure (SPOF) and keeps your application resilient. Multiple orchestration frameworks such as Docker Swarm, Kubernetes and Mesos allow you to orchestrate these applications. However it is such an important characteristic of the application, Docker Engine now has built-in orchestration. More details on this topic in a later blog.
  • Service: A replicated, distributed and load balanced service can be easily created using docker service create command. A “desired state” of the application, such as run 3 containers of Couchbase, is provided and the self-healing Docker engine ensures that that many containers are running in the cluster. If a container goes down, another container is started. If a node goes down, containers on that node are started on a different node. More on this in a later blog.
  • Zero-configuration Security: Docker 1.12 comes with mutually authenticated TLS, providing authentication, authorization and encryption to the communications of every node participating in the swarm, out of the box. More on this in a later blog.
  • Docker Stack and Distributed Application Bundle: Distributed Application Bundle, or DAB, is a multi-services distributable image format. Read further for more details.

So far, you can take a Dockerfile and create an image from it using the docker build command. A container can be started using the docker run command. Multiple containers can be easily started by giving that command multiple times. Or you can also use Docker Compose file and scale up your containers using the docker-compose scale command.

docker-lifecycle

Image is a portable format for a single container. Distributed Application Bundle, or DAB, is a new concept introduced in Docker 1.12, is a portable format for multiple containers. Each bundle can be then deployed as a Stack at runtime.

docker-stack-lifecycle

Learn more about DAB at docker.com/dab.

For simplicity, here is an analogy that can be drawn:

Dockerfile -> Image -> Container

Docker Compose -> Distributed Application Bundle -> Docker Stack

Let’s use a Docker Compose file, create a DAB from it, and deploy it as a Docker Stack.

Its important to note that this is an experimental feature in 1.12-RC2.

Create a Distributed Application Bundle from Docker Compose

Docker Compose CLI adds a new bundle command. More details can be found:

Now, let’s take a Docker Compose definition and create a DAB from it. Here is our Docker Compose definition:

This Compose file starts a WildFly and a Couchbase server. A Java EE application is pre-deployed in the WildFly server that connects to the Couchbase server and allows to perform CRUD operations using the REST API.

The source for this file is at: github.com/arun-gupta/oreilly-docker-book/blob/master/hello-javaee/docker-compose.yml.

Generate an application bundle with it:

depends_on only creates dependency between two services and makes them start in a specific order. This only ensures that the Docker container is started but the application within the container may take longer to start. So this attribute only partially solves the problem. container_name gives a specific name to the container. Relying upon a specific container name is tight coupling and does not allow to scale the container.  So both the warnings can be ignored, for now.

This command generates a file using the Compose project name, which is the directory name. So in our case, hellojavaee.dsb file is generated. This file extension has been renamed to .dab in RC3.

The generated application bundle looks like:

This file provides complete description of the services included in the application. I’m not entirely sure if Distributed Application Bundle is the most appropriate name, discuss this in #24250. It would be great if other container formats, such as Rkt, or even VMs can be supported here. But for now, Docker is the only supported format.

Initialize Swarm Mode in Docker

As mentioned above, “desired state” is now maintained by Docker Swarm. And this is now baked into Docker Engine already.

Docker Swarm concepts have evolved as well and can be read at Swarm mode key concepts. A more detailed blog on this will be coming later.

But for this blog, a new command docker swarm is now added:

Initialize a Swarm node (as a manager) in the Docker Engine:

More details about this node can be found using docker node inspect self command.

The detailed output is verbose but the relevant section is:

The output shows that the node is a manager. For a single-node cluster, this node will also act as a worker.

 

More details about the cluster can be obtained using the docker swarm inspect command.

AcceptancePolicy shows that other worker nodes can join this cluster, but a manager requires explicit approval.

Deploy a Docker Stack

Create a stack using docker deploy command:

The command usage can certainly be simplified as discussed in #24249.

See the list of services:

The output shows that two services, WildFly and Couchbase, are running. Services is also a new concept introduced in Docker 1.12. There is what gives you the “desired state” and Docker Engine works to give you that.

docker ps shows the list of containers running:

WildFly container starts up before the Couchbase container is up and running. This means the Java EE application tries to connect to the Couchbase server and fails. So the application never boots successfully.

Self-healing Docker Service

Docker Service maintains the “desired state” of an application. In our case, the desired state is to ensure that one, and only one, container for the service is running. If we remove the container, not the service, then the service will automatically start the container again.

Remove the container as:

Note, you’ve to give -f because the container is already running. Docker 1.12 self-healing mechanisms kick in and automatically restart the container. Now if you list the containers again:

This shows that a new container has been started.

Inspect the WildFly service:

Swarm assigns a random port to the service, or this can be manually updated using docker service update command. In our case, port 8080 of the container is mapped to 30004 port on the host.

Verify the Application

Check that the application is successfully deployed:

Add a new book to the application:

Verify the books again:

Learn more about this Java EE application at github.com/arun-gupta/oreilly-docker-book/tree/master/hello-javaee.

This blog showed how to create a Distributed Application Bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode.

Docker Service and Stack References

  • Docker Service Create
  • FREE book from O’Reilly: Docker for Java Developers
  • Couchbase on Containers
  • Couchbase Developer Portal
  • Ask questions on @couchbasedev or Stackoverflow
Create a Distributed Application Bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode.… Click To Tweet

Source: blog.couchbase.com/2016/july/docker-services-stack-distributed-application-bundle

JBoss EAP 7 and NoSQL using Java EE and Docker

containers, couchbase, javaee, jboss, redhat, , , ,

JBoss EAP 7 Beta is now released, many congratulations to Red Hat and particularly to the WildFly team!

There are plenty of improvements coming in this release as documented in Release Notes. One of the major themes is Java EE 7 compliance.

JBoss EAP 7 and Java EE 7

IBM and Oracle already provide commercially supported Java EE 7-compliant Application Servers. And now Red Hat will be joining this party soon as well. Although WildFly has supported Java EE 7 for 2+ years but commercial support is a critical for open source to be adopted enterprise-wide. So this is good news!

You can learn all about different Java EE 7 APIs in the DZone Refcardz that I authored along with @alrubinger.

Java EE 7 Refcardz

There are plenty of “hello world” Java EE 7 Samples that should all run with JBoss EAP. Hopefully somebody will update the pom.xml and add a new profile.

Why NoSQL?

If you are building a traditional enterprise application then you might be fine using an RDBMS. There are plenty of advantages of using RDBMS but using a NoSQL database instead has a few advantages:

  • No need to have a pre-defined schema and that makes them a schema-less database. Addition of new properties to existing objects is easy and does not require ALTER TABLE. The unstructured data gives flexibility to change the format of data any time without downtime or reduced service levels. Also there are no joins happening on the server because there is no structure and thus no relation between them.
  • Scalability, agility and performance is more important than the entire set of functionality typically provided by an RDBMS. This set of databases provide eventual consistency and/or transactions restricted to single items but more focus on CRUD.
  • NoSQL are designed to scale-out (horizontal) instead of scale-up (vertical). This is important knowing that databases, and everything else as well, is moving into the cloud. RBDMS can scale-out using sharding but requires complex management and not for the faint of heart. Queries requiring JOINs across shards is extremely inefficient.
  • RDBMS have impedance mismatch between the database structure and the domain classes. An Object Relational Mapping, such as one provided by Java Persistence API or Hibernate is needed in such case.
  • NoSQL databases are designed for less management and simpler data models lead to lower administration cost as well.

So you are all excited about NoSQL now and want to learn more:

  • Why NoSQL?
  • Why do successful enterprises rely on NoSQL?
  • Top 10 Enterprise NoSQL Usecases

In short, there are four different types of NoSQL databases:

  • Document: Couchbase, Mongo, and others
  • Key/Value: Couchbase, Redis, and others
  • Graph: Neo4J, OrientDB, and others
  • Column: Cassandra and others

Java EE 7 provides Java Persistence API that does not provide any support for NoSQL. So how do you get started with NoSQL with JBoss EAP 7?

This blog will show how to query a Couchbase database using simple Java EE application deployed on JBoss EAP 7 Beta.

What is Couchbase?

Couchbase is an open-source, NoSQL, document database. It allows to access, index, and query JSON documents while taking advantage of integrated distributed caching for high performance data access.

Developers can write applications to Couchbase using different languages (Java, Go, .NET, Node, PHP, Python, C) multiple SDKs. This blog will show how you can easily create a CRUD application using Java SDK for Couchbase.

Run JBoss EAP 7

There are two ways to start JBoss EAP 7.

Download and Run

  • Download JBoss EAP 7 Beta and unzip.
  • Start the application server as:

Docker Run

In a containerized world, you just docker run to run your JBoss EAP. However, JBoss EAP image does not exist on Docker Hub and so the image needs to be explicitly built. You still need to explicitly download JBoss EAP and then use the following Dockerfile to build the image:

The image is built as:

And then you can run the JBoss EAP 7 container as:

Notice, how application and management ports are bound to all network interfaces. This will simplify to deploy the application to this JBoss EAP instance later.

Stop the server as we will show an easier way to start it later.

Start Application Server and Database

The Java EE application will provide a HTTP CRUD interface over JSON documents stored in Couchbase. The application itself will be deployed on JBoss EAP 7 Beta. So it would require to start Couchbase and JBoss EAP.

Use the Docker Compose file from github.com/arun-gupta/docker-images/blob/master/jboss-eap7-nosql/docker-compose.yml to start Couchbase and JBoss EAP 7 container:

The application is started as:

The started containers can be seen as:

Configure Couchbase Server

Clone couchbase-javaee application. This Java EE application uses Couchbase Java SDK APIs to connect to the Couchbase server. The bootstrap code is:

and is invoked from Database abstraction.

Couchbase Server can be configured using REST API. These REST APIs are defined in a Maven profile in pom.xml of this application. And so configure Couchbase server as:

Deploy Java EE Application to JBoss

Java EE Application can be easily deployed to JBoss EAP 7 Beta using the WildFly Maven Plugin. This is also defined as a Maven profile in pom.xml as well.

Deploy the application as:

Access the Application

As mentioned earlier, the application provides HTTP CRUD API over JSON documents stored in Couchbase.

Access the application as:

CRUD operations (GET, POST, PUT, DELETE) can be performed on Airline resource in the application. Complete CRUD API is documented at github.com/arun-gupta/couchbase-javaee.

This blog explained how to access a NoSQL database from JBoss EAP 7.

Read more about Couchbase 4:

  • What’s New in Couchbase Server 4.1
  • Couchbase Server documentation
  • Talk to us on Couchbase Forums
  • Follow @couchbasedev or @couchbase

Learn more about Couchbase in this recent developer-focused webinar:

Docker Machine, Swarm and Compose for multi-container and multi-host applications with Couchbase and WildFly

containers, couchbase, javaee, wildfly, , ,

This blog will explain how to create multi-container application deployed on multiple hosts using Docker. This will be achieved using Docker Machine, Swarm and Compose.

Yes, all three tools together makes this blog that much more interesting!

Docker Swarm Machine Compose

The diagram explains the key components:

  • Docker Machine is used to provision multiple Docker hosts
  • Docker Swarm will be used to create a multi-host cluster
  • Each node in Docker Swarm cluster is registered/discovered using Consul
  • Multi-container application will be deployed using Docker Compose
  • WildFly and Couchbase are provisioned on different hosts
  • Docker multi-host networking is used for WildFly and Couchbase to communicate

In addition, Maven is used to configure Couchbase and deploy application to WildFly.

Latest instructions at Docker for Java Developers.

No story, just pure code, lets do it!

Create Discovery Service using Docker Machine

  1. Create a Machine that will host discovery service:
  2. Connect to this Machine:
  3. Run Consul service using the following Compose file:
    This Compose file is available at https://github.com/arun-gupta/docker-images/blob/master/consul/docker-compose.yml.
    Started container can be verified as:

Create Docker Swarm Cluster using Docker Machine

Swarm is fully integrated with Machine, and so is the easiest way to get started.

  1. Create a Swarm Master and point to the Consul discovery service:
    Few options to look here:

    1. --swarm configures the Machine with Swarm
    2. --swarm-master configures the created Machine to be Swarm master
    3. --swarm-discovery defines address of the discovery service
    4. --cluster-advertise advertise the machine on the network
    5. --cluster-store designate a distributed k/v storage backend for the cluster
    6. --virtualbox-disk-size sets the disk size for the created Machine to 5GB. This is required so that WildFly and Couchbase image can be downloaded on any of the nodes.
  2. Find some information about this machine:
    Note that the disk size is 5GB.
  3. Connect to the master by using the command:
  4. Find some information about the cluster:
  5. Create a new Machine to join this cluster:
    Notice no --swarm-master is specified in this command. This ensure that the created Machines are worker nodes.
  6. Create a second Swarm node to join this cluster:
  7. List all the created Machines:
    The machines that are part of the cluster have cluster’s name in the SWARM column, blank otherwise. For example,consul-machine is a standalone machine where as all other machines are part of the swarm-master cluster. The Swarm master is also identified by (master) in the SWARM column.
  8. Connect to the Swarm cluster and find some information about it:

    Note, --swarm is specified to connect to the Swarm cluster. Otherwise the command will connect to swarm-masterMachine only.

    This shows the output as:

    There are 3 nodes – one Swarm master and 2 Swarm worker nodes. There is a total of 4 containers running in this cluster – one Swarm agent on master and each node, and there is an additional swarm-agent-master running on the master. This can be verified by connecting to the master and listing all the containers.

  9. List nodes in the cluster with the following command:

Start Application Environment using Docker Compose

Make sure you are connected to the cluster by giving the command eval "$(docker-machine env --swarm swarm-master)".

  1. List all the networks created by Docker so far:
    Docker create three networks for each host automatically:

    Network Name Purpose
    bridge Default network that containers connect to. This is docker0 network in all Docker installations.
    none Container-specific networking stack
    host Adds a container on hosts networking stack. Network configuration is identical to the host.

    This explains a total of nine networks, three for each node, as shown in this Swarm cluster.

  2. Use Compose file to start WildFly and Couchbase:

    In this Compose file:

    1. Couchbase service has a custom container name defined by container_name. This name is used when creating a new environment variable COUCHBASE_URI during WildFly startup.
    2. arungupta/wildfly-admin image is used as it binds WildFly’s management to all network interfaces, and in addition also exposes port 9990. This enables WildFly Maven Plugin to be used to deploy the application.Source for this file is at https://github.com/arun-gupta/docker-images/blob/master/wildfly-couchbase-javaee7/docker-compose.yml.

    This application environment can be started as:

    --x-networking creates an overlay network for the Swarm cluster. This can be verified by listing networks again:

    Three new networks are created:

    1. Containers connected to the multi-host network are automatically connected to the docker_gwbridge network. This network allows the containers to have external connectivity outside of their cluster, and is created on each worker node.
    2. A new overlay network wildflycouchbasejavaee7 is created. Connect to different Swarm nodes and check that the overlay network exists on them.

      Lets begin with master:

      Next, with swarm-node-01:

      Finally, with swarm-node-02:

      As seen, wildflycouchbasejavaee7 overlay network exists on all Machines. This confirms that the overlay network created for Swarm cluster was added to each host in the cluster. docker_gwbridge only exists on Machines that have application containers running.

      Read more about Docker Networks.

  3. Verify that WildFly and Couchbase are running: