Tag Archives: javaee

Service Discovery with Java and Database application in Kubernetes

containers, couchbase, javaee, wildfly, , ,

This blog will show how a simple Java application can talk to a database using service discovery in Kubernetes.

 Kubernetes Logo WildFly Logo

Service Discovery with Java and Database application in DC/OS explains why service discovery is an important aspect for a multi-container application. That blog also explained how this can be done for DC/OS.

Let’s see how this can be accomplished in Kubernetes with a single instance of application server and database server. This blog will use WildFly for application server and Couchbase for database.

This blog will use the following main steps:

  • Start Kubernetes one-node cluster
  • Kubernetes application definition
  • Deploy the application
  • Access the application

Start Kubernetes Cluster

Minikube is the easiest way to start a one-node Kubernetes cluster in a VM on your laptop. The binary needs to be downloaded first and then installed.

Complete installation instructions are available at github.com/kubernetes/minikube.

The latest release can be installed on OSX as as:

It also requires kubectl to be installed. Installing and Setting up kubectl provide detailed instructions on how to setup kubectl. On OSX, it can be installed as:

Now, start the cluster as:

The kubectl version command shows more details about the kubectl client and minikube server version:

More details about the cluster can be obtained using the kubectl cluster-info command:

Kubernetes Application Definition

Application definition is defined at github.com/arun-gupta/kubernetes-java-sample/blob/master/service-discovery.yml. It consists of:

  • A Couchbase service
  • Couchbase replica set with a single pod
  • A WildFly replica set with a single pod
The key part is where the value of the COUCHBASE_URI environment variable is name of the Couchbase service. This allows the application deployed in WildFly to dynamically discovery the service and communicate with the database.

arungupta/couchbase:travel Docker image is created using github.com/arun-gupta/couchbase-javaee/blob/master/couchbase/Dockerfile.

arungupta/wildfly-couchbase-javaee:travel Docker image is created using github.com/arun-gupta/couchbase-javaee/blob/master/Dockerfile.

Java EE application waits for database initialization to be complete before it starts querying the database. This can be seen at github.com/arun-gupta/couchbase-javaee/blob/master/src/main/java/org/couchbase/sample/javaee/Database.java#L25.

Deploy Application

This application can be deployed as:

The list of service and replica set can be shown using the command kubectl get svc,rs:

Logs for the single replica of Couchbase can be obtained using the command kubectl logs rs/couchbase-rs:

Logs for the WildFly replica set can be seen using the command kubectl logs rs/wildfly-rs:

Access Application

The kubectl proxy command starts a proxy to the Kubernetes API server. Let’s start a Kubernetes proxy to access our application:

Expose the WildFly replica set as a service using:

The list of services can be seen again using kubectl get svc command:

Now, the application is accessible at:

A formatted output looks like:

Now, new pods may be added as part of Couchbase service by scaling the replica set. Existing pods may be terminated or get rescheduled. But the Java EE application will continue to access the database service using the logical name.

This blog showed how a simple Java application can talk to a database using service discovery in Kubernetes.

For further information check out:

  • Kubernetes Docs
  • Couchbase on Containers
  • Couchbase Developer Portal
  • Ask questions on Couchbase Forums or Stack Overflow
  • Download Couchbase

Microservice using Docker stack deploy – WildFly, Java EE and Couchbase

containers, couchbase, wildfly, , ,

There is plenty of material on microservices, just google it! I gave a presentation on refactoring monolith to microservices at Devoxx Belgium a couple of years back and it has good reviews:

This blog will show how Docker simplifies creation and shutting down of a microservice.

All code used in this blog is at github.com/arun-gupta/couchbase-javaee.

Microservice Definition using Compose

Docker 1.13 introduced a v3 of Docker Compose. The changes in the syntax are minimal but the key difference is addition of deploy attribute. This attribute allows to specify replicas, rolling update and restart policy for the container.

Our microservice will start a WldFly application server with a Java EE application pre-deployed. This application will talk to a Couchbase database to CRUD application data.

Here is the Compose definition:

In this Compose file:

  1. Two services in this Compose are defined by the name db and web attributes
  2. Image name for each service defined using image attribute
  3. The arungupta/couchbase:travel image starts Couchbase server, configures it using Couchbase REST API, and loads travel-sample bucket with ~32k JSON documents.
  4. The arungupta/couchbase-javaee:travel image starts WildFly and deploys application WAR file built from https://github.com/arun-gupta/couchbase-javaee. Clone that project if you want to build your own image.
  5. envrionment attribute defines environment variables accessible by the application deployed in WildFly. COUCHBASE_URI refers to the database service. This is used in the application code as shown at https://github.com/arun-gupta/couchbase-javaee/blob/master/src/main/java/org/couchbase/sample/javaee/Database.java.
  6. Port forwarding is achieved using ports attribute

  7. depends_on attribute in Compose definition file ensures the container start up order. But application-level start up needs to be ensured by the applications running inside container. In our case, WildFly starts up rather quickly but takes a few seconds for the database to start up. This means the Java EE application deployed in WildFly is not able to communicate with the database. This outlines a best practice when building micro services applications: you must code defensively and ensure in your application initialization that the micro services you depend on have started, without assuming startup order. This is shown in the database initialization code at https://github.com/arun-gupta/couchbase-javaee/blob/master/src/main/java/org/couchbase/sample/javaee/Database.java. It performs the following checks:

    1. Bucket exists
    2. Query service of Couchbase is up and running
    3. Sample bucket is fully loaded

This application can be started using docker-compose up -d command on a single host. Or a cluster of Docker engines in swarm-mode using docker stack deploy command.

Setup Docker Swarm-mode

Initialize Swarm mode using the following command:

This starts a Swarm Manager. By default, manager node are also worker but can be configured to be manager-only.

Find some information about this one-node cluster using the command docker info command:

This cluster has 1 node, and that is manager.

Alternatively, a multi-host cluster can be easily setup using Docker for AWS.

Deploy Microservice

The microservice can be started as:

This shows the output:

WildFly and Couchbase services are started on this node. Each service has a single container. If the Swarm mode is enabled on multiple nodes then the containers will be distributed across multiple nodes.

A new overlay network is created. This allows multiple containers on different hosts to communicate with each other.

Verify that the WildFly and Couchbase services are running using docker service ls:

Logs for the service can be seen using docker service logs -f webapp_web:

Make sure to wait for the last log statement to show.

Access Microservice

Get 10 airlines from the microservice:

This shows the results as:

Docker for Java Developers workshop is a self-paced hands-on lab and allows you to get started with Docker easily.

Get a single resource:

Create a new resource:

Update a resource:

Delete a resource:

Detailed output from each of these commands is at github.com/arun-gupta/couchbase-javaee.

Delete Microservice

The microservice can be removed using  the command docker stack rm webapp:

Want to get started with Couchbase? Look at Couchbase Starter Kits.

Want to learn more about running Couchbase in containers?

  • Couchbase on Containers
  • Couchbase Forums
  • Couchbase Developer Portal
  • @couchhasedev and @couchbase

Source: https://blog.couchbase.com/2017/february/microservice-using-docker-stack-deploy-wildfly-javaee-couchbase

Docker Services, Stack and Distributed Application Bundle

containers, couchbase, javaee, wildfly, , ,

docker-1.12

First Release Candidate of Docker 1.12 was announced over two weeks ago. Several new features are planned for this release.

This blog will show how to create a Distributed Application Bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode. Many thanks to @friism to help me understand these concepts.

Let’s look at the features first:

  • Built-in orchestration: A typical application is defined using a Docker Compose file. This definition consists of multiple containers and deployed on multiple hosts. This avoids Single Point of Failure (SPOF) and keeps your application resilient. Multiple orchestration frameworks such as Docker Swarm, Kubernetes and Mesos allow you to orchestrate these applications. However it is such an important characteristic of the application, Docker Engine now has built-in orchestration. More details on this topic in a later blog.
  • Service: A replicated, distributed and load balanced service can be easily created using docker service create command. A “desired state” of the application, such as run 3 containers of Couchbase, is provided and the self-healing Docker engine ensures that that many containers are running in the cluster. If a container goes down, another container is started. If a node goes down, containers on that node are started on a different node. More on this in a later blog.
  • Zero-configuration Security: Docker 1.12 comes with mutually authenticated TLS, providing authentication, authorization and encryption to the communications of every node participating in the swarm, out of the box. More on this in a later blog.
  • Docker Stack and Distributed Application Bundle: Distributed Application Bundle, or DAB, is a multi-services distributable image format. Read further for more details.

So far, you can take a Dockerfile and create an image from it using the docker build command. A container can be started using the docker run command. Multiple containers can be easily started by giving that command multiple times. Or you can also use Docker Compose file and scale up your containers using the docker-compose scale command.

docker-lifecycle

Image is a portable format for a single container. Distributed Application Bundle, or DAB, is a new concept introduced in Docker 1.12, is a portable format for multiple containers. Each bundle can be then deployed as a Stack at runtime.

docker-stack-lifecycle

Learn more about DAB at docker.com/dab.

For simplicity, here is an analogy that can be drawn:

Dockerfile -> Image -> Container

Docker Compose -> Distributed Application Bundle -> Docker Stack

Let’s use a Docker Compose file, create a DAB from it, and deploy it as a Docker Stack.

Its important to note that this is an experimental feature in 1.12-RC2.

Create a Distributed Application Bundle from Docker Compose

Docker Compose CLI adds a new bundle command. More details can be found:

Now, let’s take a Docker Compose definition and create a DAB from it. Here is our Docker Compose definition:

This Compose file starts a WildFly and a Couchbase server. A Java EE application is pre-deployed in the WildFly server that connects to the Couchbase server and allows to perform CRUD operations using the REST API.

The source for this file is at: github.com/arun-gupta/oreilly-docker-book/blob/master/hello-javaee/docker-compose.yml.

Generate an application bundle with it:

depends_on only creates dependency between two services and makes them start in a specific order. This only ensures that the Docker container is started but the application within the container may take longer to start. So this attribute only partially solves the problem. container_name gives a specific name to the container. Relying upon a specific container name is tight coupling and does not allow to scale the container.  So both the warnings can be ignored, for now.

This command generates a file using the Compose project name, which is the directory name. So in our case, hellojavaee.dsb file is generated. This file extension has been renamed to .dab in RC3.

The generated application bundle looks like:

This file provides complete description of the services included in the application. I’m not entirely sure if Distributed Application Bundle is the most appropriate name, discuss this in #24250. It would be great if other container formats, such as Rkt, or even VMs can be supported here. But for now, Docker is the only supported format.

Initialize Swarm Mode in Docker

As mentioned above, “desired state” is now maintained by Docker Swarm. And this is now baked into Docker Engine already.

Docker Swarm concepts have evolved as well and can be read at Swarm mode key concepts. A more detailed blog on this will be coming later.

But for this blog, a new command docker swarm is now added:

Initialize a Swarm node (as a manager) in the Docker Engine:

More details about this node can be found using docker node inspect self command.

The detailed output is verbose but the relevant section is:

The output shows that the node is a manager. For a single-node cluster, this node will also act as a worker.

 

More details about the cluster can be obtained using the docker swarm inspect command.

AcceptancePolicy shows that other worker nodes can join this cluster, but a manager requires explicit approval.

Deploy a Docker Stack

Create a stack using docker deploy command:

The command usage can certainly be simplified as discussed in #24249.

See the list of services:

The output shows that two services, WildFly and Couchbase, are running. Services is also a new concept introduced in Docker 1.12. There is what gives you the “desired state” and Docker Engine works to give you that.

docker ps shows the list of containers running:

WildFly container starts up before the Couchbase container is up and running. This means the Java EE application tries to connect to the Couchbase server and fails. So the application never boots successfully.

Self-healing Docker Service

Docker Service maintains the “desired state” of an application. In our case, the desired state is to ensure that one, and only one, container for the service is running. If we remove the container, not the service, then the service will automatically start the container again.

Remove the container as:

Note, you’ve to give -f because the container is already running. Docker 1.12 self-healing mechanisms kick in and automatically restart the container. Now if you list the containers again:

This shows that a new container has been started.

Inspect the WildFly service:

Swarm assigns a random port to the service, or this can be manually updated using docker service update command. In our case, port 8080 of the container is mapped to 30004 port on the host.

Verify the Application

Check that the application is successfully deployed:

Add a new book to the application:

Verify the books again:

Learn more about this Java EE application at github.com/arun-gupta/oreilly-docker-book/tree/master/hello-javaee.

This blog showed how to create a Distributed Application Bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode.

Docker Service and Stack References

  • Docker Service Create
  • FREE book from O’Reilly: Docker for Java Developers
  • Couchbase on Containers
  • Couchbase Developer Portal
  • Ask questions on @couchbasedev or Stackoverflow
Create a Distributed Application Bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode.… Click To Tweet

Source: blog.couchbase.com/2016/july/docker-services-stack-distributed-application-bundle

Docker Networking with Couchbase and WildFly

containers, couchbase, javaee, wildfly, , , , ,

Docker Multi-Host networking allows you to create virtual networks and attach containers to them so you can create the network topology that is right for your application. This blog will show how to use it with Docker Compose.

CRUD Java Application with Couchbase, Java EE, and WildFly explained how to use a Java EE application to provide a CRUD/REST interface on a data bucket in Couchbase. It required to manually download and run WildFly. The blog also used Couchbase server using Docker and required manual configuration to load travel-sample bucket.

Configure Couchbase Docker Container using REST API explained how to use Couchbase REST API to configure Couchbase Server.

Docker Multi-Host Networking

This blog will remove the explicit download of WildFly and manual configuration of Couchbase server:

  • Use Docker Compose to start WildFly and Couchbase (no download required)
  • Use a Maven profile to configure Couchbase server (no manual configuration required)
  • Uses Docker multi-host networking so that WildFly and Couchbase server can talk to each other

Lets get started!

Start Couchbase and WildFly using Docker Multi-Host Networking and Compose

  1. Start WildFly and Couchbase server using docker-compose.yml file from github.com/arun-gupta/docker-images/blob/master/wildfly-couchbase-javaee7/docker-compose.yml:
    arungupta/wildfly-admin image is used as it binds WildFly’s management to all network interfaces, and in addition also exposes port 9990. This enables WildFly Maven Plugin to be used to deploy the application.

    container_name is specified for Couchbase service and referred in WildFly service using COUCHBASE_URI. This is then used to connect to Couchbase from the Java EE application.

    The application environment is started as:

    --x-networking is an experimental switch added to Docker Compose 1.9 that allows to create a bridge or an overlay network. By default, it creates a bridge network that works on a single host. The network created can be seen as:

    Issue 2221 provide more explanation about the default networks created. wildflycouchbasejavaee7 is the new bridge network created for our application.  Issue #2345 provide some details about incorrect driver name in the output message.

Configure Couchbase Server

  1. Clone couchbase-javaee repo:

  2. Configure Couchbase server:

    exec-maven-plugin is used to invoke REST API and configure Couchbase server and is configured in a Maven profile. Make sure to setup docker.host property in pom.xml.

  3. Deploy the application to WildFly:

    Make sure to specify the correct host on CLI. In this case, this is the IP address obtained using docker-machine ip default.

Invoke the Application

  1. Invoke the REST endpoint using cURL:

    Complete set of REST endpoints are documented at CRUD Java Application with Couchbase, Java EE and WildFly. They are listed here for convenience:

    1. GET a single airline:
    2. Create a new airline using POST:

    3. Update an existing airline using PUT:
    4. Delete an existing airline using DELETE:

Enjoy!

CRUD Java Application with Couchbase, Java EE and WildFly

containers, couchbase, javaee, wildfly, , , ,

Couchbase is an open-source, NoSQL, document database. It allows to access, index, and query JSON documents while taking advantage of integrated distributed caching for high performance data access.

Developers can write applications to Couchbase using different languages (Java, Go, .NET, Node, PHP, Python, C) multiple SDKs. This blog will show how you can easily create a CRUD application using Java SDK for Couchbase.

REST with Couchbase

The application will use curl to issue REST commands to a JAX-RS endpoint deployed on WildFly. These commands will then perform CRUD operations on travel-sample bucket in Couchbase. N1QL (SQL query language for JSON) will be used to communicate with Couchbase to retrieve results. Both the “builder pattern” and raw N1QL commands will be used.

Couchbase CRUD using WildFly and Curl

TL;DR

Complete source code and instructions for the sample are available at github.com/arun-gupta/couchbase-javaee.

Lets get started!

Run Couchbase Server

Couchbase server can be easily downloaded from Couchbase Server Downloads page. In a containerized world, its a lot easier to fire up a Couchbase server using Docker.

If Docker is configured on your machine then the easiest way is to use Docker Compose for Couchbase:

Starting up the application server shows:

And then the logs can be seen as:

The database needs to be configured and is explained at Configure Couchbase Server. Make sure to install travel-sample bucket.

Deploy the Java EE Application on WildFly

  • Download WildFly 9.0.2 , unzip, and start WildFly application server as ./wildfly-9.0.0.Final/bin/standalone.sh.
  • Git clone the repo: git clone https://github.com/arun-gupta/couchbase-javaee.git
  • Change directory cd couchbase-javaee
  • Deploy the application to WildFly: mvn install -Pwildfly.

The application uses Java SDK for Couchbase by importing the following Maven coordinates:

Invoke the REST Endpoints Using cURL

GET Airline resources (limit to 10)

Lets query the database to list 10 Airline resources.

Request

Response

The N1QL query for this is written as:
And can also be alternatively written as:
You may optionally update the code to include ORDER BY clause as shown in N1QL Tutorial.

GET one Airline resource

Use id attribute to query a single Airline resource

Request

Response

POST a new Airline resource

Learn how to run N1QL queries from the CLI using CBQ tool and verify the existing sample data:

This query retrieve documents where airline’s name is Airlinair. The count is shown in metrics.resultCount.

Create a new document using POST.

Request

Response

Query again using CBQ and now the results are shown as:
Note that two JSON documents are returned instead of one as before the POST command was issued.

PUT an existing Airline resource

Update an existing resource using HTTP POST.

The data model for travel-sample bucket requires to include “id” attribute in the payload and in the URI as well.

Request

Name of the airline is updated from “Airlinair” to “Airlin Air”, all other attributes stay the same.

Response

The updated record is shown in the response.

Querying for Airlinair gives:

So the previously added record is now updated and thus does not appear in query results. Querying for Airlin Airgives:

This shows the newly updated document.

DELETE an existing Airline resource

Query for a unique id:

Notice that one document is returned.

Lets delete this document.

Request

Response

The deleted document is shown in the response.

Query again for the deleted id:

And no results are returned!

As mentioned earlier, the complete code base is at github.com/arun-gupta/couchbase-javaee.

Enjoy!

Kubernetes Application – Package Multiple Resources Together

containers, javaee, techtip, , , , ,

Deploying an application in Kubernetes require to create multiple resources such as Pods, Services, Replication Controllers, and others. Typically each resource is define in a configuration file and created using kubectl script. But if multiple resources need to be created then you need to invoke kubectl multiple times. So if you need to create the following resources:

  • MySQL Pod
  • MySQL Service
  • WildFly Replication Controller

Then the commands would look like:

Or for convenience, wrap these invocations in a shell script. But that is not very intuitive! There is a better, and more natural and intuitive way.

Kubernetes allow multiple resources to be specified in a single configuration file. This allows to create a “Kubernetes Application” that can consists of multiple resources easily.

Previous section showed how to deploy the Java EE application using multiple configuration files. This application can be delpoyed using a single configuration file as well.

An application, as discussed above, consisting of MySQL Pod, MySQL Service, and WildFly Replication Controller can be created using the following configuration file: