Category Archives: java

Creating Smaller Java Image using Docker Multi-stage Build

April 21, 2017containers, java, techtipdocker, image, javaarungupta

Two of the announcements at DockerCon 2017 directly relevant to Java developers are:

Docker Multi-stage build
Oracle JRE in Docker Store

This blog explains the purpose of Docker multi-stage build and provide examples of how they help us generate smaller and more efficient Java Docker images.

Just show me the code: github.com/arun-gupta/docker-java-multistage.

What is the issue?

Building a Docker image for a Java application typically involves building the application and package the generated artifact into an image. A Java developer would likely use Maven or Gradle to build a JAR or WAR file. If you are using the Maven base image to build the application then it will download the required dependencies from the configured repositories and keep them in the image. The number of JARs in the local repository could be significant depending upon the number of dependencies in the pom.xml. This could leave a lot of cruft in the image.

Let’s take a look at a sample Dockerfile:

FROM maven:3.5-jdk-8

COPY src /usr/src/myapp/src
COPY pom.xml /usr/src/myapp
RUN mvn -f /usr/src/myapp/pom.xml clean package

ENV WILDFLY_VERSION 10.1.0.Final
ENV WILDFLY_HOME /usr

RUN cd $WILDFLY_HOME && curl http://download.jboss.org/wildfly/$WILDFLY_VERSION/wildfly-$WILDFLY_VERSION.tar.gz | tar zx && mv $WILDFLY_HOME/wildfly-$WILDFLY_VERSION $WILDFLY_HOME/wildfly

RUN cp /usr/src/myapp/target/people-1.0-SNAPSHOT.war $WILDFLY_HOME/wildfly/standalone/deployments/people.war

EXPOSE 8080

CMD ["/usr/wildfly/bin/standalone.sh", "-b", "0.0.0.0"]

FROM maven:3.5-jdk-8

COPY src /usr/src/myapp/src

COPY pom.xml /usr/src/myapp

RUN mvn -f /usr/src/myapp/pom.xml clean package

ENV WILDFLY_VERSION 10.1.0.Final

ENV WILDFLY_HOME /usr

RUN cd $WILDFLY_HOME && curl http://download.jboss.org/wildfly/$WILDFLY_VERSION/wildfly-$WILDFLY_VERSION.tar.gz | tar zx && mv $WILDFLY_HOME/wildfly-$WILDFLY_VERSION $WILDFLY_HOME/wildfly

RUN cp /usr/src/myapp/target/people-1.0-SNAPSHOT.war $WILDFLY_HOME/wildfly/standalone/deployments/people.war

EXPOSE 8080

CMD ["/usr/wildfly/bin/standalone.sh", "-b", "0.0.0.0"]

In this Dockerfile:

maven:3.5-jdk-8 is used as the base image
Application source code is copied to the image
Maven is used to build the application artifact
WildFly is downloaded and installed
Generated artifact is copied to the deployments directory of WildFly
Finally, WildFly is started

There are several issues with this kind of flow:

Using maven as the base image restricts on what functionality is available in the image. This requires WildFly to be downloaded and configured explicitly.
Building the artifact downloads all Maven dependencies. These stay in the image and are not needed at runtime. This causes an unnecessary bloat in the image size at runtime.
Change in WildFly version will require to update the Dockerfile. This would’ve been much easier if we could use the jboss/wildfly base image by itself.
In addition, unit tests may run before packaging the artifact and integration tests after the image is created. The test dependencies and results is again not needed to live in the production image.

There are other ways to build the Docker image. For example, splitting the Dockerfile into two files. The first file will then build the artifact and copy the artifact to a common location using volume mapping. The second file will then pick up the generated artifact and then use the lean base image. This approach has also has issues where multiple Dockerfiles need to be maintained separately. Additional, there is an out-of-band hand-off between the two Dockerfiles.

Let’s see how these issues are resolved with multi-stage build.

What are Docker multi-stage build?

Multi-stage build allows multiple FROM statements in a Dockerfile. The instructions following each FROM statement and until the next one, creates an intermediate image. The final FROM statement is the final base image. Artifacts from intermediate stages can be copied using COPY --from=<image-number>, starting from 0 for the first base image. The artifacts not copied over are discarded. This allows to keep the final image lean and only include the relevant artifacts.

FROM syntax is updated to specify stage name using as <stage-name>. For example:

FROM maven:3.5-jdk-8 as BUILD

FROM maven:3.5-jdk-8 as BUILD

This allows to use the stage name instead of the number with --from option.

Let’s take a look at a sample Dockerfile:

FROM maven:3.5-jdk-8 as BUILD

COPY src /usr/src/myapp/src
COPY pom.xml /usr/src/myapp
RUN mvn -f /usr/src/myapp/pom.xml clean package

FROM jboss/wildfly:10.1.0.Final

COPY --from=BUILD /usr/src/myapp/target/people-1.0-SNAPSHOT.war /opt/jboss/wildfly/standalone/deployments/people.war

FROM maven:3.5-jdk-8 as BUILD

COPY src /usr/src/myapp/src

COPY pom.xml /usr/src/myapp

RUN mvn -f /usr/src/myapp/pom.xml clean package

FROM jboss/wildfly:10.1.0.Final

COPY --from=BUILD /usr/src/myapp/target/people-1.0-SNAPSHOT.war /opt/jboss/wildfly/standalone/deployments/people.war

In this Dockerfile:

There are two FROM instructions. This means this is a two-stage build.
maven:3.5-jdk-8 is the base image for the first build. This is used to build the WAR file for the application. The first stage is named as BUILD.
jboss/wildfly:10.1.0.Final is the second and the final base image for the build. WAR file generated in the first stage is copied over to this stage using COPY --from syntax. The file is directly copied in the WildFly deployments directory.

Let’s take a look at what are some of the advantages of this approach.

Advantages of Docker multi-stage build

One Dockerfile has the entire build process defined. There is no need to have separate Dockerfiles and then coordinate transfer of artifact between “build” Dockerfile and “run” Dockerfile using volume mapping.
Base image for the final image can be chosen appropriately to meet the runtime needs. This helps with reduction of the overall size of the runtime image. Additionally, the cruft from build time is discarded during intermediate stage.
Standard WildFly base image is used instead of downloading and configuring the distribution manually. This makes it a lot easier to update the image if a newer tag is released.

Size of the image built using a single Dockerfile is 816MB. In contrast, the size of the image built using multi-stage build is 584MB.

So, using a multi-stage helps create a much smaller image.

Is this a typical way of building Docker image? Are there other ways by which the image size can be reduced?

Sure, you can use docker-maven-plugin as shown at github.com/arun-gupta/docker-java-sample to build/test the image locally and then push to repo. But this mechanism allows you to generate and package artifact without any other dependency, including Java.

Sure, maven:jdk-8-alpine image can be used to create a smaller image. But then you’ll have to create or find a WildFly image built using jdk-8-alpine, or something similar, as well. But the cruft, such as maven repository, two Dockerfiles, sharing of artifact using volume mapping or some other similar technique would still be there.

There are other ways to craft your build cycle. But if you are using Dockerfile to build your artifact then you should seriously consider multi-stage builds.

Service Discovery with Java and Database application in Kubernetes

March 11, 2017containers, couchbase, javaee, wildflycouchbase, javaee, kubernetes, wildflyarungupta

This blog will show how a simple Java application can talk to a database using service discovery in Kubernetes.

Service Discovery with Java and Database application in DC/OS explains why service discovery is an important aspect for a multi-container application. That blog also explained how this can be done for DC/OS.

Let’s see how this can be accomplished in Kubernetes with a single instance of application server and database server. This blog will use WildFly for application server and Couchbase for database.

This blog will use the following main steps:

Start Kubernetes one-node cluster
Kubernetes application definition
Deploy the application
Access the application

Start Kubernetes Cluster

Minikube is the easiest way to start a one-node Kubernetes cluster in a VM on your laptop. The binary needs to be downloaded first and then installed.

Complete installation instructions are available at github.com/kubernetes/minikube.

The latest release can be installed on OSX as as:

curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.17.1/minikube-darwin-amd64 \
&& chmod +x minikube

curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.17.1/minikube-darwin-amd64 \

&& chmod +x minikube

It also requires kubectl to be installed. Installing and Setting up kubectl provide detailed instructions on how to setup kubectl. On OSX, it can be installed as:

curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl \
  && chmod +x ./kubectl

curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl \

&& chmod +x ./kubectl

Now, start the cluster as:

minikube start
Starting local Kubernetes cluster...
Starting VM...
Downloading Minikube ISO
 88.71 MB / 88.71 MB [==============================================] 100.00% 0s
SSH-ing files into VM...
Setting up certs...
Starting cluster components...
Connecting to cluster...
Setting up kubeconfig...
Kubectl is now configured to use the cluster.

minikube start

Starting local Kubernetes cluster...

Starting VM...

Downloading Minikube ISO

88.71 MB / 88.71 MB [==============================================] 100.00% 0s

SSH-ing files into VM...

Setting up certs...

Starting cluster components...

Connecting to cluster...

Setting up kubeconfig...

Kubectl is now configured to use the cluster.

The kubectl version command shows more details about the kubectl client and minikube server version:

kubectl version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.4", GitCommit:"7243c69eb523aa4377bce883e7c0dd76b84709a1", GitTreeState:"clean", BuildDate:"2017-03-07T23:53:09Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.3", GitCommit:"029c3a408176b55c30846f0faedf56aae5992e9b", GitTreeState:"clean", BuildDate:"1970-01-01T00:00:00Z", GoVersion:"go1.7.3", Compiler:"gc", Platform:"linux/amd64"}

kubectl version

Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.4", GitCommit:"7243c69eb523aa4377bce883e7c0dd76b84709a1", GitTreeState:"clean", BuildDate:"2017-03-07T23:53:09Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"darwin/amd64"}

Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.3", GitCommit:"029c3a408176b55c30846f0faedf56aae5992e9b", GitTreeState:"clean", BuildDate:"1970-01-01T00:00:00Z", GoVersion:"go1.7.3", Compiler:"gc", Platform:"linux/amd64"}

More details about the cluster can be obtained using the kubectl cluster-info command:

Kubernetes master is running at https://192.168.99.100:8443
KubeDNS is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Kubernetes master is running at https://192.168.99.100:8443

KubeDNS is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kube-dns

kubernetes-dashboard is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Kubernetes Application Definition

Application definition is defined at github.com/arun-gupta/kubernetes-java-sample/blob/master/service-discovery.yml. It consists of:

A Couchbase service
Couchbase replica set with a single pod
A WildFly replica set with a single pod

apiVersion: v1
kind: Service
metadata: 
  name: couchbase-service
spec: 
  selector: 
    app: couchbase-rs-pod
  ports:
    - name: admin
      port: 8091
    - name: views
      port: 8092
    - name: query
      port: 8093
    - name: memcached
      port: 11210
---
apiVersion: extensions/v1beta1
kind: ReplicaSet
metadata:
  name: couchbase-rs
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: couchbase-rs-pod
    spec:
      containers:
      - name: couchbase
        image: arungupta/couchbase:travel
        ports:
        - containerPort: 8091
        - containerPort: 8092
        - containerPort: 8093
        - containerPort: 11210
---
apiVersion: extensions/v1beta1
kind: ReplicaSet
metadata:
  name: wildfly-rs
  labels:
    name: wildfly
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: wildfly
    spec:
      containers:
      - name: wildfly-rs-pod
        image: arungupta/wildfly-couchbase-javaee:travel
        env:
        - name: COUCHBASE_URI
          value: couchbase-service
        ports:
        - containerPort: 8080

apiVersion: v1

kind: Service

metadata:

spec:

selector:

app: couchbase-rs-pod

ports:

- name: admin

port: 8091

- name: views

port: 8092

- name: query

port: 8093

- name: memcached

port: 11210

---

apiVersion: extensions/v1beta1

kind: ReplicaSet

metadata:

spec:

replicas: 1

template:

metadata:

labels:

app: couchbase-rs-pod

spec:

containers:

- name: couchbase

image: arungupta/couchbase:travel

ports:

- containerPort: 8091

- containerPort: 8092

- containerPort: 8093

- containerPort: 11210

---

apiVersion: extensions/v1beta1

kind: ReplicaSet

metadata:

labels:

spec:

replicas: 1

template:

metadata:

labels:

spec:

containers:

- name: wildfly-rs-pod

image: arungupta/wildfly-couchbase-javaee:travel

env:

- name: COUCHBASE_URI

value: couchbase-service

ports:

- containerPort: 8080

The key part is where the value of the COUCHBASE_URI environment variable is name of the Couchbase service. This allows the application deployed in WildFly to dynamically discovery the service and communicate with the database.

arungupta/couchbase:travel Docker image is created using github.com/arun-gupta/couchbase-javaee/blob/master/couchbase/Dockerfile.

arungupta/wildfly-couchbase-javaee:travel Docker image is created using github.com/arun-gupta/couchbase-javaee/blob/master/Dockerfile.

Java EE application waits for database initialization to be complete before it starts querying the database. This can be seen at github.com/arun-gupta/couchbase-javaee/blob/master/src/main/java/org/couchbase/sample/javaee/Database.java#L25.

Deploy Application

This application can be deployed as:

kubectl create -f ~/workspaces/kubernetes-java-sample/service-discovery.yml

kubectl create -f ~/workspaces/kubernetes-java-sample/service-discovery.yml

The list of service and replica set can be shown using the command kubectl get svc,rs:

NAME                    CLUSTER-IP   EXTERNAL-IP   PORT(S)                                AGE
svc/couchbase-service   10.0.0.97    <none>        8091/TCP,8092/TCP,8093/TCP,11210/TCP   27m
svc/kubernetes          10.0.0.1     <none>        443/TCP                                1h
svc/wildfly-rs          10.0.0.252   <none>        8080/TCP                               21m

NAME              DESIRED   CURRENT   READY     AGE
rs/couchbase-rs   1         1         1         27m
rs/wildfly-rs     1         1         1         27m

NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE

svc/couchbase-service 10.0.0.97 <none> 8091/TCP,8092/TCP,8093/TCP,11210/TCP 27m

svc/kubernetes 10.0.0.1 <none> 443/TCP 1h

svc/wildfly-rs 10.0.0.252 <none> 8080/TCP 21m

NAME DESIRED CURRENT READY AGE

rs/couchbase-rs 1 1 1 27m

rs/wildfly-rs 1 1 1 27m

Logs for the single replica of Couchbase can be obtained using the command kubectl logs rs/couchbase-rs:

++ set -m
++ sleep 25
++ /entrypoint.sh couchbase-server
Starting Couchbase Server -- Web UI available at http://<ip>:8091 and logs available in /opt/couchbase/var/lib/couchbase/logs
++ curl -v -X POST http://127.0.0.1:8091/pools/default -d memoryQuota=300 -d indexMemoryQuota=300

. . .

{"storageMode":"memory_optimized","indexerThreads":0,"memorySnapshotInterval":200,"stableSnapshotInterval":5000,"maxRollbackPoints":5,"logLevel":"info"}[]Type: 
++ echo 'Type: '
++ '[' '' = WORKER ']'
++ fg 1
/entrypoint.sh couchbase-server

++ set -m

++ sleep 25

++ /entrypoint.sh couchbase-server

Starting Couchbase Server -- Web UI available at http://<ip>:8091 and logs available in /opt/couchbase/var/lib/couchbase/logs

++ curl -v -X POST http://127.0.0.1:8091/pools/default -d memoryQuota=300 -d indexMemoryQuota=300

. . .

{"storageMode":"memory_optimized","indexerThreads":0,"memorySnapshotInterval":200,"stableSnapshotInterval":5000,"maxRollbackPoints":5,"logLevel":"info"}[]Type:

++ echo 'Type: '

++ '[' '' = WORKER ']'

++ fg 1

/entrypoint.sh couchbase-server

Logs for the WildFly replica set can be seen using the command kubectl logs rs/wildfly-rs:

=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /opt/jboss/wildfly

. . .

06:32:08,537 INFO  [com.couchbase.client.core.node.Node] (cb-io-1-1) Connected to Node couchbase-service
06:32:09,262 INFO  [com.couchbase.client.core.config.ConfigurationProvider] (cb-computations-3) Opened bucket travel-sample
06:32:09,366 INFO  [stdout] (ServerService Thread Pool -- 65) Sleeping for 3 secs ...
06:32:12,369 INFO  [stdout] (ServerService Thread Pool -- 65) Bucket found!
06:32:14,194 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 65) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.couchbase.sample.javaee.MyApplication
06:32:14,195 INFO  [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 65) RESTEASY002200: Adding class resource org.couchbase.sample.javaee.AirlineResource from Application class org.couchbase.sample.javaee.MyApplication
06:32:14,310 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 65) WFLYUT0021: Registered web context: /airlines
06:32:14,376 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 34) WFLYSRV0010: Deployed "airlines.war" (runtime-name : "airlines.war")
06:32:14,704 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management
06:32:14,704 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990
06:32:14,705 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 10.1.0.Final (WildFly Core 2.2.0.Final) started in 29470ms - Started 443 of 691 services (404 services are lazy, passive or on-demand)

=========================================================================

JBoss Bootstrap Environment

JBOSS_HOME: /opt/jboss/wildfly

. . .

06:32:08,537 INFO [com.couchbase.client.core.node.Node] (cb-io-1-1) Connected to Node couchbase-service

06:32:09,262 INFO [com.couchbase.client.core.config.ConfigurationProvider] (cb-computations-3) Opened bucket travel-sample

06:32:09,366 INFO [stdout] (ServerService Thread Pool -- 65) Sleeping for 3 secs ...

06:32:12,369 INFO [stdout] (ServerService Thread Pool -- 65) Bucket found!

06:32:14,194 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 65) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.couchbase.sample.javaee.MyApplication

06:32:14,195 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 65) RESTEASY002200: Adding class resource org.couchbase.sample.javaee.AirlineResource from Application class org.couchbase.sample.javaee.MyApplication

06:32:14,310 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 65) WFLYUT0021: Registered web context: /airlines

06:32:14,376 INFO [org.jboss.as.server] (ServerService Thread Pool -- 34) WFLYSRV0010: Deployed "airlines.war" (runtime-name : "airlines.war")

06:32:14,704 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management

06:32:14,704 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990

06:32:14,705 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 10.1.0.Final (WildFly Core 2.2.0.Final) started in 29470ms - Started 443 of 691 services (404 services are lazy, passive or on-demand)

Access Application

The kubectl proxy command starts a proxy to the Kubernetes API server. Let’s start a Kubernetes proxy to access our application:

kubectl proxy
Starting to serve on 127.0.0.1:8001

kubectl proxy

Starting to serve on 127.0.0.1:8001

Expose the WildFly replica set as a service using:

kubectl expose --name=wildfly-service rs/wildfly-rs

kubectl expose --name=wildfly-service rs/wildfly-rs

The list of services can be seen again using kubectl get svc command:

kubectl get svc
NAME                CLUSTER-IP   EXTERNAL-IP   PORT(S)                                AGE
couchbase-service   10.0.0.97    <none>        8091/TCP,8092/TCP,8093/TCP,11210/TCP   41m
kubernetes          10.0.0.1     <none>        443/TCP                                1h
wildfly-service     10.0.0.169   <none>        8080/TCP                               5s

kubectl get svc

NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE

couchbase-service 10.0.0.97 <none> 8091/TCP,8092/TCP,8093/TCP,11210/TCP 41m

kubernetes 10.0.0.1 <none> 443/TCP 1h

wildfly-service 10.0.0.169 <none> 8080/TCP 5s

Now, the application is accessible at:

curl http://localhost:8001/api/v1/proxy/namespaces/default/services/wildfly-service/airlines/resources/airline

curl http://localhost:8001/api/v1/proxy/namespaces/default/services/wildfly-service/airlines/resources/airline

A formatted output looks like:

[
  {
    "travel-sample": {
      "country": "United States",
      "iata": "Q5",
      "callsign": "MILE-AIR",
      "name": "40-Mile Air",
      "icao": "MLA",
      "id": 10,
      "type": "airline"
    }
  },
  {
    "travel-sample": {
      "country": "United States",
      "iata": "TQ",

. . .

     "name": "Airlinair",
      "icao": "RLA",
      "id": 1203,
      "type": "airline"
    }
  }
]

[

{

"travel-sample": {

"country": "United States",

"iata": "Q5",

"callsign": "MILE-AIR",

"name": "40-Mile Air",

"icao": "MLA",

"id": 10,

"type": "airline"

}

{

"travel-sample": {

"country": "United States",

"iata": "TQ",

. . .

"name": "Airlinair",

"icao": "RLA",

"id": 1203,

"type": "airline"

}

]

Now, new pods may be added as part of Couchbase service by scaling the replica set. Existing pods may be terminated or get rescheduled. But the Java EE application will continue to access the database service using the logical name.

This blog showed how a simple Java application can talk to a database using service discovery in Kubernetes.

For further information check out:

Kubernetes Docs
Couchbase on Containers
Couchbase Developer Portal
Ask questions on Couchbase Forums or Stack Overflow
Download Couchbase

Minikube – Rapid Dev & Testing for Kubernetes

September 30, 2016containers, couchbase, javacontainers, couchbase, kubernetes, springarungupta

One of the attendees from Kubernetes for Java Developers training suggested to try minikube for simplified Kubernetes dev and testing. This blog will show how to get started with minikube using a simple Java application.

Minikube starts a single node Kubernetes cluster on your local machine for rapid development and testing. Requirements lists the exact set of requirements for different operating systems.

This blog will show:

Start one node Kubernetes cluster
Run Couchbase service
Run Java application
View Kubernetes Dashboard

All Kubernetes resource description files used in this blog are at github.com/arun-gupta/kubernetes-java-sample/tree/master/maven.

Start Kubernetes Cluster using Minikube

Create a new directory with the name minikube.

In that directory, download kubectl CLI:

curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/v1.4.0/bin/darwin/amd64/kubectl && chmod +x kubectl

curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/v1.4.0/bin/darwin/amd64/kubectl && chmod +x kubectl

Download minikube CLI:

curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.10.0/minikube-darwin-amd64 && chmod +x minikube

curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.10.0/minikube-darwin-amd64 && chmod +x minikube

Start the cluster:

minikube start
Starting local Kubernetes cluster...
Kubectl is now configured to use the cluster.

minikube start

Starting local Kubernetes cluster...

Kubectl is now configured to use the cluster.

The list of nodes can be seen:

kubectl get nodes
NAME       STATUS    AGE
minikube   Ready     2h

kubectl get nodes

NAME STATUS AGE

minikube Ready 2h

More details about the cluster can be obtained using the kubectl cluster-info command:

kubectl cluster-info
Kubernetes master is running at https://192.168.99.100:8443
kubernetes-dashboard is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

kubectl cluster-info

Kubernetes master is running at https://192.168.99.100:8443

kubernetes-dashboard is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Behind the scenes, a Virtual Box VM is started.

Complete set of commands supported can be seen by using --help:

minikube --help
Minikube is a CLI tool that provisions and manages single-node Kubernetes clusters optimized for development workflows.

Usage:
  minikube [command]

Available Commands:
  dashboard        Opens/displays the kubernetes dashboard URL for your local cluster
  delete           Deletes a local kubernetes cluster.
  docker-env       sets up docker env variables; similar to '$(docker-machine env)'
  get-k8s-versions Gets the list of available kubernetes versions available for minikube.
  ip               Retrieve the IP address of the running cluster.
  logs             Gets the logs of the running localkube instance, used for debugging minikube, not user code.
  config           Modify minikube config
  service          Gets the kubernetes URL for the specified service in your local cluster
  ssh              Log into or run a command on a machine with SSH; similar to 'docker-machine ssh'
  start            Starts a local kubernetes cluster.
  status           Gets the status of a local kubernetes cluster.
  stop             Stops a running local kubernetes cluster.
  version          Print the version of minikube.

Flags:
      --alsologtostderr[=false]: log to standard error as well as files
      --log-flush-frequency=5s: Maximum number of seconds between log flushes
      --log_backtrace_at=:0: when logging hits line file:N, emit a stack trace
      --log_dir="": If non-empty, write log files in this directory
      --logtostderr[=false]: log to standard error instead of files
      --show-libmachine-logs[=false]: Whether or not to show logs from libmachine.
      --stderrthreshold=2: logs at or above this threshold go to stderr
      --v=0: log level for V logs
      --vmodule=: comma-separated list of pattern=N settings for file-filtered logging

Use "minikube [command] --help" for more information about a command.

minikube --help

Minikube is a CLI tool that provisions and manages single-node Kubernetes clusters optimized for development workflows.

Usage:

minikube [command]

Available Commands:

dashboard Opens/displays the kubernetes dashboard URL for your local cluster

delete Deletes a local kubernetes cluster.

docker-env sets up docker env variables; similar to '$(docker-machine env)'

get-k8s-versions Gets the list of available kubernetes versions available for minikube.

ip Retrieve the IP address of the running cluster.

logs Gets the logs of the running localkube instance, used for debugging minikube, not user code.

config Modify minikube config

service Gets the kubernetes URL for the specified service in your local cluster

ssh Log into or run a command on a machine with SSH; similar to 'docker-machine ssh'

start Starts a local kubernetes cluster.

status Gets the status of a local kubernetes cluster.

stop Stops a running local kubernetes cluster.

version Print the version of minikube.

Flags:

--alsologtostderr[=false]: log to standard error as well as files

--log-flush-frequency=5s: Maximum number of seconds between log flushes

--log_backtrace_at=:0: when logging hits line file:N, emit a stack trace

--log_dir="": If non-empty, write log files in this directory

--logtostderr[=false]: log to standard error instead of files

--show-libmachine-logs[=false]: Whether or not to show logs from libmachine.

--stderrthreshold=2: logs at or above this threshold go to stderr

--v=0: log level for V logs

--vmodule=: comma-separated list of pattern=N settings for file-filtered logging

Use "minikube [command] --help" for more information about a command.

Run Couchbase Service

Create a Couchbase service:

kubectl create -f couchbase-service.yml 
service "couchbase-service" created
replicationcontroller "couchbase-rc" created

kubectl create -f couchbase-service.yml

service "couchbase-service" created

replicationcontroller "couchbase-rc" created

This will start a Couchbase service. The service is using the pods created by the replication controller. The replication controller creates a single node Couchbase server.

The configuration file is at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/couchbase-service.yml and looks like:

apiVersion: v1
kind: Service
metadata: 
  name: couchbase-service
spec: 
  selector: 
    app: couchbase-rc-pod
  ports:
    - name: admin
      port: 8091
    - name: views
      port: 8092
    - name: query
      port: 8093
    - name: memcached
      port: 11210
---
apiVersion: v1
kind: ReplicationController
metadata:
  name: couchbase-rc
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: couchbase-rc-pod
    spec:
      containers:
      - name: couchbase
        image: arungupta/oreilly-couchbase
        ports:
        - containerPort: 8091
        - containerPort: 8092
        - containerPort: 8093
        - containerPort: 11210

apiVersion: v1

kind: Service

metadata:

spec:

selector:

app: couchbase-rc-pod

ports:

- name: admin

port: 8091

- name: views

port: 8092

- name: query

port: 8093

- name: memcached

port: 11210

---

apiVersion: v1

kind: ReplicationController

metadata:

spec:

replicas: 1

template:

metadata:

labels:

app: couchbase-rc-pod

spec:

containers:

- name: couchbase

image: arungupta/oreilly-couchbase

ports:

- containerPort: 8091

- containerPort: 8092

- containerPort: 8093

- containerPort: 11210

Run Java Application

Run the application:

kubectl create -f bootiful-couchbase.yml 
job "bootiful-couchbase" created

kubectl create -f bootiful-couchbase.yml

job "bootiful-couchbase" created

The configuration file is at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/bootiful-couchbase.yml and looks like:

apiVersion: batch/v1
kind: Job
metadata:
  name: bootiful-couchbase
  labels:
    name: bootiful-couchbase-pod
spec:
  template:
    metadata:
      name: bootiful-couchbase-pod
    spec:
      containers:
      - name: bootiful-couchbase
        image: arungupta/bootiful-couchbase
        env:
        - name: COUCHBASE_URI
          value: couchbase-service
      restartPolicy: Never

apiVersion: batch/v1

kind: Job

metadata:

labels:

spec:

template:

metadata:

spec:

containers:

- name: bootiful-couchbase

image: arungupta/bootiful-couchbase

env:

- name: COUCHBASE_URI

value: couchbase-service

restartPolicy: Never

This is run-once job which runs a Java (Spring Boot) application and upserts (insert or update) a JSON document in Couchbase.

In this job, COUCHBASE_URI environment variable value is set to couchbase-service. This is the service name created earlier. Docker image used for this service is arungupta/bootiful-couchbase and is created using fabric8-maven-plugin as shown at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/webapp/pom.xml#L57-L68. Specifically, the command for the Docker image is:

java -Dspring.couchbase.bootstrap-hosts=$COUCHBASE_URI -jar /maven/${project.artifactId}.jar

java -Dspring.couchbase.bootstrap-hosts=$COUCHBASE_URI -jar /maven/${project.artifactId}.jar

This ensures that COUCHBASE_URI environment variable is overriding spring.couchbase.bootstrap-hosts property as defined in application.properties of the Spring Boot application.

Kubernetes Dashboard

Kubernetes 1.4 included an updated dashboard. For minikube, this can be opened using the following command:

minikube dashboard
Waiting, endpoint for service is not ready yet...Opening kubernetes dashboard in default browser...

minikube dashboard

Waiting, endpoint for service is not ready yet...Opening kubernetes dashboard in default browser...

The default view is shown below:

But in our case, a few resources have already been created and so this will look like as shown:

Notice, our Jobs, Replication Controllers and Pods are shown here.

Shutdown Kubernetes Cluster

The cluster can be easily shutdown:

minikube stop
Stopping local Kubernetes cluster...
Machine stopped.

minikube stop

Stopping local Kubernetes cluster...

Machine stopped.

couchbase.com/containers provide more details about running Couchbase using different orchestration frameworks. Further references:

Couchbase Forums or StackOverflow
Follow us at @couchbasedev or @couchbase
Read more about Couchbase Server

Source: blog.couchbase.com/2016/september/minikube-rapid-dev–testing-kubernetes

Getting Started with Kubernetes 1.4 using Spring Boot and Couchbase

September 28, 2016containers, couchbase, javacontainers, couchbase, kubernetes, springarungupta

Kubernetes 1.4 was released earlier this week. Read the blog announcement and CHANGELOG. There are quite a few new features in this release but the key ones that I’m excited about are:

Install Kubernetes using kubeadm command. This is in addition to the usual mechanism of downloading from https://github.com/kubernetes/kubernetes/releases. The kubeadm init and kubeadm join commands looks very similar to docker swarm init and docker swarm join for Docker Swarm Mode.
Federated Replica Sets
ScheduledJob allows to run batch jobs at regular intervals.
Constraining pods to a node and affinity and anti-affinity of pods
Priority scheduling of pods
Nice looking Kubernetes Dashboard (more on this later)

This blog will show:

Create a Kubernetes cluster using Amazon Web Services
Create a Couchbase service
Run a Spring Boot application that stores a JSON document in Couchbase

All the resource description files in this blog are at github.com/arun-gupta/kubernetes-java-sample/tree/master/maven.

Start Kubernetes Cluster

Download binary github.com/kubernetes/kubernetes/releases/download/v1.4.0/kubernetes.tar.gz and extract

Include kubernetes/cluster in PATH

Start a 2-node Kubernetes cluster:

NUM_NODES=2 NODE_SIZE=m3.medium KUBERNETES_PROVIDER=aws kube-up.sh

NUM_NODES=2 NODE_SIZE=m3.medium KUBERNETES_PROVIDER=aws kube-up.sh

The log will be shown as:

... Starting cluster in us-west-2a using provider aws
... calling verify-prereqs
... calling kube-up
Starting cluster using os distro: jessie
Uploading to Amazon S3
+++ Staging server tars to S3 Storage: kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel
upload: ../../../../../var/folders/81/ttv4n16x7p390cttrm_675y00000gn/T/kubernetes.XXXXXX.bCmvLbtK/s3/bootstrap-script to s3://kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script
Uploaded server tars:
  SERVER_BINARY_TAR_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/kubernetes-server-linux-amd64.tar.gz
  SALT_TAR_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/kubernetes-salt.tar.gz
  BOOTSTRAP_SCRIPT_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script
INSTANCEPROFILE arn:aws:iam::598307997273:instance-profile/kubernetes-master    2016-07-29T15:13:35Z    AIPAJF3XKLNKOXOTQOCT4   kubernetes-master       /
ROLES   arn:aws:iam::598307997273:role/kubernetes-master        2016-07-29T15:13:33Z    /       AROAI3Q2KFBD5PCKRXCRM   kubernetes-master
ASSUMEROLEPOLICYDOCUMENT        2012-10-17
STATEMENT       sts:AssumeRole  Allow
PRINCIPAL       ec2.amazonaws.com
INSTANCEPROFILE arn:aws:iam::598307997273:instance-profile/kubernetes-minion    2016-07-29T15:13:39Z    AIPAIYSH5DJA4UPQIP4BE   kubernetes-minion       /
ROLES   arn:aws:iam::598307997273:role/kubernetes-minion        2016-07-29T15:13:37Z    /       AROAIQ57MPQYSHRPQCT2Q   kubernetes-minion
ASSUMEROLEPOLICYDOCUMENT        2012-10-17
STATEMENT       sts:AssumeRole  Allow
PRINCIPAL       ec2.amazonaws.com
Using SSH key with (AWS) fingerprint: SHA256:dX/5wpWuUxYar2NFuGwiZuRiydiZCyx4DGoZ5/jL/j8
Creating vpc.
Adding tag to vpc-6b5b4b0f: Name=kubernetes-vpc
Adding tag to vpc-6b5b4b0f: KubernetesCluster=kubernetes
Using VPC vpc-6b5b4b0f
Adding tag to dopt-8fe770eb: Name=kubernetes-dhcp-option-set
Adding tag to dopt-8fe770eb: KubernetesCluster=kubernetes
Using DHCP option set dopt-8fe770eb
Creating subnet.
Adding tag to subnet-623a0206: KubernetesCluster=kubernetes
Using subnet subnet-623a0206
Creating Internet Gateway.
Using Internet Gateway igw-251eab41
Associating route table.
Creating route table
Adding tag to rtb-d43cedb3: KubernetesCluster=kubernetes
Associating route table rtb-d43cedb3 to subnet subnet-623a0206
Adding route to route table rtb-d43cedb3
Using Route Table rtb-d43cedb3
Creating master security group.
Creating security group kubernetes-master-kubernetes.
Adding tag to sg-d20ca0ab: KubernetesCluster=kubernetes
Creating minion security group.
Creating security group kubernetes-minion-kubernetes.
Adding tag to sg-cd0ca0b4: KubernetesCluster=kubernetes
Using master security group: kubernetes-master-kubernetes sg-d20ca0ab
Using minion security group: kubernetes-minion-kubernetes sg-cd0ca0b4
Creating master disk: size 20GB, type gp2
Adding tag to vol-99a30b11: Name=kubernetes-master-pd
Adding tag to vol-99a30b11: KubernetesCluster=kubernetes
Allocated Elastic IP for master: 52.40.9.27
Adding tag to vol-99a30b11: kubernetes.io/master-ip=52.40.9.27
Generating certs for alternate-names: IP:52.40.9.27,IP:172.20.0.9,IP:10.0.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local,DNS:kubernetes-master
Starting Master
Adding tag to i-f95bdae1: Name=kubernetes-master
Adding tag to i-f95bdae1: Role=kubernetes-master
Adding tag to i-f95bdae1: KubernetesCluster=kubernetes
Waiting for master to be ready
Attempt 1 to check for master nodeWaiting for instance i-f95bdae1 to be running (currently pending)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be running (currently pending)
Sleeping for 3 seconds...
 [master running]
Attaching IP 52.40.9.27 to instance i-f95bdae1
Attaching persistent data volume (vol-99a30b11) to master
2016-09-29T05:14:28.098Z        /dev/sdb        i-f95bdae1      attaching       vol-99a30b11
cluster "aws_kubernetes" set.
user "aws_kubernetes" set.
context "aws_kubernetes" set.
switched to context "aws_kubernetes".
user "aws_kubernetes-basic-auth" set.
Wrote config for aws_kubernetes to /Users/arungupta/.kube/config
Creating minion configuration
Creating autoscaling group
 0 minions started; waiting
 0 minions started; waiting
 0 minions started; waiting
 0 minions started; waiting
 2 minions started; ready
Waiting for cluster initialization.

  This will continually check to see if the API for kubernetes is reachable.
  This might loop forever if there was some uncaught error during start
  up.

..............................................................................................................................................................................................................................Kubernetes cluster created.
Sanity checking cluster...
Attempt 1 to check Docker on node @ 54.70.225.33 ...working
Attempt 1 to check Docker on node @ 54.71.36.48 ...working

Kubernetes cluster is running.  The master is running at:

  https://52.40.9.27

The user name and password to use is located in /Users/arungupta/.kube/config.

... calling validate-cluster
Waiting for 2 ready nodes. 0 ready nodes, 0 registered. Retrying.
Waiting for 2 ready nodes. 0 ready nodes, 0 registered. Retrying.
Waiting for 2 ready nodes. 0 ready nodes, 0 registered. Retrying.
Waiting for 2 ready nodes. 0 ready nodes, 2 registered. Retrying.
Waiting for 2 ready nodes. 0 ready nodes, 2 registered. Retrying.
Found 2 node(s).
NAME                                         STATUS    AGE
ip-172-20-0-111.us-west-2.compute.internal   Ready     39s
ip-172-20-0-112.us-west-2.compute.internal   Ready     42s
Validate output:
NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok                   
controller-manager   Healthy   ok                   
etcd-0               Healthy   {"health": "true"}   
etcd-1               Healthy   {"health": "true"}   
Cluster validation succeeded
Done, listing cluster services:

Kubernetes master is running at https://52.40.9.27
Elasticsearch is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging
Heapster is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/heapster
Kibana is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kibana-logging
KubeDNS is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
Grafana is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana
InfluxDB is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

... Starting cluster in us-west-2a using provider aws

... calling verify-prereqs

... calling kube-up

Starting cluster using os distro: jessie

Uploading to Amazon S3

+++ Staging server tars to S3 Storage: kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel

upload: ../../../../../var/folders/81/ttv4n16x7p390cttrm_675y00000gn/T/kubernetes.XXXXXX.bCmvLbtK/s3/bootstrap-script to s3://kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script

Uploaded server tars:

SERVER_BINARY_TAR_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/kubernetes-server-linux-amd64.tar.gz

SALT_TAR_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/kubernetes-salt.tar.gz

BOOTSTRAP_SCRIPT_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script

INSTANCEPROFILE arn:aws:iam::598307997273:instance-profile/kubernetes-master 2016-07-29T15:13:35Z AIPAJF3XKLNKOXOTQOCT4 kubernetes-master /

ROLES arn:aws:iam::598307997273:role/kubernetes-master 2016-07-29T15:13:33Z / AROAI3Q2KFBD5PCKRXCRM kubernetes-master

ASSUMEROLEPOLICYDOCUMENT 2012-10-17

STATEMENT sts:AssumeRole Allow

PRINCIPAL ec2.amazonaws.com

INSTANCEPROFILE arn:aws:iam::598307997273:instance-profile/kubernetes-minion 2016-07-29T15:13:39Z AIPAIYSH5DJA4UPQIP4BE kubernetes-minion /

ROLES arn:aws:iam::598307997273:role/kubernetes-minion 2016-07-29T15:13:37Z / AROAIQ57MPQYSHRPQCT2Q kubernetes-minion

ASSUMEROLEPOLICYDOCUMENT 2012-10-17

STATEMENT sts:AssumeRole Allow

PRINCIPAL ec2.amazonaws.com

Using SSH key with (AWS) fingerprint: SHA256:dX/5wpWuUxYar2NFuGwiZuRiydiZCyx4DGoZ5/jL/j8

Creating vpc.

Adding tag to vpc-6b5b4b0f: Name=kubernetes-vpc

Adding tag to vpc-6b5b4b0f: KubernetesCluster=kubernetes

Using VPC vpc-6b5b4b0f

Adding tag to dopt-8fe770eb: Name=kubernetes-dhcp-option-set

Adding tag to dopt-8fe770eb: KubernetesCluster=kubernetes

Using DHCP option set dopt-8fe770eb

Creating subnet.

Adding tag to subnet-623a0206: KubernetesCluster=kubernetes

Using subnet subnet-623a0206

Creating Internet Gateway.

Using Internet Gateway igw-251eab41

Associating route table.

Creating route table

Adding tag to rtb-d43cedb3: KubernetesCluster=kubernetes

Associating route table rtb-d43cedb3 to subnet subnet-623a0206

Adding route to route table rtb-d43cedb3

Using Route Table rtb-d43cedb3

Creating master security group.

Creating security group kubernetes-master-kubernetes.

Adding tag to sg-d20ca0ab: KubernetesCluster=kubernetes

Creating minion security group.

Creating security group kubernetes-minion-kubernetes.

Adding tag to sg-cd0ca0b4: KubernetesCluster=kubernetes

Using master security group: kubernetes-master-kubernetes sg-d20ca0ab

Using minion security group: kubernetes-minion-kubernetes sg-cd0ca0b4

Creating master disk: size 20GB, type gp2

Adding tag to vol-99a30b11: Name=kubernetes-master-pd

Adding tag to vol-99a30b11: KubernetesCluster=kubernetes

Allocated Elastic IP for master: 52.40.9.27

Adding tag to vol-99a30b11: kubernetes.io/master-ip=52.40.9.27

Generating certs for alternate-names: IP:52.40.9.27,IP:172.20.0.9,IP:10.0.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local,DNS:kubernetes-master

Starting Master

Adding tag to i-f95bdae1: Name=kubernetes-master

Adding tag to i-f95bdae1: Role=kubernetes-master

Adding tag to i-f95bdae1: KubernetesCluster=kubernetes

Waiting for master to be ready

Attempt 1 to check for master nodeWaiting for instance i-f95bdae1 to be running (currently pending)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be running (currently pending)

Sleeping for 3 seconds...

[master running]

Attaching IP 52.40.9.27 to instance i-f95bdae1

Attaching persistent data volume (vol-99a30b11) to master

2016-09-29T05:14:28.098Z /dev/sdb i-f95bdae1 attaching vol-99a30b11

cluster "aws_kubernetes" set.

user "aws_kubernetes" set.

context "aws_kubernetes" set.

switched to context "aws_kubernetes".

user "aws_kubernetes-basic-auth" set.

Wrote config for aws_kubernetes to /Users/arungupta/.kube/config

Creating minion configuration

Creating autoscaling group

0 minions started; waiting

2 minions started; ready

Waiting for cluster initialization.

This will continually check to see if the API for kubernetes is reachable.

This might loop forever if there was some uncaught error during start

up.

..............................................................................................................................................................................................................................Kubernetes cluster created.

Sanity checking cluster...

Attempt 1 to check Docker on node @ 54.70.225.33 ...working

Attempt 1 to check Docker on node @ 54.71.36.48 ...working

Kubernetes cluster is running. The master is running at:

https://52.40.9.27

The user name and password to use is located in /Users/arungupta/.kube/config.

... calling validate-cluster

Waiting for 2 ready nodes. 0 ready nodes, 0 registered. Retrying.

Waiting for 2 ready nodes. 0 ready nodes, 2 registered. Retrying.

Found 2 node(s).

NAME STATUS AGE

ip-172-20-0-111.us-west-2.compute.internal Ready 39s

ip-172-20-0-112.us-west-2.compute.internal Ready 42s

Validate output:

NAME STATUS MESSAGE ERROR

scheduler Healthy ok

controller-manager Healthy ok

etcd-0 Healthy {"health": "true"}

etcd-1 Healthy {"health": "true"}

Cluster validation succeeded

Done, listing cluster services:

Kubernetes master is running at https://52.40.9.27

Elasticsearch is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging

Heapster is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/heapster

Kibana is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kibana-logging

KubeDNS is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kube-dns

kubernetes-dashboard is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

Grafana is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana

InfluxDB is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

This shows that the Kubernetes cluster has started successfully.

Deploy Couchbase Service

Create Couchbase service and replication controller:

kubectl.sh create -f couchbase-service.yml
service "couchbase-service" created
replicationcontroller "couchbase-rc" created

kubectl.sh create -f couchbase-service.yml

service "couchbase-service" created

replicationcontroller "couchbase-rc" created

The configuration file is at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/couchbase-service.yml.

This creates a Couchbase service and the backing replication controller. Name of the service is couchbase-service. This will be used later by the Spring Boot application to communicate with the database.

Check the status of pods:

kubectl.sh get -w pods
NAME                 READY     STATUS              RESTARTS   AGE
couchbase-rc-gu9gl   0/1       ContainerCreating   0          6s
NAME                 READY     STATUS    RESTARTS   AGE
couchbase-rc-gu9gl   1/1       Running   0          2m

kubectl.sh get -w pods

NAME READY STATUS RESTARTS AGE

couchbase-rc-gu9gl 0/1 ContainerCreating 0 6s

NAME READY STATUS RESTARTS AGE

couchbase-rc-gu9gl 1/1 Running 0 2m

Note, how the pod status changes from ContainerCreating to Running. The image is downloaded and started in the meanwhile.

Run Spring Boot Application

Run the application:

kubectl.sh create -f bootiful-couchbase.yml 
pod "bootiful-couchbase" created

kubectl.sh create -f bootiful-couchbase.yml

pod "bootiful-couchbase" created

The configuration file is at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/bootiful-couchbase.yml. In this service, COUCHBASE_URI environment variable value is set to couchbase-service. This is the service name created earlier.

Docker image used for this service is arungupta/bootiful-couchbase and is created using fabric8-maven-plugin as shown at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/webapp/pom.xml#L57-L68. Specifically, the command for the Docker image is:

java -Dspring.couchbase.bootstrap-hosts=$COUCHBASE_URI -jar /maven/${project.artifactId}.jar

java -Dspring.couchbase.bootstrap-hosts=$COUCHBASE_URI -jar /maven/${project.artifactId}.jar

This ensures that COUCHBASE_URI environment variable is overriding spring.couchbase.bootstrap-hosts property as defined in application.properties of the Spring Boot application.

Get the logs:

kubectl.sh logs -f bootiful-couchbase

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.4.0.RELEASE)

2016-09-29 05:37:29.227  INFO 5 --- [           main] org.example.webapp.Application           : Starting Application v1.0-SNAPSHOT on bootiful-couchbase with PID 5 (/maven/bootiful-couchbase.jar started by root in /)
2016-09-29 05:37:29.259  INFO 5 --- [           main] org.example.webapp.Application           : No active profile set, falling back to default profiles: default
2016-09-29 05:37:29.696  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@4ccabbaa: startup date [Thu Sep 29 05:37:29 UTC 2016]; root of context hierarchy
2016-09-29 05:37:34.375  INFO 5 --- [           main] c.c.client.core.env.CoreEnvironment      : ioPoolSize is less than 3 (1), setting to: 3
2016-09-29 05:37:34.376  INFO 5 --- [           main] c.c.client.core.env.CoreEnvironment      : computationPoolSize is less than 3 (1), setting to: 3
2016-09-29 05:37:35.026  INFO 5 --- [           main] com.couchbase.client.core.CouchbaseCore  : CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile='null', sslKeystorePassword='null', queryEnabled=false, queryPort=8093, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=3, computationPoolSize=3, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, searchServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.2.8 (git: 2.2.8, core: 1.2.9), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=1000, dcpConnectionBufferSize=20971520, dcpConnectionBufferAckThreshold=0.2, dcpConnectionName=dcp/core-io, callbacksOnIoPool=false, queryTimeout=7500, viewTimeout=7500, kvTimeout=2500, connectTimeout=5000, disconnectTimeout=25000, dnsSrvEnabled=false}
2016-09-29 05:37:36.063  INFO 5 --- [      cb-io-1-1] com.couchbase.client.core.node.Node      : Connected to Node couchbase-service
2016-09-29 05:37:36.256  INFO 5 --- [      cb-io-1-1] com.couchbase.client.core.node.Node      : Disconnected from Node couchbase-service
2016-09-29 05:37:37.727  INFO 5 --- [      cb-io-1-2] com.couchbase.client.core.node.Node      : Connected to Node couchbase-service
2016-09-29 05:37:38.316  INFO 5 --- [-computations-3] c.c.c.core.config.ConfigurationProvider  : Opened bucket books
2016-09-29 05:37:40.655  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
Book{isbn=978-1-4919-1889-0, name=Minecraft Modding with Forge, cost=29.99}
2016-09-29 05:37:41.497  INFO 5 --- [           main] org.example.webapp.Application           : Started Application in 14.64 seconds (JVM running for 16.631)
2016-09-29 05:37:41.514  INFO 5 --- [       Thread-5] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@4ccabbaa: startup date [Thu Sep 29 05:37:29 UTC 2016]; root of context hierarchy
2016-09-29 05:37:41.528  INFO 5 --- [       Thread-5] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown
2016-09-29 05:37:41.577  INFO 5 --- [      cb-io-1-2] com.couchbase.client.core.node.Node      : Disconnected from Node couchbase-service
2016-09-29 05:37:41.578  INFO 5 --- [       Thread-5] c.c.c.core.config.ConfigurationProvider  : Closed bucket books

kubectl.sh logs -f bootiful-couchbase

. ____ _ __ _ _

/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \

( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \

\\/ ___)| |_)| | | | | || (_| | ) ) ) )

' |____| .__|_| |_|_| |_\__, | / / / /

=========|_|==============|___/=/_/_/_/

:: Spring Boot :: (v1.4.0.RELEASE)

2016-09-29 05:37:29.227 INFO 5 --- [ main] org.example.webapp.Application : Starting Application v1.0-SNAPSHOT on bootiful-couchbase with PID 5 (/maven/bootiful-couchbase.jar started by root in /)

2016-09-29 05:37:29.259 INFO 5 --- [ main] org.example.webapp.Application : No active profile set, falling back to default profiles: default

2016-09-29 05:37:29.696 INFO 5 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@4ccabbaa: startup date [Thu Sep 29 05:37:29 UTC 2016]; root of context hierarchy

2016-09-29 05:37:34.375 INFO 5 --- [ main] c.c.client.core.env.CoreEnvironment : ioPoolSize is less than 3 (1), setting to: 3

2016-09-29 05:37:34.376 INFO 5 --- [ main] c.c.client.core.env.CoreEnvironment : computationPoolSize is less than 3 (1), setting to: 3

2016-09-29 05:37:35.026 INFO 5 --- [ main] com.couchbase.client.core.CouchbaseCore : CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile='null', sslKeystorePassword='null', queryEnabled=false, queryPort=8093, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=3, computationPoolSize=3, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, searchServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.2.8 (git: 2.2.8, core: 1.2.9), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=1000, dcpConnectionBufferSize=20971520, dcpConnectionBufferAckThreshold=0.2, dcpConnectionName=dcp/core-io, callbacksOnIoPool=false, queryTimeout=7500, viewTimeout=7500, kvTimeout=2500, connectTimeout=5000, disconnectTimeout=25000, dnsSrvEnabled=false}

2016-09-29 05:37:36.063 INFO 5 --- [ cb-io-1-1] com.couchbase.client.core.node.Node : Connected to Node couchbase-service

2016-09-29 05:37:36.256 INFO 5 --- [ cb-io-1-1] com.couchbase.client.core.node.Node : Disconnected from Node couchbase-service

2016-09-29 05:37:37.727 INFO 5 --- [ cb-io-1-2] com.couchbase.client.core.node.Node : Connected to Node couchbase-service

2016-09-29 05:37:38.316 INFO 5 --- [-computations-3] c.c.c.core.config.ConfigurationProvider : Opened bucket books

2016-09-29 05:37:40.655 INFO 5 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup

Book{isbn=978-1-4919-1889-0, name=Minecraft Modding with Forge, cost=29.99}

2016-09-29 05:37:41.497 INFO 5 --- [ main] org.example.webapp.Application : Started Application in 14.64 seconds (JVM running for 16.631)

2016-09-29 05:37:41.514 INFO 5 --- [ Thread-5] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@4ccabbaa: startup date [Thu Sep 29 05:37:29 UTC 2016]; root of context hierarchy

2016-09-29 05:37:41.528 INFO 5 --- [ Thread-5] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown

2016-09-29 05:37:41.577 INFO 5 --- [ cb-io-1-2] com.couchbase.client.core.node.Node : Disconnected from Node couchbase-service

2016-09-29 05:37:41.578 INFO 5 --- [ Thread-5] c.c.c.core.config.ConfigurationProvider : Closed bucket books

The main output statement to look in this is

Book{isbn=978-1-4919-1889-0, name=Minecraft Modding with Forge, cost=29.99}

Book{isbn=978-1-4919-1889-0, name=Minecraft Modding with Forge, cost=29.99}

This indicates that the JSON document is upserted (either inserted or updated) in the Couchbase database.

Kubernetes Dashboard

Kubernetes Dashboard is look more comprehensive and claimed to have 90% parity with the CLI. Use kubectl.sh config view command to view the configuration information about the cluster. It looks like:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: REDACTED
    server: https://52.40.9.27
  name: aws_kubernetes
contexts:
- context:
    cluster: aws_kubernetes
    user: aws_kubernetes
  name: aws_kubernetes
current-context: aws_kubernetes
kind: Config
preferences: {}
users:
- name: aws_kubernetes
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
    token: 3GuTCLvFnINHed9dWICICidlrSv8C0kg
- name: aws_kubernetes-basic-auth
  user:
    password: 8pxC121Oj7kN0nCa
    username: admin

apiVersion: v1

clusters:

- cluster:

certificate-authority-data: REDACTED

server: https://52.40.9.27

contexts:

- context:

cluster: aws_kubernetes

user: aws_kubernetes

current-context: aws_kubernetes

kind: Config

preferences: {}

users:

- name: aws_kubernetes

user:

client-certificate-data: REDACTED

client-key-data: REDACTED

token: 3GuTCLvFnINHed9dWICICidlrSv8C0kg

- name: aws_kubernetes-basic-auth

user:

password: 8pxC121Oj7kN0nCa

username: admin

The clusters.cluster.server property value shows the location of Kubernetes master. The users property show two users that can be used to access the dashboard. Second one uses basic authentication and so copy the username and password property value. In our case, Dashboard UI is accessible at https://52.40.9.27/ui.

All the Kubernetes resources can be easily seen in this fancy dashboard.

Shutdown Kubernetes Cluster

Finally, shutdown the Kubernetes cluster:

kube-down.sh
Bringing down cluster using provider: aws
Deleting instances in VPC: vpc-6b5b4b0f
Deleting auto-scaling group: kubernetes-minion-group-us-west-2a
Deleting auto-scaling launch configuration: kubernetes-minion-group-us-west-2a
Deleting auto-scaling group: kubernetes-minion-group-us-west-2a
Waiting for instances to be deleted
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
All instances deleted
Releasing Elastic IP: 52.40.9.27
Deleting volume vol-99a30b11
Cleaning up resources in VPC: vpc-6b5b4b0f
Cleaning up security group: sg-cd0ca0b4
Cleaning up security group: sg-d20ca0ab
Deleting security group: sg-cd0ca0b4
Deleting security group: sg-d20ca0ab
Deleting VPC: vpc-6b5b4b0f
Done

kube-down.sh

Bringing down cluster using provider: aws

Deleting instances in VPC: vpc-6b5b4b0f

Deleting auto-scaling group: kubernetes-minion-group-us-west-2a

Deleting auto-scaling launch configuration: kubernetes-minion-group-us-west-2a

Deleting auto-scaling group: kubernetes-minion-group-us-west-2a

Waiting for instances to be deleted

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)

Sleeping for 3 seconds...

All instances deleted

Releasing Elastic IP: 52.40.9.27

Deleting volume vol-99a30b11

Cleaning up resources in VPC: vpc-6b5b4b0f

Cleaning up security group: sg-cd0ca0b4

Cleaning up security group: sg-d20ca0ab

Deleting security group: sg-cd0ca0b4

Deleting security group: sg-d20ca0ab

Deleting VPC: vpc-6b5b4b0f

Done

couchbase.com/containers provide more details about running Couchbase using different orchestration frameworks.

Further references:

Couchbase Forums or StackOverflow
Follow us at @couchbasedev or @couchbase
Read more about Couchbase Server

Source: blog.couchbase.com/2016/september/kubernetes-1.4-spring-boot-couchbase

Deployment Pipeline using Docker, Jenkins, Java and Couchbase

September 9, 2016containers, couchbase, javacontainers, couchbase, docker, java, jenkinsarungupta

This blog explains how to create a Deployment Pipeline using Jenkins and Docker for a Java application talking to a database.

Jenkins support the creation of pipelines. They are built with simple text scripts that use a Pipeline DSL (domain-specific language) based on the Groovy programming language.

The script, typically called Jenkinsfile, defines multiple steps to execute both simple and complex tasks according to the parameters that you establish. Once created, pipelines can build code and orchestrate the work required to drive applications from commit to delivery.

A pipeline consists of steps, node and stage. A pipeline is executed on a node – a computer that is part of Jenkins installation. A pipeline often consists of multiple stages. A stage consists of multiple steps. Read Getting Started with Pipeline for more details.

For our application, here is the basic flow:

Complete source code for the application used is at github.com/arun-gupta/docker-jenkins-pipeline.

The application is defined in the webapp directory. It opens a connection to the Couchbase database and stores a simple JSON document using Couchbase Java SDK. The application also has a test that verifies that the database indeed contains the document that was persisted.

Many thanks to @alexsotob for helping me with Jenkins configuration.

Let’s get started!

Download and Install Jenkins

Download Jenkins from jenkins.io. This was tested with Jenkins 2.21.

Start Jenkins:

JENKINS_HOME=~/.jenkins java -jar ~/Downloads/jenkins-2.21.war --httpPort=9090

JENKINS_HOME=~/.jenkins java -jar ~/Downloads/jenkins-2.21.war --httpPort=9090

This command starts Jenkins by specifying the home directory where all the configuration information is stored. It also defines the port on which Jenkins is listening, 9090 in this case.

First start of Jenkins shows the following message in the console:

*************************************************************
*************************************************************
*************************************************************

Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:

3521fbc3d40448efa8942f8e464b2dd9

This may also be found at: /Users/arungupta/.jenkins/secrets/initialAdminPassword

*************************************************************
*************************************************************
*************************************************************

*************************************************************

Jenkins initial setup is required. An admin user has been created and a password generated.

Please use the following password to proceed to installation:

3521fbc3d40448efa8942f8e464b2dd9

This may also be found at: /Users/arungupta/.jenkins/secrets/initialAdminPassword

*************************************************************

Copy the password shown here. This will be used to unlock Jenkins.

Access the Jenkins console at localhost:9090 and paste the password:Click on Next.
Create the first admin user as shown:

Click on Save and Finish.
Click on Install suggested plugins:
A bunch of default plugins are installed:
Found it surprising that Ant and Subversion are the default plugins.
Login screen is prompted.

Enter the username and password specified earlier.
Finally, Jenkins is ready to use:

That’s quite a bit of steps to get started with basic Jenkins. Do I really have to jump through all these hoops to get started with Jenkins? Is there an easier, simpler, dumber, lazier way to start Jenkins? Follow Convention-over-Configuration and give me one-click pre-configured installation.

Install Jenkins Plugins

Install the required plugins in Jenkins.

If your Java project is built using Maven, then you need to configure Maven in Jenkins. Click on Manage Jenkins, Global Tool Configuration, Maven installations, and specify the location of Maven.
Name the tool as Maven3 as that is the name used in the configuration later.Again a bit lame, why can’t Jenkins pick up the default location of Maven instead of expecting the user to specify a location.
Click on Manage Jenkins, Manage Plugins, Available tab, search for docker pipe. Select CloudBees Docker Pipeline, click on Install without restart.

Click on Install without restart.Docker Pipeline Plugin plugin understands the Jenkinsfile and executes the commands listed there.
Next screen shows the list of plugins that are installed:
The last line shows that CloudBees Docker Pipeline plugin is installed successfully. Select Restart Jenkins checkbox. This will install restart Jenkins as well.

Create Jenkins Job

Let’s create a job in Jenkins that will run the pipeline.

After Jenkins restarts, it shows the login screen. Enter the username and password created earlier. This brings you back to Installing Plugins/Upgrades page. Click on the Jenkins icon in the top left corner to see the main dashboard:
Click on create new jobs, give the name as docker-jenkins-pipeline and choose the type as Pipeline:Click on OK.
Configure Pipeline as shown:
Local git repo is used in this case. You can certainly choose a repo hosted on github. Further, this repo can be configured with a git hook or poll at a constant interval to trigger the pipeline.Click on Save to save the configuration.

Run Jenkins Build

Before you start the job, Couchbase database need to be explicitly started as:

docker run -d --name db -p 8091-8093:8091-8093 -p 11210:11210 arungupta/oreilly-couchbase:latest

docker run -d --name db -p 8091-8093:8091-8093 -p 11210:11210 arungupta/oreilly-couchbase:latest

This will be resolved after #9 is fixed. Make sure you can access Couchbase at http://localhost:8091, use Administrator as the login and password as the password. Click on Data Buckets tab and see the books bucket created.

Click on Build Now and you should see an output similar to:

All green is good!

Let’s try to understand what happened behind the scene.

Jenkinsfile describes how the pipeline is built. At the top level, it has four stages – Package, Create Docker Image, Run Application and Run Tests. Each stage is shown as a box in Jenkins dashboard. Total time taken for each stage is shown in the box.

Let’s understand what happens in each stage.

Package – Application source code lives in the webapp directory. Maven command mvn clean package -DskipTests is used to create a JAR file of the application. Note that the maven project also includes the tests and are explicitly skipped using -DskipTests. Typically, tests would be in a separate downstream project.Maven project creates a far JAR file of the application and includes all the dependencies.
Create Docker Image – Docker image of the application is built using the Dockerfile in the webapp directory. The image simply includes the fat JAR and runs it using java -jar.Each image is tagged with the build number using ${env.BUILD_NUMBER}.
Run Application – Running the application involves running the application Docker container.IP address of the database container is identified using the docker inspect command.The database container and the application container are both running in the default bridge network. This allows the two containers to communicate with each other. Another enhancement would be to run the pipeline in a swarm mode cluster. This would require to create and use an overlay network.
Run Tests – Tests are run against the container using the mvn test command. If the tests pass the image is pushed to Docker Hub. The test results are captured either way.This stage also shows the usage of try/catch/finally block in Jenkinsfile.If the tests pass then the image is pushed to Docker Hub. In this case, it is available at hub.docker.com/r/arungupta/docker-jenkins-pipeline/tags/.

Some TODOs …

Move the tests to a downstream project (#7)
Use Git hook or poll to trigger pipeline (#8)
Automate database startup/shutdown (#9)
Run pipeline in a cluster of Docker Engines with Swarm mode (#10)
Show alternate configuration to push image to bintray (#11)

Another pain point is that global variables syntax does not seem to be documented anywhere. It is only available at <JENKINS-HOST>:<JENKINS-PORT>/job/docker-jenkins-pipeline/pipeline-syntax/globals. This is again slightly lame!

“not impossible, just not implemented yet” #sadpanda

Some further references to read:

Getting Started with the Jenkinsfile
CloudBees Docker Pipeline Plugin
CloudBees Docker Pipeline Plugin User Guide
Jenkinsfile DSL Reference
Jenkins Pipeline Talk from JavaZone 2016

More information about Couchbase:

Couchbase Developer Portal
Couchbase Forums
@couchbasedev or @couchbase

Feel free to file bugs at github.com/arun-gupta/docker-jenkins-pipeline/issues or send PR.

Source: blog.couchbase.com/2016/september/deployment-pipeline-docker-jenkins-java-couchbase

Docker Services, Stack and Distributed Application Bundle

July 1, 2016containers, couchbase, javaee, wildflycouchbase, docker, javaee, wildflyarungupta

First Release Candidate of Docker 1.12 was announced over two weeks ago. Several new features are planned for this release.

This blog will show how to create a Distributed Application Bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode. Many thanks to @friism to help me understand these concepts.

Let’s look at the features first:

Built-in orchestration: A typical application is defined using a Docker Compose file. This definition consists of multiple containers and deployed on multiple hosts. This avoids Single Point of Failure (SPOF) and keeps your application resilient. Multiple orchestration frameworks such as Docker Swarm, Kubernetes and Mesos allow you to orchestrate these applications. However it is such an important characteristic of the application, Docker Engine now has built-in orchestration. More details on this topic in a later blog.
Service: A replicated, distributed and load balanced service can be easily created using docker service create command. A “desired state” of the application, such as run 3 containers of Couchbase, is provided and the self-healing Docker engine ensures that that many containers are running in the cluster. If a container goes down, another container is started. If a node goes down, containers on that node are started on a different node. More on this in a later blog.
Zero-configuration Security: Docker 1.12 comes with mutually authenticated TLS, providing authentication, authorization and encryption to the communications of every node participating in the swarm, out of the box. More on this in a later blog.
Docker Stack and Distributed Application Bundle: Distributed Application Bundle, or DAB, is a multi-services distributable image format. Read further for more details.

So far, you can take a Dockerfile and create an image from it using the docker build command. A container can be started using the docker run command. Multiple containers can be easily started by giving that command multiple times. Or you can also use Docker Compose file and scale up your containers using the docker-compose scale command.

Image is a portable format for a single container. Distributed Application Bundle, or DAB, is a new concept introduced in Docker 1.12, is a portable format for multiple containers. Each bundle can be then deployed as a Stack at runtime.

Learn more about DAB at docker.com/dab.

For simplicity, here is an analogy that can be drawn:

Dockerfile -> Image -> Container

Docker Compose -> Distributed Application Bundle -> Docker Stack

Let’s use a Docker Compose file, create a DAB from it, and deploy it as a Docker Stack.

Its important to note that this is an experimental feature in 1.12-RC2.

Create a Distributed Application Bundle from Docker Compose

Docker Compose CLI adds a new bundle command. More details can be found:

docker-compose bundle --help
Generate a Docker bundle from the Compose file.

Local images will be pushed to a Docker registry, and remote images
will be pulled to fetch an image digest.

Usage: bundle [options]

Options:
    -o, --output PATH          Path to write the bundle file to.
                               Defaults to "<project name>.dsb".

docker-compose bundle --help

Generate a Docker bundle from the Compose file.

Local images will be pushed to a Docker registry, and remote images

will be pulled to fetch an image digest.

Usage: bundle [options]

Options:

-o, --output PATH Path to write the bundle file to.

Defaults to "<project name>.dsb".

Now, let’s take a Docker Compose definition and create a DAB from it. Here is our Docker Compose definition:

version: "2"
services:
  db:
    container_name: "db"
    image: arungupta/oreilly-couchbase:latest
    ports:
      - 8091:8091
      - 8092:8092 
      - 8093:8093 
      - 11210:11210
  web:
    image: arungupta/oreilly-wildfly:latest
    depends_on:
      - db
    environment:
      - COUCHBASE_URI=db
    ports:
      - 8080:8080

version: "2"

services:

db:

container_name: "db"

image: arungupta/oreilly-couchbase:latest

ports:

- 8091:8091

- 8092:8092

- 8093:8093

- 11210:11210

web:

image: arungupta/oreilly-wildfly:latest

depends_on:

- db

environment:

- COUCHBASE_URI=db

ports:

- 8080:8080

This Compose file starts a WildFly and a Couchbase server. A Java EE application is pre-deployed in the WildFly server that connects to the Couchbase server and allows to perform CRUD operations using the REST API.

The source for this file is at: github.com/arun-gupta/oreilly-docker-book/blob/master/hello-javaee/docker-compose.yml.

Generate an application bundle with it:

docker-compose bundle
WARNING: Unsupported key 'depends_on' in services.web - ignoring
WARNING: Unsupported key 'container_name' in services.db - ignoring
Wrote bundle to hellojavaee.dsb

docker-compose bundle

WARNING: Unsupported key 'depends_on' in services.web - ignoring

WARNING: Unsupported key 'container_name' in services.db - ignoring

Wrote bundle to hellojavaee.dsb

depends_on only creates dependency between two services and makes them start in a specific order. This only ensures that the Docker container is started but the application within the container may take longer to start. So this attribute only partially solves the problem. container_name gives a specific name to the container. Relying upon a specific container name is tight coupling and does not allow to scale the container. So both the warnings can be ignored, for now.

This command generates a file using the Compose project name, which is the directory name. So in our case, hellojavaee.dsb file is generated. This file extension has been renamed to .dab in RC3.

The generated application bundle looks like:

{
  "services": {
    "db": {
      "Image": "arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c", 
      "Networks": [
        "default"
      ], 
      "Ports": [
        {
          "Port": 8091, 
          "Protocol": "tcp"
        }, 
        {
          "Port": 8092, 
          "Protocol": "tcp"
        }, 
        {
          "Port": 8093, 
          "Protocol": "tcp"
        }, 
        {
          "Port": 11210, 
          "Protocol": "tcp"
        }
      ]
    }, 
    "web": {
      "Env": [
        "COUCHBASE_URI=db"
      ], 
      "Image": "arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914", 
      "Networks": [
        "default"
      ], 
      "Ports": [
        {
          "Port": 8080, 
          "Protocol": "tcp"
        }
      ]
    }
  }, 
  "version": "0.1"
}

{

"services": {

"db": {

"Image": "arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c",

"Networks": [

"default"

"Ports": [

{

"Port": 8091,

"Protocol": "tcp"

{

"Port": 8092,

"Protocol": "tcp"

{

"Port": 8093,

"Protocol": "tcp"

{

"Port": 11210,

"Protocol": "tcp"

}

]

"web": {

"Env": [

"COUCHBASE_URI=db"

"Image": "arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914",

"Networks": [

"default"

"Ports": [

{

"Port": 8080,

"Protocol": "tcp"

}

]

}

"version": "0.1"

}

This file provides complete description of the services included in the application. I’m not entirely sure if Distributed Application Bundle is the most appropriate name, discuss this in #24250. It would be great if other container formats, such as Rkt, or even VMs can be supported here. But for now, Docker is the only supported format.

Initialize Swarm Mode in Docker

As mentioned above, “desired state” is now maintained by Docker Swarm. And this is now baked into Docker Engine already.

Docker Swarm concepts have evolved as well and can be read at Swarm mode key concepts. A more detailed blog on this will be coming later.

But for this blog, a new command docker swarm is now added:

docker swarm --help

Usage:  docker swarm COMMAND

Manage Docker Swarm

Options:
      --help   Print usage

Commands:
  init        Initialize a Swarm
  join        Join a Swarm as a node and/or manager
  update      Update the Swarm
  leave       Leave a Swarm
  inspect     Inspect the Swarm

Run 'docker swarm COMMAND --help' for more information on a command.

docker swarm --help

Usage: docker swarm COMMAND

Manage Docker Swarm

Options:

--help Print usage

Commands:

init Initialize a Swarm

join Join a Swarm as a node and/or manager

update Update the Swarm

leave Leave a Swarm

inspect Inspect the Swarm

Run 'docker swarm COMMAND --help' for more information on a command.

Initialize a Swarm node (as a manager) in the Docker Engine:

docker swarm init
Swarm initialized: current node (ek9p1k8r8ox7iiua5c247skci) is now a manager.

docker swarm init

Swarm initialized: current node (ek9p1k8r8ox7iiua5c247skci) is now a manager.

More details about this node can be found using docker node inspect self command.

The detailed output is verbose but the relevant section is:

"Spec": {
            "Role": "manager",
            "Membership": "accepted",
            "Availability": "active"
        },

"Spec": {

"Role": "manager",

"Membership": "accepted",

"Availability": "active"

The output shows that the node is a manager. For a single-node cluster, this node will also act as a worker.

More details about the cluster can be obtained using the docker swarm inspect command.

docker swarm inspect
[
    {
        "ID": "1rcvu7m9mv2c8hiaijr7an9zk",
        "Version": {
            "Index": 1895
        },
        "CreatedAt": "2016-07-01T23:52:38.074748177Z",
        "UpdatedAt": "2016-07-02T04:54:32.79093117Z",
        "Spec": {
            "Name": "default",
            "AcceptancePolicy": {
                "Policies": [
                    {
                        "Role": "worker",
                        "Autoaccept": true
                    },
                    {
                        "Role": "manager",
                        "Autoaccept": false
                    }
                ]
            },
            "Orchestration": {
                "TaskHistoryRetentionLimit": 10
            },
            "Raft": {
                "SnapshotInterval": 10000,
                "LogEntriesForSlowFollowers": 500,
                "HeartbeatTick": 1,
                "ElectionTick": 3
            },
            "Dispatcher": {
                "HeartbeatPeriod": 5000000000
            },
            "CAConfig": {
                "NodeCertExpiry": 7776000000000000
            }
        }
    }
]

docker swarm inspect

[

{

"ID": "1rcvu7m9mv2c8hiaijr7an9zk",

"Version": {

"Index": 1895

"CreatedAt": "2016-07-01T23:52:38.074748177Z",

"UpdatedAt": "2016-07-02T04:54:32.79093117Z",

"Spec": {

"Name": "default",

"AcceptancePolicy": {

"Policies": [

{

"Role": "worker",

"Autoaccept": true

{

"Role": "manager",

"Autoaccept": false

}

]

"Orchestration": {

"TaskHistoryRetentionLimit": 10

"Raft": {

"SnapshotInterval": 10000,

"LogEntriesForSlowFollowers": 500,

"HeartbeatTick": 1,

"ElectionTick": 3

"Dispatcher": {

"HeartbeatPeriod": 5000000000

"CAConfig": {

"NodeCertExpiry": 7776000000000000

}

]

AcceptancePolicy shows that other worker nodes can join this cluster, but a manager requires explicit approval.

Deploy a Docker Stack

Create a stack using docker deploy command:

docker deploy -f hellojavaee.dsb hellojavaee
Loading bundle from hellojavaee.dsb
Creating network hellojavaee_default
Creating service hellojavaee_db
Creating service hellojavaee_web

docker deploy -f hellojavaee.dsb hellojavaee

Loading bundle from hellojavaee.dsb

Creating network hellojavaee_default

Creating service hellojavaee_db

Creating service hellojavaee_web

The command usage can certainly be simplified as discussed in #24249.

See the list of services:

docker service ls
ID            NAME             REPLICAS  IMAGE                                                                                                COMMAND
2g8kmrimztes  hellojavaee_web  1/1       arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914    
46xhlb15cc60  hellojavaee_db   1/1       arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c

docker service ls

ID NAME REPLICAS IMAGE COMMAND

2g8kmrimztes hellojavaee_web 1/1 arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914

46xhlb15cc60 hellojavaee_db 1/1 arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c

The output shows that two services, WildFly and Couchbase, are running. Services is also a new concept introduced in Docker 1.12. There is what gives you the “desired state” and Docker Engine works to give you that.

docker ps shows the list of containers running:

CONTAINER ID        IMAGE                                                                                                 COMMAND                  CREATED             STATUS              PORTS                                                        NAMES
622756277f40        arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c   "/entrypoint.sh /opt/"   3 seconds ago       Up 1 seconds        8091-8093/tcp, 11207/tcp, 11210-11211/tcp, 18091-18092/tcp   hellojavaee_db.1.19enwdt6i5m853m5675tx3z29
abf8703ed713        arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914     "/opt/jboss/wildfly/b"   3 seconds ago       Up 1 seconds        8080/tcp                                                     hellojavaee_web.1.70piloz6j4zt06co8htzisgyl

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

622756277f40 arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c "/entrypoint.sh /opt/" 3 seconds ago Up 1 seconds 8091-8093/tcp, 11207/tcp, 11210-11211/tcp, 18091-18092/tcp hellojavaee_db.1.19enwdt6i5m853m5675tx3z29

abf8703ed713 arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914 "/opt/jboss/wildfly/b" 3 seconds ago Up 1 seconds 8080/tcp hellojavaee_web.1.70piloz6j4zt06co8htzisgyl

WildFly container starts up before the Couchbase container is up and running. This means the Java EE application tries to connect to the Couchbase server and fails. So the application never boots successfully.

Self-healing Docker Service

Docker Service maintains the “desired state” of an application. In our case, the desired state is to ensure that one, and only one, container for the service is running. If we remove the container, not the service, then the service will automatically start the container again.

Remove the container as:

docker rm -f abf8703ed713

docker rm -f abf8703ed713

Note, you’ve to give -f because the container is already running. Docker 1.12 self-healing mechanisms kick in and automatically restart the container. Now if you list the containers again:

CONTAINER ID        IMAGE                                                                                                 COMMAND                  CREATED             STATUS                  PORTS                                                        NAMES
db483ac27e41        arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914     "/opt/jboss/wildfly/b"   1 seconds ago       Up Less than a second   8080/tcp                                                     hellojavaee_web.1.ddvwdmojjysf46d4n3x4g8uv4
622756277f40        arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c   "/entrypoint.sh /opt/"   26 seconds ago      Up 25 seconds           8091-8093/tcp, 11207/tcp, 11210-11211/tcp, 18091-18092/tcp   hellojavaee_db.1.19enwdt6i5m853m5675tx3z29

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

db483ac27e41 arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914 "/opt/jboss/wildfly/b" 1 seconds ago Up Less than a second 8080/tcp hellojavaee_web.1.ddvwdmojjysf46d4n3x4g8uv4

622756277f40 arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c "/entrypoint.sh /opt/" 26 seconds ago Up 25 seconds 8091-8093/tcp, 11207/tcp, 11210-11211/tcp, 18091-18092/tcp hellojavaee_db.1.19enwdt6i5m853m5675tx3z29

This shows that a new container has been started.

Inspect the WildFly service:

docker service inspect hellojavaee_web
[
    {
        "ID": "54otfi6dc9bis7z6gc6ubynwc",
        "Version": {
            "Index": 328
        },
        "CreatedAt": "2016-07-02T01:36:35.735767569Z",
        "UpdatedAt": "2016-07-02T01:36:35.739240775Z",
        "Spec": {
            "Name": "hellojavaee_web",
            "Labels": {
                "com.docker.stack.namespace": "hellojavaee"
            },
            "TaskTemplate": {
                "ContainerSpec": {
                    "Image": "arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914",
                    "Env": [
                        "COUCHBASE_URI=db"
                    ]
                }
            },
            "Mode": {
                "Replicated": {
                    "Replicas": 1
                }
            },
            "Networks": [
                {
                    "Target": "epw57lz7txtfchmbf6u0cimis",
                    "Aliases": [
                        "web"
                    ]
                }
            ],
            "EndpointSpec": {
                "Mode": "vip",
                "Ports": [
                    {
                        "Protocol": "tcp",
                        "TargetPort": 8080
                    }
                ]
            }
        },
        "Endpoint": {
            "Spec": {},
            "Ports": [
                {
                    "Protocol": "tcp",
                    "TargetPort": 8080,
                    "PublishedPort": 30004
                }
            ],
            "VirtualIPs": [
                {
                    "NetworkID": "9lpz688ir3pzexubkcb828ikg",
                    "Addr": "10.255.0.5/16"
                },
                {
                    "NetworkID": "epw57lz7txtfchmbf6u0cimis",
                    "Addr": "10.0.0.4/24"
                }
            ]
        }
    }
]

docker service inspect hellojavaee_web

[

{

"ID": "54otfi6dc9bis7z6gc6ubynwc",

"Version": {

"Index": 328

"CreatedAt": "2016-07-02T01:36:35.735767569Z",

"UpdatedAt": "2016-07-02T01:36:35.739240775Z",

"Spec": {

"Name": "hellojavaee_web",

"Labels": {

"com.docker.stack.namespace": "hellojavaee"

"TaskTemplate": {

"ContainerSpec": {

"Image": "arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914",

"Env": [

"COUCHBASE_URI=db"

]

}

"Mode": {

"Replicated": {

"Replicas": 1

}

"Networks": [

{

"Target": "epw57lz7txtfchmbf6u0cimis",

"Aliases": [

"web"

]

}

"EndpointSpec": {

"Mode": "vip",

"Ports": [

{

"Protocol": "tcp",

"TargetPort": 8080

}

]

}

"Endpoint": {

"Spec": {},

"Ports": [

{

"Protocol": "tcp",

"TargetPort": 8080,

"PublishedPort": 30004

}

"VirtualIPs": [

{

"NetworkID": "9lpz688ir3pzexubkcb828ikg",

"Addr": "10.255.0.5/16"

{

"NetworkID": "epw57lz7txtfchmbf6u0cimis",

"Addr": "10.0.0.4/24"

}

]

}

]

Swarm assigns a random port to the service, or this can be manually updated using docker service update command. In our case, port 8080 of the container is mapped to 30004 port on the host.

Verify the Application

Check that the application is successfully deployed:

curl http://localhost:30004/books/resources/book
[{"books":0}]

curl http://localhost:30004/books/resources/book

[{"books":0}]

Add a new book to the application:

curl -v \
> -H "Content-Type: application/json" \
> -X POST -d '{
>   "isbn": "978-1-4919-1889-0",
>   "name": "Minecraft Modding with Forge",
>   "cost": 29.99
> }' \
> http://localhost:30004/books/resources/book
*   Trying ::1...
* Connected to localhost (::1) port 30004 (#0)
> POST /books/resources/book HTTP/1.1
> Host: localhost:30004
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 92
> 
* upload completely sent off: 92 out of 92 bytes
< HTTP/1.1 200 OK
< Connection: keep-alive
< X-Powered-By: Undertow/1
< Server: WildFly/10
< Content-Type: application/octet-stream
< Content-Length: 88
< Date: Sat, 02 Jul 2016 01:39:49 GMT
< 
* Connection #0 to host localhost left intact
{"name":"Minecraft Mhttp://localhost:30004/books/resources/book-1-4919-1889-0"}

curl -v \

> -H "Content-Type: application/json" \

> -X POST -d '{

> "isbn": "978-1-4919-1889-0",

> "name": "Minecraft Modding with Forge",

> "cost": 29.99

> }' \

> http://localhost:30004/books/resources/book

* Trying ::1...

* Connected to localhost (::1) port 30004 (#0)

> POST /books/resources/book HTTP/1.1

> Host: localhost:30004

> User-Agent: curl/7.43.0

> Accept: */*

> Content-Type: application/json

> Content-Length: 92

* upload completely sent off: 92 out of 92 bytes

< HTTP/1.1 200 OK

< Connection: keep-alive

< X-Powered-By: Undertow/1

< Server: WildFly/10

< Content-Type: application/octet-stream

< Content-Length: 88

< Date: Sat, 02 Jul 2016 01:39:49 GMT

* Connection #0 to host localhost left intact

{"name":"Minecraft Mhttp://localhost:30004/books/resources/book-1-4919-1889-0"}

Verify the books again:

curl http://localhost:30004/books/resources/book
[{"books":{"name":"Minecraft Modding with Forge","cost":29.99,"id":"1","isbn":"978-1-4919-1889-0"}}, {"books":1}]

curl http://localhost:30004/books/resources/book

[{"books":{"name":"Minecraft Modding with Forge","cost":29.99,"id":"1","isbn":"978-1-4919-1889-0"}}, {"books":1}]

Learn more about this Java EE application at github.com/arun-gupta/oreilly-docker-book/tree/master/hello-javaee.

This blog showed how to create a Distributed Application Bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode.

Docker Service and Stack References

Docker Service Create
FREE book from O’Reilly: Docker for Java Developers
Couchbase on Containers
Couchbase Developer Portal
Ask questions on @couchbasedev or Stackoverflow

Create a Distributed Application Bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode.… Click To Tweet

Source: blog.couchbase.com/2016/july/docker-services-stack-distributed-application-bundle

Docker 1.10, Machine 0.6.0, Compose 1.6.0 – better volumes and networking

February 4, 2016containers, couchbase, javacouchbase, dockerarungupta

Docker 1.10 is now released!

Read about all the new features in Docker 1.10. A quick summary:

New Compose file format
Much better networking
Much better security
Swarm becomes 1.1, with Mesos integration

Read Docker 1.10 release notes.

Lets look at some of the key components.

Docker Machine 0.6.0

Docker Machine makes it really easy to create Docker hosts on your computer, on cloud providers and inside your own data center. It creates servers, installs Docker on them, then configures the Docker client to talk to them.

Latest version can be installed as:

~> curl -L https://github.com/docker/machine/releases/download/v0.6.0/docker-machine-`uname -s`-`uname -m` >/usr/local/bin/docker-machine && \
~>   chmod +x /usr/local/bin/docker-machine
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   601    0   601    0     0    362      0 --:--:--  0:00:01 --:--:--   362
100 36.6M  100 36.6M    0     0  2822k      0  0:00:13  0:00:13 --:--:-- 4899k
~ > docker-machine version
docker-machine version 0.6.0, build e27fb87

~> curl -L https://github.com/docker/machine/releases/download/v0.6.0/docker-machine-`uname -s`-`uname -m` >/usr/local/bin/docker-machine && \

~> chmod +x /usr/local/bin/docker-machine

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

100 601 0 601 0 0 362 0 --:--:-- 0:00:01 --:--:-- 362

100 36.6M 100 36.6M 0 0 2822k 0 0:00:13 0:00:13 --:--:-- 4899k

~ > docker-machine version

docker-machine version 0.6.0, build e27fb87

docker-machine now shows the Docker server version:

~ > docker-machine ls
NAME        ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
couchbase   -        virtualbox   Running   tcp://192.168.99.100:2376           v1.9.1    
netbeans    -        virtualbox   Saved                                         Unknown

~ > docker-machine ls

NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS

couchbase - virtualbox Running tcp://192.168.99.100:2376 v1.9.1

netbeans - virtualbox Saved Unknown

The latest server version is 1.10. And so docker upgrade command can be used to fix that:

~ > docker-machine upgrade couchbase
Waiting for SSH to be available...
Detecting the provisioner...
Upgrading docker...
Stopping machine to do the upgrade...
Upgrading machine "couchbase"...
Default Boot2Docker ISO is out-of-date, downloading the latest release...
Latest release for github.com/boot2docker/boot2docker is v1.10.0
Downloading /Users/arungupta/.docker/machine/cache/boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v1.10.0/boot2docker.iso...
0%....10%....20%....30%....40%....50%....60%....70%....80%....90%....100%
Copying /Users/arungupta/.docker/machine/cache/boot2docker.iso to /Users/arungupta/.docker/machine/machines/couchbase/boot2docker.iso...
Starting machine back up...
(couchbase) Check network to re-create if needed...
(couchbase) Waiting for an IP...
Restarting docker...

~ > docker-machine upgrade couchbase

Waiting for SSH to be available...

Detecting the provisioner...

Upgrading docker...

Stopping machine to do the upgrade...

Upgrading machine "couchbase"...

Default Boot2Docker ISO is out-of-date, downloading the latest release...

Latest release for github.com/boot2docker/boot2docker is v1.10.0

Downloading /Users/arungupta/.docker/machine/cache/boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v1.10.0/boot2docker.iso...

0%....10%....20%....30%....40%....50%....60%....70%....80%....90%....100%

Copying /Users/arungupta/.docker/machine/cache/boot2docker.iso to /Users/arungupta/.docker/machine/machines/couchbase/boot2docker.iso...

Starting machine back up...

(couchbase) Check network to re-create if needed...

(couchbase) Waiting for an IP...

Restarting docker...

The updated list of Machines is now shown as:

~ > docker-machine ls
NAME        ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
couchbase   -        virtualbox   Running   tcp://192.168.99.100:2376           v1.10.0   
netbeans    -        virtualbox   Saved                                         Unknown

~ > docker-machine ls

NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS

couchbase - virtualbox Running tcp://192.168.99.100:2376 v1.10.0

netbeans - virtualbox Saved Unknown

Notice that Docker version is now 1.10.

Set up the environment variables such that Docker client can talk to it:

eval $(docker-machine env couchbase)

eval $(docker-machine env couchbase)

Docker Client 1.10

Lets download the latest client to connect to this Docker Engine.

> curl https://get.docker.com/builds/Darwin/x86_64/docker-latest > /usr/local/bin/docker
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10.0M  100 10.0M    0     0  3836k      0  0:00:02  0:00:02 --:--:-- 3836k
> docker version
Client:
 Version:      1.10.0
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   590d5108
 Built:        Thu Feb  4 19:55:25 2016
 OS/Arch:      darwin/amd64

Server:
 Version:      1.10.0
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   590d5108
 Built:        Thu Feb  4 19:55:25 2016
 OS/Arch:      linux/amd64

> curl https://get.docker.com/builds/Darwin/x86_64/docker-latest > /usr/local/bin/docker

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

100 10.0M 100 10.0M 0 0 3836k 0 0:00:02 0:00:02 --:--:-- 3836k

> docker version

Client:

Version: 1.10.0

API version: 1.22

Go version: go1.5.3

Git commit: 590d5108

Built: Thu Feb 4 19:55:25 2016

OS/Arch: darwin/amd64

Server:

Version: 1.10.0

API version: 1.22

Go version: go1.5.3

Git commit: 590d5108

Built: Thu Feb 4 19:55:25 2016

OS/Arch: linux/amd64

Client and Server versions are shown separately.

Run Couchbase container as:

~ > docker run -d -p 8091-8093:8091-8093 -p 11210:11210 arungupta/couchbase-node
4783d72298d1f27255f12cf765ed1121c7656d09a866bc350354e38787627a79

~ > docker run -d -p 8091-8093:8091-8093 -p 11210:11210 arungupta/couchbase-node

4783d72298d1f27255f12cf765ed1121c7656d09a866bc350354e38787627a79

This starts up a fully-configured Couchbase server. It can be accessed at 192.168.99.100:8091 and looks like as shown:

Note, 192.168.99.100 is obtained using docker-machine ip <MACHINE-NAME>.

Couchbase Developer Portal provide more details about the Couchbase Server.

Docker Compose 1.6.0

Docker Compose is a tool for defining and running complex applications with Docker. With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.

Learn more about Docker Compose 1.6.0.

Install the latest version:

curl -L https://github.com/docker/compose/releases/download/1.6.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   601    0   601    0     0   1067      0 --:--:-- --:--:-- --:--:--  1065
100 4837k  100 4837k    0     0   945k      0  0:00:05  0:00:05 --:--:-- 1426k
chmod +x /usr/local/bin/docker-compose

curl -L https://github.com/docker/compose/releases/download/1.6.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0

100 601 0 601 0 0 1067 0 --:--:-- --:--:-- --:--:-- 1065

100 4837k 100 4837k 0 0 945k 0 0:00:05 0:00:05 --:--:-- 1426k

chmod +x /usr/local/bin/docker-compose

The experimental flags --x-networking and --x-network-driver, introduced in Compose 1.5, have been removed. Its no longer experimental and is the recommended way to enable communication between containers.

Compose 1.6.0 requires Docker Engine 1.9.1 or later, or 1.10.0 if you’re using version 2 of the Compose File format.

Updating Compose File

Compose 1.6 introduces a new version of the Compose file. Read more details about Upgrading Compose File.

Compose 1.6 will continue to run older version of Compose files. But now networking and volumes are first class citizens.

Here is an example of version 1 of Compose file:

mycouchbase:
  image: arungupta/couchbase-node
  ports:
    - 8091:8091
    - 8092:8092 
    - 8093:8093 
    - 11210:11210

mycouchbase:

image: arungupta/couchbase-node

ports:

- 8091:8091

- 8092:8092

- 8093:8093

- 11210:11210

Here is a version 2 of Compose file:

version: "2"
services:
  mycouchbase:
    image: arungupta/couchbase
    ports:
      - 8091:8091
      - 8092:8092 
      - 8093:8093 
      - 11210:11210

version: "2"

services:

mycouchbase:

image: arungupta/couchbase

ports:

- 8091:8091

- 8092:8092

- 8093:8093

- 11210:11210

For simple use cases, the two main changes are:

Add a version: '2' line at the top of the file.
Indent the whole file by one level and put a services: key at the top.

Running services in this Compose file is:

~ > docker-compose up -d
Creating network "couchbase_default" with the default driver
Pulling mycouchbase (arungupta/couchbase:latest)...
latest: Pulling from arungupta/couchbase
a50d2b3f7efa: Already exists
27e1b0c788a2: Already exists
622b8fa00f81: Already exists
a3ed95caeb02: Pull complete
0bc56fd185bd: Already exists
10f0979b6cbd: Already exists
9be3d8460ed2: Already exists
59e718ec07e9: Already exists
a0b9aa29408d: Already exists
e2717fa0b388: Pull complete
Digest: sha256:9131c5a283d79e11b6556c94b0b0f8ceba7daf5ba5982bac850100099019959f
Status: Downloaded newer image for arungupta/couchbase:latest
Creating couchbase_mycouchbase_1

~ > docker-compose up -d

Creating network "couchbase_default" with the default driver

Pulling mycouchbase (arungupta/couchbase:latest)...

latest: Pulling from arungupta/couchbase

a50d2b3f7efa: Already exists

27e1b0c788a2: Already exists

622b8fa00f81: Already exists

a3ed95caeb02: Pull complete

0bc56fd185bd: Already exists

10f0979b6cbd: Already exists

9be3d8460ed2: Already exists

59e718ec07e9: Already exists

a0b9aa29408d: Already exists

e2717fa0b388: Pull complete

Digest: sha256:9131c5a283d79e11b6556c94b0b0f8ceba7daf5ba5982bac850100099019959f

Status: Downloaded newer image for arungupta/couchbase:latest

Creating couchbase_mycouchbase_1

This starts a fully configured Couchbase server based upon the image as explained at github.com/arun-gupta/docker-images/tree/master/couchbase-node.

Docker Swarm 1.1

Docker Swarm is native clustering for Docker. It allows you to create and access a pool of Docker hosts using the full suite of Docker tools. Because Docker Swarm serves the standard Docker API, any tool that already communicates with a Docker daemon can use Swarm to transparently scale to multiple hosts.

A new experimental support for container rescheduling on node failure is added.

Read more details about setting up Docker Swarm Cluster.

Finally, here are some useful links:

Docker Toolbox 1.10
Docker 1.10 Release Notes
Docker 1.10 Security Improvements
Docker for Java Developers

Enjoy!

JBoss EAP 7 and NoSQL using Java EE and Docker

December 21, 2015containers, couchbase, javaee, jboss, redhatcouchbase, docker, javaee7, jboss, nosqlarungupta

JBoss EAP 7 Beta is now released, many congratulations to Red Hat and particularly to the WildFly team!

There are plenty of improvements coming in this release as documented in Release Notes. One of the major themes is Java EE 7 compliance.

JBoss EAP 7 and Java EE 7

IBM and Oracle already provide commercially supported Java EE 7-compliant Application Servers. And now Red Hat will be joining this party soon as well. Although WildFly has supported Java EE 7 for 2+ years but commercial support is a critical for open source to be adopted enterprise-wide. So this is good news!

You can learn all about different Java EE 7 APIs in the DZone Refcardz that I authored along with @alrubinger.

Java EE 7 Refcardz

There are plenty of “hello world” Java EE 7 Samples that should all run with JBoss EAP. Hopefully somebody will update the pom.xml and add a new profile.

Why NoSQL?

If you are building a traditional enterprise application then you might be fine using an RDBMS. There are plenty of advantages of using RDBMS but using a NoSQL database instead has a few advantages:

No need to have a pre-defined schema and that makes them a schema-less database. Addition of new properties to existing objects is easy and does not require ALTER TABLE. The unstructured data gives flexibility to change the format of data any time without downtime or reduced service levels. Also there are no joins happening on the server because there is no structure and thus no relation between them.
Scalability, agility and performance is more important than the entire set of functionality typically provided by an RDBMS. This set of databases provide eventual consistency and/or transactions restricted to single items but more focus on CRUD.
NoSQL are designed to scale-out (horizontal) instead of scale-up (vertical). This is important knowing that databases, and everything else as well, is moving into the cloud. RBDMS can scale-out using sharding but requires complex management and not for the faint of heart. Queries requiring JOINs across shards is extremely inefficient.
RDBMS have impedance mismatch between the database structure and the domain classes. An Object Relational Mapping, such as one provided by Java Persistence API or Hibernate is needed in such case.
NoSQL databases are designed for less management and simpler data models lead to lower administration cost as well.

So you are all excited about NoSQL now and want to learn more:

Why NoSQL?
Why do successful enterprises rely on NoSQL?
Top 10 Enterprise NoSQL Usecases

In short, there are four different types of NoSQL databases:

Document: Couchbase, Mongo, and others
Key/Value: Couchbase, Redis, and others
Graph: Neo4J, OrientDB, and others
Column: Cassandra and others

Java EE 7 provides Java Persistence API that does not provide any support for NoSQL. So how do you get started with NoSQL with JBoss EAP 7?

This blog will show how to query a Couchbase database using simple Java EE application deployed on JBoss EAP 7 Beta.

What is Couchbase?

Couchbase is an open-source, NoSQL, document database. It allows to access, index, and query JSON documents while taking advantage of integrated distributed caching for high performance data access.

Developers can write applications to Couchbase using different languages (Java, Go, .NET, Node, PHP, Python, C) multiple SDKs. This blog will show how you can easily create a CRUD application using Java SDK for Couchbase.

Run JBoss EAP 7

There are two ways to start JBoss EAP 7.

Download and Run

Download JBoss EAP 7 Beta and unzip.

Start the application server as:

./jboss-eap-7.0/bin/standalone.sh 
=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /Users/arungupta/tools/jboss-eap-7.0

  JAVA: java

  JAVA_OPTS:  -server -verbose:gc -Xloggc:"/Users/arungupta/tools/jboss-eap-7.0/standalone/log/gc.log" -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms1303m -Xmx1303m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

=========================================================================

21:22:58,773 INFO  [org.jboss.modules] (main) JBoss Modules version 1.4.4.Final-redhat-1

. . .

21:23:21,441 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management
21:23:21,442 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990
21:23:21,442 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: EAP 7.0.0.Beta1 (WildFly Core 2.0.3.Final-redhat-1) started in 22950ms - Started 261 of 509 services (332 services are lazy, passive or on-demand)

./jboss-eap-7.0/bin/standalone.sh

=========================================================================

JBoss Bootstrap Environment

JBOSS_HOME: /Users/arungupta/tools/jboss-eap-7.0

JAVA: java

JAVA_OPTS: -server -verbose:gc -Xloggc:"/Users/arungupta/tools/jboss-eap-7.0/standalone/log/gc.log" -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms1303m -Xmx1303m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

=========================================================================

21:22:58,773 INFO [org.jboss.modules] (main) JBoss Modules version 1.4.4.Final-redhat-1

. . .

21:23:21,441 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management

21:23:21,442 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990

21:23:21,442 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: EAP 7.0.0.Beta1 (WildFly Core 2.0.3.Final-redhat-1) started in 22950ms - Started 261 of 509 services (332 services are lazy, passive or on-demand)

Docker Run

In a containerized world, you just docker run to run your JBoss EAP. However, JBoss EAP image does not exist on Docker Hub and so the image needs to be explicitly built. You still need to explicitly download JBoss EAP and then use the following Dockerfile to build the image:

# Use latest jboss/base-jdk:8 image as the base
FROM jboss/base-jdk:8

# Set the JBOSS_VERSION env variable
ENV JBOSS_VERSION 7.0.0.Beta
ENV JBOSS_HOME /opt/jboss/jboss-eap-7.0/

COPY jboss-eap-$JBOSS_VERSION.zip $HOME

# Add the JBoss distribution to /opt, and make jboss the owner of the extracted zip content
# Make sure the distribution is available from a well-known place
RUN cd $HOME \
    && unzip jboss-eap-$JBOSS_VERSION.zip \
    && rm jboss-eap-$JBOSS_VERSION.zip

# Ensure signals are forwarded to the JVM process correctly for graceful shutdown
ENV LAUNCH_JBOSS_IN_BACKGROUND true

# Expose the ports we're interested in
EXPOSE 8080 9990

# Set the default command to run on boot
# This will boot JBoss EAP in the standalone mode and bind to all interface
CMD ["/opt/jboss/jboss-eap-7.0/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]

# Use latest jboss/base-jdk:8 image as the base

FROM jboss/base-jdk:8

# Set the JBOSS_VERSION env variable

ENV JBOSS_VERSION 7.0.0.Beta

ENV JBOSS_HOME /opt/jboss/jboss-eap-7.0/

COPY jboss-eap-$JBOSS_VERSION.zip $HOME

# Add the JBoss distribution to /opt, and make jboss the owner of the extracted zip content

# Make sure the distribution is available from a well-known place

RUN cd $HOME \

&& unzip jboss-eap-$JBOSS_VERSION.zip \

&& rm jboss-eap-$JBOSS_VERSION.zip

# Ensure signals are forwarded to the JVM process correctly for graceful shutdown

ENV LAUNCH_JBOSS_IN_BACKGROUND true

# Expose the ports we're interested in

EXPOSE 8080 9990

# Set the default command to run on boot

# This will boot JBoss EAP in the standalone mode and bind to all interface

CMD ["/opt/jboss/jboss-eap-7.0/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]

The image is built as:

docker build -t arungupta/jboss-eap:7-beta .

docker build -t arungupta/jboss-eap:7-beta .

And then you can run the JBoss EAP 7 container as:

docker run -it -p 8080:8080 arungupta/jboss-eap:7-beta
=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /opt/jboss/jboss-eap-7.0/

  JAVA: /usr/lib/jvm/java/bin/java

  JAVA_OPTS:  -server -verbose:gc -Xloggc:"/opt/jboss/jboss-eap-7.0//standalone/log/gc.log" -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms1303m -Xmx1303m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

=========================================================================

20:51:12,551 INFO  [org.jboss.modules] (main) JBoss Modules version 1.4.4.Final-redhat-1
20:51:12,824 INFO  [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final-redhat-1

. . .

20:51:16,750 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management
20:51:16,758 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://0.0.0.0:9990
20:51:16,759 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: EAP 7.0.0.Beta1 (WildFly Core 2.0.3.Final-redhat-1) started in 4529ms - Started 261 of 509 services (332 services are lazy, passive or on-demand)

docker run -it -p 8080:8080 arungupta/jboss-eap:7-beta

=========================================================================

JBoss Bootstrap Environment

JBOSS_HOME: /opt/jboss/jboss-eap-7.0/

JAVA: /usr/lib/jvm/java/bin/java

JAVA_OPTS: -server -verbose:gc -Xloggc:"/opt/jboss/jboss-eap-7.0//standalone/log/gc.log" -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms1303m -Xmx1303m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

=========================================================================

20:51:12,551 INFO [org.jboss.modules] (main) JBoss Modules version 1.4.4.Final-redhat-1

20:51:12,824 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final-redhat-1

. . .

20:51:16,750 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management

20:51:16,758 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://0.0.0.0:9990

20:51:16,759 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: EAP 7.0.0.Beta1 (WildFly Core 2.0.3.Final-redhat-1) started in 4529ms - Started 261 of 509 services (332 services are lazy, passive or on-demand)

Notice, how application and management ports are bound to all network interfaces. This will simplify to deploy the application to this JBoss EAP instance later.

Stop the server as we will show an easier way to start it later.

Start Application Server and Database

The Java EE application will provide a HTTP CRUD interface over JSON documents stored in Couchbase. The application itself will be deployed on JBoss EAP 7 Beta. So it would require to start Couchbase and JBoss EAP.

Use the Docker Compose file from github.com/arun-gupta/docker-images/blob/master/jboss-eap7-nosql/docker-compose.yml to start Couchbase and JBoss EAP 7 container:

mycouchbase:
  container_name: "db"
  image: couchbase/server
  ports:
    - 8091:8091
    - 8092:8092 
    - 8093:8093 
    - 11210:11210
jboss:
  image: arungupta/jboss-eap:7-beta
  environment:
    - COUCHBASE_URI=db
  ports:
    - 8080:8080
    - 9990:9990

mycouchbase:

container_name: "db"

image: couchbase/server

ports:

- 8091:8091

- 8092:8092

- 8093:8093

- 11210:11210

jboss:

image: arungupta/jboss-eap:7-beta

environment:

- COUCHBASE_URI=db

ports:

- 8080:8080

- 9990:9990

The application is started as:

docker-compose --x-networking up -d
Creating network "jbosseap7nosql" with driver "None"
Starting jbosseap7nosql_jboss_1
Creating db

docker-compose --x-networking up -d

Creating network "jbosseap7nosql" with driver "None"

Starting jbosseap7nosql_jboss_1

Creating db

The started containers can be seen as:

docker ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS                                                                                               NAMES
154436dfbfb1        couchbase/server             "/entrypoint.sh couch"   10 seconds ago      Up 8 seconds        0.0.0.0:8091-8093->8091-8093/tcp, 11207/tcp, 11211/tcp, 18091-18092/tcp, 0.0.0.0:11210->11210/tcp   db
cb76d4e38df3        arungupta/jboss-eap:7-beta   "/opt/jboss/jboss-eap"   10 seconds ago      Up 9 seconds        0.0.0.0:8080->8080/tcp, 0.0.0.0:9990->9990/tcp                                                      jbosseap7nosql_jboss_1

docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

154436dfbfb1 couchbase/server "/entrypoint.sh couch" 10 seconds ago Up 8 seconds 0.0.0.0:8091-8093->8091-8093/tcp, 11207/tcp, 11211/tcp, 18091-18092/tcp, 0.0.0.0:11210->11210/tcp db

cb76d4e38df3 arungupta/jboss-eap:7-beta "/opt/jboss/jboss-eap" 10 seconds ago Up 9 seconds 0.0.0.0:8080->8080/tcp, 0.0.0.0:9990->9990/tcp jbosseap7nosql_jboss_1

Configure Couchbase Server

Clone couchbase-javaee application. This Java EE application uses Couchbase Java SDK APIs to connect to the Couchbase server. The bootstrap code is:

CouchbaseCluster.create(System.getenv("COUCHBASE_URI"));

CouchbaseCluster.create(System.getenv("COUCHBASE_URI"));

and is invoked from Database abstraction.

Couchbase Server can be configured using REST API. These REST APIs are defined in a Maven profile in pom.xml of this application. And so configure Couchbase server as:

mvn install -Pcouchbase -Ddocker.host=$(docker-machine ip couchbase)
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building couchbase-javaee 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ couchbase-javaee ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ couchbase-javaee ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ couchbase-javaee ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/arungupta/workspaces/couchbase-javaee/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ couchbase-javaee ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ couchbase-javaee ---
[INFO] 
[INFO] --- maven-war-plugin:2.1.1:war (default-war) @ couchbase-javaee ---
[INFO] Packaging webapp
[INFO] Assembling webapp [couchbase-javaee] in [/Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee]
[INFO] Processing war project
[INFO] Copying webapp resources [/Users/arungupta/workspaces/couchbase-javaee/src/main/webapp]
[INFO] Webapp assembled in [82 msecs]
[INFO] Building war: /Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee.war
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ couchbase-javaee ---
[INFO] Installing /Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee.war to /Users/arungupta/.m2/repository/org/couchbase/sample/couchbase-javaee/1.0-SNAPSHOT/couchbase-javaee-1.0-SNAPSHOT.war
[INFO] Installing /Users/arungupta/workspaces/couchbase-javaee/pom.xml to /Users/arungupta/.m2/repository/org/couchbase/sample/couchbase-javaee/1.0-SNAPSHOT/couchbase-javaee-1.0-SNAPSHOT.pom
[INFO] 
[INFO] --- exec-maven-plugin:1.4.0:exec (Configure memory) @ couchbase-javaee ---
* Hostname was NOT found in DNS cache
*   Trying 192.168.99.102...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 192.168.99.102 (192.168.99.102) port 8091 (#0)
> POST /pools/default HTTP/1.1
> User-Agent: curl/7.37.1
> Host: 192.168.99.102:8091
> Accept: */*
> Content-Length: 36
> Content-Type: application/x-www-form-urlencoded
> 
} [data not shown]
* upload completely sent off: 36 out of 36 bytes
< HTTP/1.1 200 OK
* Server Couchbase Server is not blacklisted
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 21 Dec 2015 21:35:10 GMT
< Content-Length: 0
< Cache-Control: no-cache
< 
100    36    0     0  100    36      0  15510 --:--:-- --:--:-- --:--:-- 18000
* Connection #0 to host 192.168.99.102 left intact
[INFO] 
[INFO] --- exec-maven-plugin:1.4.0:exec (Configure services) @ couchbase-javaee ---
* Hostname was NOT found in DNS cache
*   Trying 192.168.99.102...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 192.168.99.102 (192.168.99.102) port 8091 (#0)
> POST /node/controller/setupServices HTTP/1.1
> User-Agent: curl/7.37.1
> Host: 192.168.99.102:8091
> Accept: */*
> Content-Length: 26
> Content-Type: application/x-www-form-urlencoded
> 
} [data not shown]
* upload completely sent off: 26 out of 26 bytes
< HTTP/1.1 200 OK
* Server Couchbase Server is not blacklisted
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 21 Dec 2015 21:35:10 GMT
< Content-Length: 0
< Cache-Control: no-cache
< 
100    26    0     0  100    26      0   9976 --:--:-- --:--:-- --:--:-- 13000
* Connection #0 to host 192.168.99.102 left intact
[INFO] 
[INFO] --- exec-maven-plugin:1.4.0:exec (Setup credentials) @ couchbase-javaee ---
* Hostname was NOT found in DNS cache
*   Trying 192.168.99.102...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 192.168.99.102 (192.168.99.102) port 8091 (#0)
> POST /settings/web HTTP/1.1
> User-Agent: curl/7.37.1
> Host: 192.168.99.102:8091
> Accept: */*
> Content-Length: 50
> Content-Type: application/x-www-form-urlencoded
> 
} [data not shown]
* upload completely sent off: 50 out of 50 bytes
< HTTP/1.1 200 OK
* Server Couchbase Server is not blacklisted
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 21 Dec 2015 21:35:10 GMT
< Content-Type: application/json
< Content-Length: 44
< Cache-Control: no-cache
< 
{ [data not shown]
100    94  100    44  100    50   6880   7818 --:--:-- --:--:-- --:--:--  8333
* Connection #0 to host 192.168.99.102 left intact
{"newBaseUri":"http://192.168.99.102:8091/"}[INFO] 
[INFO] --- exec-maven-plugin:1.4.0:exec (Install travel-sample bucket) @ couchbase-javaee ---
* Hostname was NOT found in DNS cache
*   Trying 192.168.99.102...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 192.168.99.102 (192.168.99.102) port 8091 (#0)
* Server auth using Basic with user 'Administrator'
> POST /sampleBuckets/install HTTP/1.1
> Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==
> User-Agent: curl/7.37.1
> Host: 192.168.99.102:8091
> Accept: */*
> Content-Length: 17
> Content-Type: application/x-www-form-urlencoded
> 
} [data not shown]
* upload completely sent off: 17 out of 17 bytes
< HTTP/1.1 202 Accepted
* Server Couchbase Server is not blacklisted
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 21 Dec 2015 21:35:11 GMT
< Content-Type: application/json
< Content-Length: 2
< Cache-Control: no-cache
< 
{ [data not shown]
100    19  100     2  100    17     41    355 --:--:-- --:--:-- --:--:--   361
* Connection #0 to host 192.168.99.102 left intact
[][INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.094 s
[INFO] Finished at: 2015-12-21T13:35:11-08:00
[INFO] Final Memory: 13M/309M
[INFO] ------------------------------------------------------------------------

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

mvn install -Pcouchbase -Ddocker.host=$(docker-machine ip couchbase)

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building couchbase-javaee 1.0-SNAPSHOT

[INFO] ------------------------------------------------------------------------

[INFO]

[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ couchbase-javaee ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] Copying 0 resource

[INFO]

[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ couchbase-javaee ---

[INFO] Nothing to compile - all classes are up to date

[INFO]

[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ couchbase-javaee ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] skip non existing resourceDirectory /Users/arungupta/workspaces/couchbase-javaee/src/test/resources

[INFO]

[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ couchbase-javaee ---

[INFO] Nothing to compile - all classes are up to date

[INFO]

[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ couchbase-javaee ---

[INFO]

[INFO] --- maven-war-plugin:2.1.1:war (default-war) @ couchbase-javaee ---

[INFO] Packaging webapp

[INFO] Assembling webapp [couchbase-javaee] in [/Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee]

[INFO] Processing war project

[INFO] Copying webapp resources [/Users/arungupta/workspaces/couchbase-javaee/src/main/webapp]

[INFO] Webapp assembled in [82 msecs]

[INFO] Building war: /Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee.war

[INFO]

[INFO] --- maven-install-plugin:2.4:install (default-install) @ couchbase-javaee ---

[INFO] Installing /Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee.war to /Users/arungupta/.m2/repository/org/couchbase/sample/couchbase-javaee/1.0-SNAPSHOT/couchbase-javaee-1.0-SNAPSHOT.war

[INFO] Installing /Users/arungupta/workspaces/couchbase-javaee/pom.xml to /Users/arungupta/.m2/repository/org/couchbase/sample/couchbase-javaee/1.0-SNAPSHOT/couchbase-javaee-1.0-SNAPSHOT.pom

[INFO]

[INFO] --- exec-maven-plugin:1.4.0:exec (Configure memory) @ couchbase-javaee ---

* Hostname was NOT found in DNS cache

* Trying 192.168.99.102...

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.99.102 (192.168.99.102) port 8091 (#0)

> POST /pools/default HTTP/1.1

> User-Agent: curl/7.37.1

> Host: 192.168.99.102:8091

> Accept: */*

> Content-Length: 36

> Content-Type: application/x-www-form-urlencoded

} [data not shown]

* upload completely sent off: 36 out of 36 bytes

< HTTP/1.1 200 OK

* Server Couchbase Server is not blacklisted

< Server: Couchbase Server

< Pragma: no-cache

< Date: Mon, 21 Dec 2015 21:35:10 GMT

< Content-Length: 0

< Cache-Control: no-cache

100 36 0 0 100 36 0 15510 --:--:-- --:--:-- --:--:-- 18000

* Connection #0 to host 192.168.99.102 left intact

[INFO]

[INFO] --- exec-maven-plugin:1.4.0:exec (Configure services) @ couchbase-javaee ---

* Hostname was NOT found in DNS cache

* Trying 192.168.99.102...

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.99.102 (192.168.99.102) port 8091 (#0)

> POST /node/controller/setupServices HTTP/1.1

> User-Agent: curl/7.37.1

> Host: 192.168.99.102:8091

> Accept: */*

> Content-Length: 26

> Content-Type: application/x-www-form-urlencoded

} [data not shown]

* upload completely sent off: 26 out of 26 bytes

< HTTP/1.1 200 OK

* Server Couchbase Server is not blacklisted

< Server: Couchbase Server

< Pragma: no-cache

< Date: Mon, 21 Dec 2015 21:35:10 GMT

< Content-Length: 0

< Cache-Control: no-cache

100 26 0 0 100 26 0 9976 --:--:-- --:--:-- --:--:-- 13000

* Connection #0 to host 192.168.99.102 left intact

[INFO]

[INFO] --- exec-maven-plugin:1.4.0:exec (Setup credentials) @ couchbase-javaee ---

* Hostname was NOT found in DNS cache

* Trying 192.168.99.102...

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.99.102 (192.168.99.102) port 8091 (#0)

> POST /settings/web HTTP/1.1

> User-Agent: curl/7.37.1

> Host: 192.168.99.102:8091

> Accept: */*

> Content-Length: 50

> Content-Type: application/x-www-form-urlencoded

} [data not shown]

* upload completely sent off: 50 out of 50 bytes

< HTTP/1.1 200 OK

* Server Couchbase Server is not blacklisted

< Server: Couchbase Server

< Pragma: no-cache

< Date: Mon, 21 Dec 2015 21:35:10 GMT

< Content-Type: application/json

< Content-Length: 44

< Cache-Control: no-cache

{ [data not shown]

100 94 100 44 100 50 6880 7818 --:--:-- --:--:-- --:--:-- 8333

* Connection #0 to host 192.168.99.102 left intact

{"newBaseUri":"http://192.168.99.102:8091/"}[INFO]

[INFO] --- exec-maven-plugin:1.4.0:exec (Install travel-sample bucket) @ couchbase-javaee ---

* Hostname was NOT found in DNS cache

* Trying 192.168.99.102...

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.99.102 (192.168.99.102) port 8091 (#0)

* Server auth using Basic with user 'Administrator'

> POST /sampleBuckets/install HTTP/1.1

> Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==

> User-Agent: curl/7.37.1

> Host: 192.168.99.102:8091

> Accept: */*

> Content-Length: 17

> Content-Type: application/x-www-form-urlencoded

} [data not shown]

* upload completely sent off: 17 out of 17 bytes

< HTTP/1.1 202 Accepted

* Server Couchbase Server is not blacklisted

< Server: Couchbase Server

< Pragma: no-cache

< Date: Mon, 21 Dec 2015 21:35:11 GMT

< Content-Type: application/json

< Content-Length: 2

< Cache-Control: no-cache

{ [data not shown]

100 19 100 2 100 17 41 355 --:--:-- --:--:-- --:--:-- 361

* Connection #0 to host 192.168.99.102 left intact

[][INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 2.094 s

[INFO] Finished at: 2015-12-21T13:35:11-08:00

[INFO] Final Memory: 13M/309M

[INFO] ------------------------------------------------------------------------

Deploy Java EE Application to JBoss

Java EE Application can be easily deployed to JBoss EAP 7 Beta using the WildFly Maven Plugin. This is also defined as a Maven profile in pom.xml as well.

Deploy the application as:

mvn install -Pwildfly -Dwildfly.hostname=$(docker-machine ip couchbase) -Dwildfly.username=admin -Dwildfly.password=Admin#007
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building couchbase-javaee 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ couchbase-javaee ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ couchbase-javaee ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ couchbase-javaee ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/arungupta/workspaces/couchbase-javaee/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ couchbase-javaee ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ couchbase-javaee ---
[INFO] 
[INFO] --- maven-war-plugin:2.1.1:war (default-war) @ couchbase-javaee ---
[INFO] Packaging webapp
[INFO] Assembling webapp [couchbase-javaee] in [/Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee]
[INFO] Processing war project
[INFO] Copying webapp resources [/Users/arungupta/workspaces/couchbase-javaee/src/main/webapp]
[INFO] Webapp assembled in [62 msecs]
[INFO] Building war: /Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee.war
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ couchbase-javaee ---
[INFO] Installing /Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee.war to /Users/arungupta/.m2/repository/org/couchbase/sample/couchbase-javaee/1.0-SNAPSHOT/couchbase-javaee-1.0-SNAPSHOT.war
[INFO] Installing /Users/arungupta/workspaces/couchbase-javaee/pom.xml to /Users/arungupta/.m2/repository/org/couchbase/sample/couchbase-javaee/1.0-SNAPSHOT/couchbase-javaee-1.0-SNAPSHOT.pom
[INFO] 
[INFO] >>> wildfly-maven-plugin:1.1.0.Alpha4:deploy (default) > package @ couchbase-javaee >>>
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ couchbase-javaee ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ couchbase-javaee ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ couchbase-javaee ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/arungupta/workspaces/couchbase-javaee/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ couchbase-javaee ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ couchbase-javaee ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
[INFO] 
[INFO] --- maven-war-plugin:2.1.1:war (default-war) @ couchbase-javaee ---
[INFO] Packaging webapp
[INFO] Assembling webapp [couchbase-javaee] in [/Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee]
[INFO] Processing war project
[INFO] Copying webapp resources [/Users/arungupta/workspaces/couchbase-javaee/src/main/webapp]
[INFO] Webapp assembled in [20 msecs]
[INFO] Building war: /Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee.war
[INFO] 
[INFO] <<< wildfly-maven-plugin:1.1.0.Alpha4:deploy (default) < package @ couchbase-javaee <<<
[INFO] 
[INFO] --- wildfly-maven-plugin:1.1.0.Alpha4:deploy (default) @ couchbase-javaee ---
Dec 21, 2015 1:43:34 PM org.xnio.Xnio <clinit>
INFO: XNIO version 3.3.1.Final
Dec 21, 2015 1:43:34 PM org.xnio.nio.NioXnio <clinit>
INFO: XNIO NIO Implementation Version 3.3.1.Final
Dec 21, 2015 1:43:34 PM org.jboss.remoting3.EndpointImpl <clinit>
INFO: JBoss Remoting version 4.0.9.Final
[INFO] Authenticating against security realm: ManagementRealm
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17.010 s
[INFO] Finished at: 2015-12-21T13:43:48-08:00
[INFO] Final Memory: 17M/217M
[INFO] ------------------------------------------------------------------------

mvn install -Pwildfly -Dwildfly.hostname=$(docker-machine ip couchbase) -Dwildfly.username=admin -Dwildfly.password=Admin#007

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building couchbase-javaee 1.0-SNAPSHOT

[INFO] ------------------------------------------------------------------------

[INFO]

[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ couchbase-javaee ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] Copying 0 resource

[INFO]

[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ couchbase-javaee ---

[INFO] Nothing to compile - all classes are up to date

[INFO]

[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ couchbase-javaee ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] skip non existing resourceDirectory /Users/arungupta/workspaces/couchbase-javaee/src/test/resources

[INFO]

[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ couchbase-javaee ---

[INFO] Nothing to compile - all classes are up to date

[INFO]

[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ couchbase-javaee ---

[INFO]

[INFO] --- maven-war-plugin:2.1.1:war (default-war) @ couchbase-javaee ---

[INFO] Packaging webapp

[INFO] Assembling webapp [couchbase-javaee] in [/Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee]

[INFO] Processing war project

[INFO] Copying webapp resources [/Users/arungupta/workspaces/couchbase-javaee/src/main/webapp]

[INFO] Webapp assembled in [62 msecs]

[INFO] Building war: /Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee.war

[INFO]

[INFO] --- maven-install-plugin:2.4:install (default-install) @ couchbase-javaee ---

[INFO] Installing /Users/arungupta/workspaces/couchbase-javaee/pom.xml to /Users/arungupta/.m2/repository/org/couchbase/sample/couchbase-javaee/1.0-SNAPSHOT/couchbase-javaee-1.0-SNAPSHOT.pom

[INFO]

[INFO] >>> wildfly-maven-plugin:1.1.0.Alpha4:deploy (default) > package @ couchbase-javaee >>>

[INFO]

[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ couchbase-javaee ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] Copying 0 resource

[INFO]

[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ couchbase-javaee ---

[INFO] Nothing to compile - all classes are up to date

[INFO]

[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ couchbase-javaee ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] skip non existing resourceDirectory /Users/arungupta/workspaces/couchbase-javaee/src/test/resources

[INFO]

[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ couchbase-javaee ---

[INFO] Nothing to compile - all classes are up to date

[INFO]

[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ couchbase-javaee ---

[INFO] Skipping execution of surefire because it has already been run for this configuration

[INFO]

[INFO] --- maven-war-plugin:2.1.1:war (default-war) @ couchbase-javaee ---

[INFO] Packaging webapp

[INFO] Assembling webapp [couchbase-javaee] in [/Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee]

[INFO] Processing war project

[INFO] Copying webapp resources [/Users/arungupta/workspaces/couchbase-javaee/src/main/webapp]

[INFO] Webapp assembled in [20 msecs]

[INFO] Building war: /Users/arungupta/workspaces/couchbase-javaee/target/couchbase-javaee.war

[INFO]

[INFO] <<< wildfly-maven-plugin:1.1.0.Alpha4:deploy (default) < package @ couchbase-javaee <<<

[INFO]

[INFO] --- wildfly-maven-plugin:1.1.0.Alpha4:deploy (default) @ couchbase-javaee ---

Dec 21, 2015 1:43:34 PM org.xnio.Xnio <clinit>

INFO: XNIO version 3.3.1.Final

Dec 21, 2015 1:43:34 PM org.xnio.nio.NioXnio <clinit>

INFO: XNIO NIO Implementation Version 3.3.1.Final

Dec 21, 2015 1:43:34 PM org.jboss.remoting3.EndpointImpl <clinit>

INFO: JBoss Remoting version 4.0.9.Final

[INFO] Authenticating against security realm: ManagementRealm

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 17.010 s

[INFO] Finished at: 2015-12-21T13:43:48-08:00

[INFO] Final Memory: 17M/217M

[INFO] ------------------------------------------------------------------------

Access the Application

As mentioned earlier, the application provides HTTP CRUD API over JSON documents stored in Couchbase.

Access the application as:

curl -v http://$(docker-machine ip couchbase):8080/couchbase-javaee/resources/airline
* Hostname was NOT found in DNS cache
*   Trying 192.168.99.102...
* Connected to 192.168.99.102 (192.168.99.102) port 8080 (#0)
> GET /couchbase-javaee/resources/airline HTTP/1.1
> User-Agent: curl/7.37.1
> Host: 192.168.99.102:8080
> Accept: */*
> 
< HTTP/1.1 200 OK
< Connection: keep-alive
< X-Powered-By: Undertow/1
* Server JBoss-EAP/7 is not blacklisted
< Server: JBoss-EAP/7
< Content-Type: application/octet-stream
< Content-Length: 1402
< Date: Mon, 21 Dec 2015 21:45:40 GMT
< 
* Connection #0 to host 192.168.99.102 left intact
[{"travel-sample":{"country":"United States","iata":"Q5","callsign":"MILE-AIR","name":"40-Mile Air","icao":"MLA","id":10,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"TQ","callsign":"TXW","name":"Texas Wings","icao":"TXW","id":10123,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"A1","callsign":"atifly","name":"Atifly","icao":"A1F","id":10226,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":null,"callsign":null,"name":"Jc royal.britannica","icao":"JRB","id":10642,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"ZQ","callsign":"LOCAIR","name":"Locair","icao":"LOC","id":10748,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"K5","callsign":"SASQUATCH","name":"SeaPort Airlines","icao":"SQH","id":10765,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"KO","callsign":"ACE AIR","name":"Alaska Central Express","icao":"AER","id":109,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":"5W","callsign":"FLYSTAR","name":"Astraeus","icao":"AEU","id":112,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"UU","callsign":"REUNION","name":"Air Austral","icao":"REU","id":1191,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","id":1203,"type":"airline"}}]

curl -v http://$(docker-machine ip couchbase):8080/couchbase-javaee/resources/airline

* Hostname was NOT found in DNS cache

* Trying 192.168.99.102...

* Connected to 192.168.99.102 (192.168.99.102) port 8080 (#0)

> GET /couchbase-javaee/resources/airline HTTP/1.1

> User-Agent: curl/7.37.1

> Host: 192.168.99.102:8080

> Accept: */*

< HTTP/1.1 200 OK

< Connection: keep-alive

< X-Powered-By: Undertow/1

* Server JBoss-EAP/7 is not blacklisted

< Server: JBoss-EAP/7

< Content-Type: application/octet-stream

< Content-Length: 1402

< Date: Mon, 21 Dec 2015 21:45:40 GMT

* Connection #0 to host 192.168.99.102 left intact

[{"travel-sample":{"country":"United States","iata":"Q5","callsign":"MILE-AIR","name":"40-Mile Air","icao":"MLA","id":10,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"TQ","callsign":"TXW","name":"Texas Wings","icao":"TXW","id":10123,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"A1","callsign":"atifly","name":"Atifly","icao":"A1F","id":10226,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":null,"callsign":null,"name":"Jc royal.britannica","icao":"JRB","id":10642,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"ZQ","callsign":"LOCAIR","name":"Locair","icao":"LOC","id":10748,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"K5","callsign":"SASQUATCH","name":"SeaPort Airlines","icao":"SQH","id":10765,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"KO","callsign":"ACE AIR","name":"Alaska Central Express","icao":"AER","id":109,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":"5W","callsign":"FLYSTAR","name":"Astraeus","icao":"AEU","id":112,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"UU","callsign":"REUNION","name":"Air Austral","icao":"REU","id":1191,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","id":1203,"type":"airline"}}]

CRUD operations (GET, POST, PUT, DELETE) can be performed on Airline resource in the application. Complete CRUD API is documented at github.com/arun-gupta/couchbase-javaee.

This blog explained how to access a NoSQL database from JBoss EAP 7.

Docker Machine, Swarm and Compose for multi-container and multi-host applications with Couchbase and WildFly

December 3, 2015containers, couchbase, javaee, wildflycouchbase, docker, javaee7, wildflyarungupta

This blog will explain how to create multi-container application deployed on multiple hosts using Docker. This will be achieved using Docker Machine, Swarm and Compose.

Yes, all three tools together makes this blog that much more interesting!

Docker Swarm Machine Compose

The diagram explains the key components:

Docker Machine is used to provision multiple Docker hosts
Docker Swarm will be used to create a multi-host cluster
Each node in Docker Swarm cluster is registered/discovered using Consul
Multi-container application will be deployed using Docker Compose
WildFly and Couchbase are provisioned on different hosts
Docker multi-host networking is used for WildFly and Couchbase to communicate

In addition, Maven is used to configure Couchbase and deploy application to WildFly.

Latest instructions at Docker for Java Developers.

No story, just pure code, lets do it!

Create Discovery Service using Docker Machine

Create a Machine that will host discovery service:

docker-machine create -d=virtualbox consul-machine
Running pre-create checks...
Creating machine...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Provisioning created instance...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
To see how to connect Docker to this machine, run: docker-machine env consul-machine

docker-machine create -d=virtualbox consul-machine

Running pre-create checks...

Creating machine...

Waiting for machine to be running, this may take a few minutes...

Machine is running, waiting for SSH to be available...

Detecting operating system of created instance...

Provisioning created instance...

Copying certs to the local machine directory...

Copying certs to the remote machine...

Setting Docker configuration on the remote daemon...

To see how to connect Docker to this machine, run: docker-machine env consul-machine

Connect to this Machine:

eval $(docker-machine env consul-machine)

eval $(docker-machine env consul-machine)

Run Consul service using the following Compose file:

myconsul:
  image: progrium/consul
  restart: always
  hostname: consul
  ports:
    - 8500:8500
  command: "-server -bootstrap"

myconsul:

image: progrium/consul

restart: always

hostname: consul

ports:

- 8500:8500

command: "-server -bootstrap"

This Compose file is available at https://github.com/arun-gupta/docker-images/blob/master/consul/docker-compose.yml.

docker-compose up -d
Pulling myconsul (progrium/consul:latest)...
latest: Pulling from progrium/consul
3b4d28ce80e4: Pull complete
e5ab901dcf2d: Pull complete
30ad296c0ea0: Pull complete
3dba40dec256: Pull complete
f2ef4387b95e: Pull complete
53bc8dcc4791: Pull complete
75ed0b50ba1d: Pull complete
17c3a7ed5521: Pull complete
8aca9e0ecf68: Pull complete
4d1828359d36: Pull complete
46ed7df7f742: Pull complete
b5e8ce623ef8: Pull complete
049dca6ef253: Pull complete
bdb608bc4555: Pull complete
8b3d489cfb73: Pull complete
c74500bbce24: Pull complete
9f3e605442f6: Pull complete
d9125e9e799b: Pull complete
Digest: sha256:8cc8023462905929df9a79ff67ee435a36848ce7a10f18d6d0faba9306b97274
Status: Downloaded newer image for progrium/consul:latest
Creating consul_myconsul_1

docker-compose up -d

Pulling myconsul (progrium/consul:latest)...

latest: Pulling from progrium/consul

3b4d28ce80e4: Pull complete

e5ab901dcf2d: Pull complete

30ad296c0ea0: Pull complete

3dba40dec256: Pull complete

f2ef4387b95e: Pull complete

53bc8dcc4791: Pull complete

75ed0b50ba1d: Pull complete

17c3a7ed5521: Pull complete

8aca9e0ecf68: Pull complete

4d1828359d36: Pull complete

46ed7df7f742: Pull complete

b5e8ce623ef8: Pull complete

049dca6ef253: Pull complete

bdb608bc4555: Pull complete

8b3d489cfb73: Pull complete

c74500bbce24: Pull complete

9f3e605442f6: Pull complete

d9125e9e799b: Pull complete

Digest: sha256:8cc8023462905929df9a79ff67ee435a36848ce7a10f18d6d0faba9306b97274

Status: Downloaded newer image for progrium/consul:latest

Creating consul_myconsul_1

Started container can be verified as:

docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                            NAMES
f05d8dd11e7f        progrium/consul     "/bin/start -server -"   30 seconds ago      Up 29 seconds       53/tcp, 53/udp, 8300-8302/tcp, 8400/tcp, 0.0.0.0:8500->8500/tcp, 8301-8302/udp   consul_myconsul_1

docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

f05d8dd11e7f progrium/consul "/bin/start -server -" 30 seconds ago Up 29 seconds 53/tcp, 53/udp, 8300-8302/tcp, 8400/tcp, 0.0.0.0:8500->8500/tcp, 8301-8302/udp consul_myconsul_1

Create Docker Swarm Cluster using Docker Machine

Swarm is fully integrated with Machine, and so is the easiest way to get started.

Create a Swarm Master and point to the Consul discovery service:

docker-machine create -d virtualbox --virtualbox-disk-size "5000" --swarm --swarm-master --swarm-discovery="consul://$(docker-machine ip consul-machine):8500" --engine-opt="cluster-store=consul://$(docker-machine ip consul-machine):8500" --engine-opt="cluster-advertise=eth1:2376" swarm-master
Running pre-create checks...
Creating machine...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Provisioning created instance...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Configuring swarm...
To see how to connect Docker to this machine, run: docker-machine env swarm-master

docker-machine create -d virtualbox --virtualbox-disk-size "5000" --swarm --swarm-master --swarm-discovery="consul://$(docker-machine ip consul-machine):8500" --engine-opt="cluster-store=consul://$(docker-machine ip consul-machine):8500" --engine-opt="cluster-advertise=eth1:2376" swarm-master

Running pre-create checks...

Creating machine...

Waiting for machine to be running, this may take a few minutes...

Machine is running, waiting for SSH to be available...

Detecting operating system of created instance...

Provisioning created instance...

Copying certs to the local machine directory...

Copying certs to the remote machine...

Setting Docker configuration on the remote daemon...

Configuring swarm...

To see how to connect Docker to this machine, run: docker-machine env swarm-master

Few options to look here:

--swarm configures the Machine with Swarm
--swarm-master configures the created Machine to be Swarm master
--swarm-discovery defines address of the discovery service
--cluster-advertise advertise the machine on the network
--cluster-store designate a distributed k/v storage backend for the cluster
--virtualbox-disk-size sets the disk size for the created Machine to 5GB. This is required so that WildFly and Couchbase image can be downloaded on any of the nodes.

Find some information about this machine:

docker-machine inspect --format='{{json .Driver}}'  swarm-master
{"Boot2DockerImportVM":"","Boot2DockerURL":"","CPU":1,"DiskSize":5000,"HostOnlyCIDR":"192.168.99.1/24","HostOnlyNicType":"82540EM","HostOnlyPromiscMode":"deny","IPAddress":"192.168.99.102","MachineName":"swarm-master","Memory":1024,"NoShare":false,"SSHPort":51972,"SSHUser":"docker","StorePath":"/Users/arungupta/.docker/machine","SwarmDiscovery":"consul://192.168.99.100:8500","SwarmHost":"tcp://0.0.0.0:3376","SwarmMaster":true,"VBoxManager":{}}

docker-machine inspect --format='{{json .Driver}}' swarm-master

{"Boot2DockerImportVM":"","Boot2DockerURL":"","CPU":1,"DiskSize":5000,"HostOnlyCIDR":"192.168.99.1/24","HostOnlyNicType":"82540EM","HostOnlyPromiscMode":"deny","IPAddress":"192.168.99.102","MachineName":"swarm-master","Memory":1024,"NoShare":false,"SSHPort":51972,"SSHUser":"docker","StorePath":"/Users/arungupta/.docker/machine","SwarmDiscovery":"consul://192.168.99.100:8500","SwarmHost":"tcp://0.0.0.0:3376","SwarmMaster":true,"VBoxManager":{}}

Note that the disk size is 5GB.

Connect to the master by using the command:

eval "$(docker-machine env --swarm swarm-master)"

eval "$(docker-machine env --swarm swarm-master)"

Find some information about the cluster:

docker info
Containers: 2
Images: 1
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 1
 swarm-master: 192.168.99.102:2376
  └ Containers: 2
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.021 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.1.13-boot2docker, operatingsystem=Boot2Docker 1.9.1 (TCL 6.4.1); master : cef800b - Fri Nov 20 19:33:59 UTC 2015, provider=virtualbox, storagedriver=aufs
CPUs: 1
Total Memory: 1.021 GiB
Name: d074fd97682e

docker info

Containers: 2

Images: 1

Role: primary

Strategy: spread

Filters: health, port, dependency, affinity, constraint

Nodes: 1

swarm-master: 192.168.99.102:2376

└ Containers: 2

└ Reserved CPUs: 0 / 1

└ Reserved Memory: 0 B / 1.021 GiB

└ Labels: executiondriver=native-0.2, kernelversion=4.1.13-boot2docker, operatingsystem=Boot2Docker 1.9.1 (TCL 6.4.1); master : cef800b - Fri Nov 20 19:33:59 UTC 2015, provider=virtualbox, storagedriver=aufs

CPUs: 1

Total Memory: 1.021 GiB

Name: d074fd97682e

Create a new Machine to join this cluster:

docker-machine create -d virtualbox --virtualbox-disk-size "5000" --swarm --swarm-discovery="consul://$(docker-machine ip consul-machine):8500" --engine-opt="cluster-store=consul://$(docker-machine ip consul-machine):8500" --engine-opt="cluster-advertise=eth1:2376" swarm-node-01
Running pre-create checks...
Creating machine...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Provisioning created instance...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Configuring swarm...
To see how to connect Docker to this machine, run: docker-machine env swarm-node-01

docker-machine create -d virtualbox --virtualbox-disk-size "5000" --swarm --swarm-discovery="consul://$(docker-machine ip consul-machine):8500" --engine-opt="cluster-store=consul://$(docker-machine ip consul-machine):8500" --engine-opt="cluster-advertise=eth1:2376" swarm-node-01

Running pre-create checks...

Creating machine...

Waiting for machine to be running, this may take a few minutes...

Machine is running, waiting for SSH to be available...

Detecting operating system of created instance...

Provisioning created instance...

Copying certs to the local machine directory...

Copying certs to the remote machine...

Setting Docker configuration on the remote daemon...

Configuring swarm...

To see how to connect Docker to this machine, run: docker-machine env swarm-node-01

Notice no --swarm-master is specified in this command. This ensure that the created Machines are worker nodes.

Create a second Swarm node to join this cluster:

docker-machine create -d virtualbox --virtualbox-disk-size "5000" --swarm --swarm-discovery="consul://$(docker-machine ip consul-machine):8500" --engine-opt="cluster-store=consul://$(docker-machine ip consul-machine):8500" --engine-opt="cluster-advertise=eth1:2376" swarm-node-02
Running pre-create checks...
Creating machine...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Provisioning created instance...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Configuring swarm...
To see how to connect Docker to this machine, run: docker-machine env swarm-node-02

Running pre-create checks...

Creating machine...

Waiting for machine to be running, this may take a few minutes...

Machine is running, waiting for SSH to be available...

Detecting operating system of created instance...

Provisioning created instance...

Copying certs to the local machine directory...

Copying certs to the remote machine...

Setting Docker configuration on the remote daemon...

Configuring swarm...

To see how to connect Docker to this machine, run: docker-machine env swarm-node-02

List all the created Machines:

docker-machine ls
NAME             ACTIVE   DRIVER       STATE     URL                         SWARM
consul-machine   -        virtualbox   Running   tcp://192.168.99.100:2376
swarm-master     *        virtualbox   Running   tcp://192.168.99.101:2376   swarm-master (master)
swarm-node-01    -        virtualbox   Running   tcp://192.168.99.102:2376   swarm-master
swarm-node-02    -        virtualbox   Running   tcp://192.168.99.103:2376   swarm-master

docker-machine ls

NAME ACTIVE DRIVER STATE URL SWARM

consul-machine - virtualbox Running tcp://192.168.99.100:2376

swarm-master * virtualbox Running tcp://192.168.99.101:2376 swarm-master (master)

swarm-node-01 - virtualbox Running tcp://192.168.99.102:2376 swarm-master

swarm-node-02 - virtualbox Running tcp://192.168.99.103:2376 swarm-master

The machines that are part of the cluster have cluster’s name in the SWARM column, blank otherwise. For example,consul-machine is a standalone machine where as all other machines are part of the swarm-master cluster. The Swarm master is also identified by (master) in the SWARM column.

Connect to the Swarm cluster and find some information about it:

eval "$(docker-machine env --swarm swarm-master)"
docker info

eval "$(docker-machine env --swarm swarm-master)"

docker info

Note, --swarm is specified to connect to the Swarm cluster. Otherwise the command will connect to swarm-masterMachine only.

This shows the output as:

docker info
Containers: 4
Images: 3
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 3
 swarm-master: 192.168.99.102:2376
  └ Containers: 2
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.021 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.1.13-boot2docker, operatingsystem=Boot2Docker 1.9.1 (TCL 6.4.1); master : cef800b - Fri Nov 20 19:33:59 UTC 2015, provider=virtualbox, storagedriver=aufs
 swarm-node-01: 192.168.99.103:2376
  └ Containers: 1
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.021 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.1.13-boot2docker, operatingsystem=Boot2Docker 1.9.1 (TCL 6.4.1); master : cef800b - Fri Nov 20 19:33:59 UTC 2015, provider=virtualbox, storagedriver=aufs
 swarm-node-02: 192.168.99.104:2376
  └ Containers: 1
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.021 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.1.13-boot2docker, operatingsystem=Boot2Docker 1.9.1 (TCL 6.4.1); master : cef800b - Fri Nov 20 19:33:59 UTC 2015, provider=virtualbox, storagedriver=aufs
CPUs: 3
Total Memory: 3.064 GiB
Name: d074fd97682e

docker info

Containers: 4

Images: 3

Role: primary

Strategy: spread

Filters: health, port, dependency, affinity, constraint

Nodes: 3

swarm-master: 192.168.99.102:2376

└ Containers: 2

└ Reserved CPUs: 0 / 1

└ Reserved Memory: 0 B / 1.021 GiB

swarm-node-01: 192.168.99.103:2376

└ Containers: 1

└ Reserved CPUs: 0 / 1

└ Reserved Memory: 0 B / 1.021 GiB

swarm-node-02: 192.168.99.104:2376

└ Containers: 1

└ Reserved CPUs: 0 / 1

└ Reserved Memory: 0 B / 1.021 GiB

CPUs: 3

Total Memory: 3.064 GiB

Name: d074fd97682e

There are 3 nodes – one Swarm master and 2 Swarm worker nodes. There is a total of 4 containers running in this cluster – one Swarm agent on master and each node, and there is an additional swarm-agent-master running on the master. This can be verified by connecting to the master and listing all the containers.

List nodes in the cluster with the following command:

docker run swarm list consul://$(docker-machine ip consul-machine):8500
192.168.99.102:2376
192.168.99.103:2376
192.168.99.104:2376

docker run swarm list consul://$(docker-machine ip consul-machine):8500

192.168.99.102:2376

192.168.99.103:2376

192.168.99.104:2376

Start Application Environment using Docker Compose

Make sure you are connected to the cluster by giving the command eval "$(docker-machine env --swarm swarm-master)".

List all the networks created by Docker so far:

docker network ls
NETWORK ID          NAME                   DRIVER
33a619ddc5d2        swarm-node-02/bridge   bridge
e0b73c96ffec        swarm-node-02/none     null
b315e67f0363        swarm-node-02/host     host
879d6167be47        swarm-master/bridge    bridge
f771ddc7d957        swarm-node-01/none     null
e042754df336        swarm-node-01/host     host
d2f3b512f9dc        swarm-node-01/bridge   bridge
5b5bcf135d7b        swarm-master/none      null
fffc34eae907        swarm-master/host      host

docker network ls

NETWORK ID NAME DRIVER

33a619ddc5d2 swarm-node-02/bridge bridge

e0b73c96ffec swarm-node-02/none null

b315e67f0363 swarm-node-02/host host

879d6167be47 swarm-master/bridge bridge

f771ddc7d957 swarm-node-01/none null

e042754df336 swarm-node-01/host host

d2f3b512f9dc swarm-node-01/bridge bridge

5b5bcf135d7b swarm-master/none null

fffc34eae907 swarm-master/host host

Docker create three networks for each host automatically:

Network Name	Purpose
`bridge`	Default network that containers connect to. This is `docker0` network in all Docker installations.
`none`	Container-specific networking stack
`host`	Adds a container on hosts networking stack. Network configuration is identical to the host.

This explains a total of nine networks, three for each node, as shown in this Swarm cluster.

Use Compose file to start WildFly and Couchbase:

mycouchbase:
  container_name: "db"
  image: couchbase/server
  ports:
    - 8091:8091
    - 8092:8092
    - 8093:8093
    - 11210:11210
mywildfly:
  image: arungupta/wildfly-admin
  environment:
    - COUCHBASE_URI=db
  ports:
    - 8080:8080
    - 9990:9990

mycouchbase:

container_name: "db"

image: couchbase/server

ports:

- 8091:8091

- 8092:8092

- 8093:8093

- 11210:11210

mywildfly:

image: arungupta/wildfly-admin

environment:

- COUCHBASE_URI=db

ports:

- 8080:8080

- 9990:9990

In this Compose file:

Couchbase service has a custom container name defined by container_name. This name is used when creating a new environment variable COUCHBASE_URI during WildFly startup.
arungupta/wildfly-admin image is used as it binds WildFly’s management to all network interfaces, and in addition also exposes port 9990. This enables WildFly Maven Plugin to be used to deploy the application.Source for this file is at https://github.com/arun-gupta/docker-images/blob/master/wildfly-couchbase-javaee7/docker-compose.yml.

This application environment can be started as:

docker-compose --x-networking up -d
Creating network "wildflycouchbasejavaee7" with driver "None"
Pulling mywildfly (arungupta/wildfly-admin:latest)...
swarm-node-02: Pulling arungupta/wildfly-admin:latest... : downloaded
swarm-master: Pulling arungupta/wildfly-admin:latest... : downloaded
swarm-node-01: Pulling arungupta/wildfly-admin:latest... : downloaded
Creating wildflycouchbasejavaee7_mywildfly_1
Pulling mycouchbase (couchbase/server:latest)...
swarm-node-02: Pulling couchbase/server:latest... : downloaded
swarm-master: Pulling couchbase/server:latest... : downloaded
swarm-node-01: Pulling couchbase/server:latest... : downloaded
Creating db

docker-compose --x-networking up -d

Creating network "wildflycouchbasejavaee7" with driver "None"

Pulling mywildfly (arungupta/wildfly-admin:latest)...

swarm-node-02: Pulling arungupta/wildfly-admin:latest... : downloaded

swarm-master: Pulling arungupta/wildfly-admin:latest... : downloaded

swarm-node-01: Pulling arungupta/wildfly-admin:latest... : downloaded

Creating wildflycouchbasejavaee7_mywildfly_1

Pulling mycouchbase (couchbase/server:latest)...

swarm-node-02: Pulling couchbase/server:latest... : downloaded

swarm-master: Pulling couchbase/server:latest... : downloaded

swarm-node-01: Pulling couchbase/server:latest... : downloaded

Creating db

--x-networking creates an overlay network for the Swarm cluster. This can be verified by listing networks again:

docker network ls
NETWORK ID          NAME                            DRIVER
5e93fc34b4d9        swarm-node-01/docker_gwbridge   bridge
1c041242f51d        wildflycouchbasejavaee7         overlay
cc8697c6ce13        swarm-master/docker_gwbridge    bridge
f771ddc7d957        swarm-node-01/none              null
879d6167be47        swarm-master/bridge             bridge
5b5bcf135d7b        swarm-master/none               null
fffc34eae907        swarm-master/host               host
e042754df336        swarm-node-01/host              host
d2f3b512f9dc        swarm-node-01/bridge            bridge
33a619ddc5d2        swarm-node-02/bridge            bridge
e0b73c96ffec        swarm-node-02/none              null
b315e67f0363        swarm-node-02/host              host

docker network ls

NETWORK ID NAME DRIVER

5e93fc34b4d9 swarm-node-01/docker_gwbridge bridge

1c041242f51d wildflycouchbasejavaee7 overlay

cc8697c6ce13 swarm-master/docker_gwbridge bridge

f771ddc7d957 swarm-node-01/none null

879d6167be47 swarm-master/bridge bridge

5b5bcf135d7b swarm-master/none null

fffc34eae907 swarm-master/host host

e042754df336 swarm-node-01/host host

d2f3b512f9dc swarm-node-01/bridge bridge

33a619ddc5d2 swarm-node-02/bridge bridge

e0b73c96ffec swarm-node-02/none null

b315e67f0363 swarm-node-02/host host

Three new networks are created:

Containers connected to the multi-host network are automatically connected to the docker_gwbridge network. This network allows the containers to have external connectivity outside of their cluster, and is created on each worker node.

A new overlay network wildflycouchbasejavaee7 is created. Connect to different Swarm nodes and check that the overlay network exists on them.

Lets begin with master:

eval "$(docker-machine env swarm-master)"
docker network ls
NETWORK ID          NAME                      DRIVER
1c041242f51d        wildflycouchbasejavaee7   overlay
879d6167be47        bridge                    bridge
5b5bcf135d7b        none                      null
fffc34eae907        host                      host
cc8697c6ce13        docker_gwbridge           bridge

eval "$(docker-machine env swarm-master)"

docker network ls

NETWORK ID NAME DRIVER

1c041242f51d wildflycouchbasejavaee7 overlay

879d6167be47 bridge bridge

5b5bcf135d7b none null

fffc34eae907 host host

cc8697c6ce13 docker_gwbridge bridge

Next, with swarm-node-01:

eval "$(docker-machine env swarm-node-01)"
docker network ls
NETWORK ID          NAME                      DRIVER
1c041242f51d        wildflycouchbasejavaee7   overlay
d2f3b512f9dc        bridge                    bridge
f771ddc7d957        none                      null
e042754df336        host                      host
5e93fc34b4d9        docker_gwbridge           bridge

eval "$(docker-machine env swarm-node-01)"

docker network ls

NETWORK ID NAME DRIVER

1c041242f51d wildflycouchbasejavaee7 overlay

d2f3b512f9dc bridge bridge

f771ddc7d957 none null

e042754df336 host host

5e93fc34b4d9 docker_gwbridge bridge

Finally, with swarm-node-02:

eval "$(docker-machine env swarm-node-02)"
docker network ls
NETWORK ID          NAME                      DRIVER
1c041242f51d        wildflycouchbasejavaee7   overlay
e0b73c96ffec        none                      null
b315e67f0363        host                      host
33a619ddc5d2        bridge                    bridge

eval "$(docker-machine env swarm-node-02)"

docker network ls

NETWORK ID NAME DRIVER

1c041242f51d wildflycouchbasejavaee7 overlay

e0b73c96ffec none null

b315e67f0363 host host

33a619ddc5d2 bridge bridge

As seen, wildflycouchbasejavaee7 overlay network exists on all Machines. This confirms that the overlay network created for Swarm cluster was added to each host in the cluster. docker_gwbridge only exists on Machines that have application containers running.

Configure Application and Database

Clone https://github.com/arun-gupta/couchbase-javaee.git. This workspace contains a simple Java EE application that is deployed on WildFly and provides a REST API over travel-sample bucket in Couchbase.

Couchbase server can be configured using REST API. The application contains a Maven profile that allows to configure Couchbase server with travel-sample bucket. This can be invoked as:

mvn install -Pcouchbase -Ddocker.host=$(docker-machine ip swarm-master)

. . .

* Server auth using Basic with user 'Administrator'
> POST /sampleBuckets/install HTTP/1.1
> Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==

. . .

} [data not shown]
* upload completely sent off: 17 out of 17 bytes
< HTTP/1.1 202 Accepted
* Server Couchbase Server is not blacklisted
< Server: Couchbase Server

. . .

mvn install -Pcouchbase -Ddocker.host=$(docker-machine ip swarm-master)

. . .

* Server auth using Basic with user 'Administrator'

> POST /sampleBuckets/install HTTP/1.1

> Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==

. . .

} [data not shown]

* upload completely sent off: 17 out of 17 bytes

< HTTP/1.1 202 Accepted

* Server Couchbase Server is not blacklisted

< Server: Couchbase Server

. . .

Deploy the application to WildFly by specifying three parameters:

Host IP address where WildFly is running
Username of a user in WildFly’s administrative realm
Password of the user specified in WildFly’s administrative realm

mvn install -Pwildfly -Dwildfly.hostname=$(docker-machine ip swarm-node-01) -Dwildfly.username=admin -Dwildfly.password=Admin#007

. . .

Nov 29, 2015 12:11:14 AM org.xnio.Xnio <clinit>
INFO: XNIO version 3.3.1.Final
Nov 29, 2015 12:11:14 AM org.xnio.nio.NioXnio <clinit>
INFO: XNIO NIO Implementation Version 3.3.1.Final
Nov 29, 2015 12:11:15 AM org.jboss.remoting3.EndpointImpl <clinit>
INFO: JBoss Remoting version 4.0.9.Final
[INFO] Authenticating against security realm: ManagementRealm
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

. . .

mvn install -Pwildfly -Dwildfly.hostname=$(docker-machine ip swarm-node-01) -Dwildfly.username=admin -Dwildfly.password=Admin#007

. . .

Nov 29, 2015 12:11:14 AM org.xnio.Xnio <clinit>

INFO: XNIO version 3.3.1.Final

Nov 29, 2015 12:11:14 AM org.xnio.nio.NioXnio <clinit>

INFO: XNIO NIO Implementation Version 3.3.1.Final

Nov 29, 2015 12:11:15 AM org.jboss.remoting3.EndpointImpl <clinit>

INFO: JBoss Remoting version 4.0.9.Final

[INFO] Authenticating against security realm: ManagementRealm

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

. . .

Access Application

Now that WildFly and Couchbase server have started, lets access the application. You need to specify IP address of the Machine where WildFly is running:

curl http://$(docker-machine ip swarm-node-01):8080/couchbase-javaee/resources/airline
[{"travel-sample":{"id":10123,"iata":"TQ","icao":"TXW","name":"Texas Wings","callsign":"TXW","type":"airline","country":"United States"}}, {"travel-sample":{"id":10642,"iata":null,"icao":"JRB","name":"Jc royal.britannica","callsign":null,"type":"airline","country":"United Kingdom"}}, {"travel-sample":{"id":112,"iata":"5W","icao":"AEU","name":"Astraeus","callsign":"FLYSTAR","type":"airline","country":"United Kingdom"}}, {"travel-sample":{"id":1355,"iata":"BA","icao":"BAW","name":"British Airways","callsign":"SPEEDBIRD","type":"airline","country":"United Kingdom"}}, {"travel-sample":{"id":10765,"iata":"K5","icao":"SQH","name":"SeaPort Airlines","callsign":"SASQUATCH","type":"airline","country":"United States"}}, {"travel-sample":{"id":13633,"iata":"WQ","icao":"PQW","name":"PanAm World Airways","callsign":null,"type":"airline","country":"United States"}}, {"travel-sample":{"id":139,"iata":"SB","icao":"ACI","name":"Air Caledonie International","callsign":"AIRCALIN","type":"airline","country":"France"}}, {"travel-sample":{"id":13391,"iata":"-+","icao":"--+","name":"U.S. Air","callsign":null,"type":"airline","country":"United States"}}, {"travel-sample":{"id":1191,"iata":"UU","icao":"REU","name":"Air Austral","callsign":"REUNION","type":"airline","country":"France"}}, {"travel-sample":{"id":1316,"iata":"FL","icao":"TRS","name":"AirTran Airways","callsign":"CITRUS","type":"airline","country":"United States"}}]

curl http://$(docker-machine ip swarm-node-01):8080/couchbase-javaee/resources/airline

[{"travel-sample":{"id":10123,"iata":"TQ","icao":"TXW","name":"Texas Wings","callsign":"TXW","type":"airline","country":"United States"}}, {"travel-sample":{"id":10642,"iata":null,"icao":"JRB","name":"Jc royal.britannica","callsign":null,"type":"airline","country":"United Kingdom"}}, {"travel-sample":{"id":112,"iata":"5W","icao":"AEU","name":"Astraeus","callsign":"FLYSTAR","type":"airline","country":"United Kingdom"}}, {"travel-sample":{"id":1355,"iata":"BA","icao":"BAW","name":"British Airways","callsign":"SPEEDBIRD","type":"airline","country":"United Kingdom"}}, {"travel-sample":{"id":10765,"iata":"K5","icao":"SQH","name":"SeaPort Airlines","callsign":"SASQUATCH","type":"airline","country":"United States"}}, {"travel-sample":{"id":13633,"iata":"WQ","icao":"PQW","name":"PanAm World Airways","callsign":null,"type":"airline","country":"United States"}}, {"travel-sample":{"id":139,"iata":"SB","icao":"ACI","name":"Air Caledonie International","callsign":"AIRCALIN","type":"airline","country":"France"}}, {"travel-sample":{"id":13391,"iata":"-+","icao":"--+","name":"U.S. Air","callsign":null,"type":"airline","country":"United States"}}, {"travel-sample":{"id":1191,"iata":"UU","icao":"REU","name":"Air Austral","callsign":"REUNION","type":"airline","country":"France"}}, {"travel-sample":{"id":1316,"iata":"FL","icao":"TRS","name":"AirTran Airways","callsign":"CITRUS","type":"airline","country":"United States"}}]

Complete set of REST API for this application is documented at github.com/arun-gupta/couchbase-javaee.

Latest instructions at Docker for Java Developers.

Enjoy!

JavaOne4Kids 2015 Wrapup – Devoxx4Kids and Oracle Academy Together!

December 2, 2015conferences, java, kidsdevoxx4kids, java, javaone, minecraftarungupta

JavaOne4Kids is focused on promoting technology to next generation of developers; kids who want to learn more about programming, robotics and engineering.

Oracle Academy collaborated with Devoxx4Kids to bring kids content that includes several topics like Minecraft Modding, Java, Python, Scratch, Raspberry Pi, Arduino, NAO robot, LEGO Mindstorms, Greenfoot, Alice, and others at JavaOne 2015.

The attendance grew 3x from last year and it was certainly very heartening to see that!

If you live in/around San Francisco Bay Area, and want a more continued experience through out the year, then its highly recommend to join meetup.com/Devoxx4Kids-BayArea/!

Here are some statistics from the event:

A survey was sent to the attendees and some of them responded back. 95% of responses rated were happy with the event:

javaone4kids-2015-rate

90%+ would recommend JavaOne4Kids to a friend:

javaone4kids-2015-recommend

Instructors seem to have done a good job with 97% presenting in good, very good, and excellent way:

javaone4kids-2015-clear-way

Minecraft Modding continues to be the top rated workshop:

javaone4kids-2015-course

Here are some pictures from the event:

Check out the complete album:

JavaOne4Kids 2015 Album

Picture is worth a thousand words, and a video is worth a million words. Check out kids in action from the event, and then subsequently in JavaOne Community Keynote:

It takes a village to run an event like this. This was certainly not possible without the impeccable support from Oracle team, instructors, and volunteers who helped us through out the event!

Do we expect these kids to come back to again next year? Yes, absolutely!

At least, 88% of them want to come back

javaone4kids-2015-another-event

Don’t forget to join the local meetup.com/devoxx4kids-bayarea for local events in Bay Area.

Docker Networking with Couchbase and WildFly

November 26, 2015containers, couchbase, javaee, wildflycouchbase, docker, javaee, maven, rest, wildflyarungupta

Docker Multi-Host networking allows you to create virtual networks and attach containers to them so you can create the network topology that is right for your application. This blog will show how to use it with Docker Compose.

CRUD Java Application with Couchbase, Java EE, and WildFly explained how to use a Java EE application to provide a CRUD/REST interface on a data bucket in Couchbase. It required to manually download and run WildFly. The blog also used Couchbase server using Docker and required manual configuration to load travel-sample bucket.

Configure Couchbase Docker Container using REST API explained how to use Couchbase REST API to configure Couchbase Server.

This blog will remove the explicit download of WildFly and manual configuration of Couchbase server:

Use Docker Compose to start WildFly and Couchbase (no download required)
Use a Maven profile to configure Couchbase server (no manual configuration required)
Uses Docker multi-host networking so that WildFly and Couchbase server can talk to each other

Lets get started!

Start Couchbase and WildFly using Docker Multi-Host Networking and Compose

Start WildFly and Couchbase server using docker-compose.yml file from github.com/arun-gupta/docker-images/blob/master/wildfly-couchbase-javaee7/docker-compose.yml:

mycouchbase:
  container_name: "db"
  image: couchbase/server
  ports:
    - 8091:8091
    - 8092:8092 
    - 8093:8093 
    - 11210:11210
mywildfly:
  image: arungupta/wildfly-admin
  environment:
    - COUCHBASE_URI=db
  ports:
    - 8080:8080
    - 9990:9990

mycouchbase:

container_name: "db"

image: couchbase/server

ports:

- 8091:8091

- 8092:8092

- 8093:8093

- 11210:11210

mywildfly:

image: arungupta/wildfly-admin

environment:

- COUCHBASE_URI=db

ports:

- 8080:8080

- 9990:9990

arungupta/wildfly-admin image is used as it binds WildFly’s management to all network interfaces, and in addition also exposes port 9990. This enables WildFly Maven Plugin to be used to deploy the application.

container_name is specified for Couchbase service and referred in WildFly service using COUCHBASE_URI. This is then used to connect to Couchbase from the Java EE application.

The application environment is started as:

docker-compose --x-networking up -d
Creating network "wildflycouchbasejavaee7" with driver "None"
Creating wildflycouchbasejavaee7_mywildfly_1
Creating db

docker-compose --x-networking up -d

Creating network "wildflycouchbasejavaee7" with driver "None"

Creating wildflycouchbasejavaee7_mywildfly_1

Creating db

--x-networking is an experimental switch added to Docker Compose 1.9 that allows to create a bridge or an overlay network. By default, it creates a bridge network that works on a single host. The network created can be seen as:

docker network ls
NETWORK ID          NAME                      DRIVER
968bde2b7745        bridge                    bridge              
79e937bcf1f6        none                      null                
d06a4bd59a4b        host                      host                
5e26a9a205ff        wildflycouchbasejavaee7   bridge

docker network ls

NETWORK ID NAME DRIVER

968bde2b7745 bridge bridge

79e937bcf1f6 none null

d06a4bd59a4b host host

5e26a9a205ff wildflycouchbasejavaee7 bridge

Issue 2221 provide more explanation about the default networks created. wildflycouchbasejavaee7 is the new bridge network created for our application. Issue #2345 provide some details about incorrect driver name in the output message.

Configure Couchbase Server

Clone couchbase-javaee repo:

https://github.com/arun-gupta/couchbase-javaee

https://github.com/arun-gupta/couchbase-javaee

Configure Couchbase server:

mvn install -Pcouchbase
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building couchbase-javaee 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 


. . .

* Server auth using Basic with user 'Administrator'
> POST /sampleBuckets/install HTTP/1.1
> Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==
> User-Agent: curl/7.37.1
> Host: 192.168.99.100:8091

. . .

< Cache-Control: no-cache
< 
{ [data not shown]
100    19  100     2  100    17     50    425 --:--:-- --:--:-- --:--:--   435
* Connection #0 to host 192.168.99.100 left intact
[][INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

mvn install -Pcouchbase

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building couchbase-javaee 1.0-SNAPSHOT

[INFO] ------------------------------------------------------------------------

[INFO]

. . .

* Server auth using Basic with user 'Administrator'

> POST /sampleBuckets/install HTTP/1.1

> Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==

> User-Agent: curl/7.37.1

> Host: 192.168.99.100:8091

. . .

< Cache-Control: no-cache

{ [data not shown]

100 19 100 2 100 17 50 425 --:--:-- --:--:-- --:--:-- 435

* Connection #0 to host 192.168.99.100 left intact

[][INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

exec-maven-plugin is used to invoke REST API and configure Couchbase server and is configured in a Maven profile. Make sure to setup docker.host property in pom.xml.

Deploy the application to WildFly:

mvn install -Pwildfly -Dwildfly.hostname=192.168.99.100 -Dwildfly.username=admin -Dwildfly.password=Admin#007
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building couchbase-javaee 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------

. . .

Nov 26, 2015 5:17:39 AM org.xnio.Xnio <clinit>
INFO: XNIO version 3.3.1.Final
Nov 26, 2015 5:17:39 AM org.xnio.nio.NioXnio <clinit>
INFO: XNIO NIO Implementation Version 3.3.1.Final
Nov 26, 2015 5:17:39 AM org.jboss.remoting3.EndpointImpl <clinit>
INFO: JBoss Remoting version 4.0.9.Final
[INFO] Authenticating against security realm: ManagementRealm
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

mvn install -Pwildfly -Dwildfly.hostname=192.168.99.100 -Dwildfly.username=admin -Dwildfly.password=Admin#007

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building couchbase-javaee 1.0-SNAPSHOT

[INFO] ------------------------------------------------------------------------

. . .

Nov 26, 2015 5:17:39 AM org.xnio.Xnio <clinit>

INFO: XNIO version 3.3.1.Final

Nov 26, 2015 5:17:39 AM org.xnio.nio.NioXnio <clinit>

INFO: XNIO NIO Implementation Version 3.3.1.Final

Nov 26, 2015 5:17:39 AM org.jboss.remoting3.EndpointImpl <clinit>

INFO: JBoss Remoting version 4.0.9.Final

[INFO] Authenticating against security realm: ManagementRealm

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

Make sure to specify the correct host on CLI. In this case, this is the IP address obtained using docker-machine ip default.

Invoke the Application

Invoke the REST endpoint using cURL:

curl -v http://192.168.99.100:8080/couchbase-javaee/resources/airline
* Hostname was NOT found in DNS cache
*   Trying 192.168.99.100...
* Connected to 192.168.99.100 (192.168.99.100) port 8080 (#0)
> GET /couchbase-javaee/resources/airline HTTP/1.1
> User-Agent: curl/7.37.1
> Host: 192.168.99.100:8080
> Accept: */*
> 
< HTTP/1.1 200 OK
< Connection: keep-alive
< X-Powered-By: Undertow/1
* Server WildFly/8 is not blacklisted
< Server: WildFly/8
< Content-Type: application/octet-stream
< Content-Length: 1393
< Date: Thu, 26 Nov 2015 13:18:36 GMT
< 
* Connection #0 to host 192.168.99.100 left intact
[{"travel-sample":{"id":112,"iata":"5W","icao":"AEU","name":"Astraeus","callsign":"FLYSTAR","type":"airline","country":"United Kingdom"}}, {"travel-sample":{"id":109,"iata":"KO","icao":"AER","name":"Alaska Central Express","callsign":"ACE AIR","type":"airline","country":"United States"}}, {"travel-sample":{"id":1191,"iata":"UU","icao":"REU","name":"Air Austral","callsign":"REUNION","type":"airline","country":"France"}}, {"travel-sample":{"id":1203,"iata":"A5","icao":"RLA","name":"Airlinair","callsign":"AIRLINAIR","type":"airline","country":"France"}}, {"travel-sample":{"id":10642,"iata":null,"icao":"JRB","name":"Jc royal.britannica","callsign":null,"type":"airline","country":"United Kingdom"}}, {"travel-sample":{"id":13633,"iata":"WQ","icao":"PQW","name":"PanAm World Airways","callsign":null,"type":"airline","country":"United States"}}, {"travel-sample":{"id":137,"iata":"AF","icao":"AFR","name":"Air France","callsign":"AIRFRANS","type":"airline","country":"France"}}, {"travel-sample":{"id":10748,"iata":"ZQ","icao":"LOC","name":"Locair","callsign":"LOCAIR","type":"airline","country":"United States"}}, {"travel-sample":{"id":10,"iata":"Q5","icao":"MLA","name":"40-Mile Air","callsign":"MILE-AIR","type":"airline","country":"United States"}}, {"travel-sample":{"id":10226,"iata":"A1","icao":"A1F","name":"Atifly","callsign":"atifly","type":"airline","country":"United States"}}]

curl -v http://192.168.99.100:8080/couchbase-javaee/resources/airline

* Hostname was NOT found in DNS cache

* Trying 192.168.99.100...

* Connected to 192.168.99.100 (192.168.99.100) port 8080 (#0)

> GET /couchbase-javaee/resources/airline HTTP/1.1

> User-Agent: curl/7.37.1

> Host: 192.168.99.100:8080

> Accept: */*

< HTTP/1.1 200 OK

< Connection: keep-alive

< X-Powered-By: Undertow/1

* Server WildFly/8 is not blacklisted

< Server: WildFly/8

< Content-Type: application/octet-stream

< Content-Length: 1393

< Date: Thu, 26 Nov 2015 13:18:36 GMT

* Connection #0 to host 192.168.99.100 left intact

[{"travel-sample":{"id":112,"iata":"5W","icao":"AEU","name":"Astraeus","callsign":"FLYSTAR","type":"airline","country":"United Kingdom"}}, {"travel-sample":{"id":109,"iata":"KO","icao":"AER","name":"Alaska Central Express","callsign":"ACE AIR","type":"airline","country":"United States"}}, {"travel-sample":{"id":1191,"iata":"UU","icao":"REU","name":"Air Austral","callsign":"REUNION","type":"airline","country":"France"}}, {"travel-sample":{"id":1203,"iata":"A5","icao":"RLA","name":"Airlinair","callsign":"AIRLINAIR","type":"airline","country":"France"}}, {"travel-sample":{"id":10642,"iata":null,"icao":"JRB","name":"Jc royal.britannica","callsign":null,"type":"airline","country":"United Kingdom"}}, {"travel-sample":{"id":13633,"iata":"WQ","icao":"PQW","name":"PanAm World Airways","callsign":null,"type":"airline","country":"United States"}}, {"travel-sample":{"id":137,"iata":"AF","icao":"AFR","name":"Air France","callsign":"AIRFRANS","type":"airline","country":"France"}}, {"travel-sample":{"id":10748,"iata":"ZQ","icao":"LOC","name":"Locair","callsign":"LOCAIR","type":"airline","country":"United States"}}, {"travel-sample":{"id":10,"iata":"Q5","icao":"MLA","name":"40-Mile Air","callsign":"MILE-AIR","type":"airline","country":"United States"}}, {"travel-sample":{"id":10226,"iata":"A1","icao":"A1F","name":"Atifly","callsign":"atifly","type":"airline","country":"United States"}}]

Complete set of REST endpoints are documented at CRUD Java Application with Couchbase, Java EE and WildFly. They are listed here for convenience:

GET a single airline:

curl -v http://192.168.99.100:8080/couchbase-javaee/resources/airline/139

curl -v http://192.168.99.100:8080/couchbase-javaee/resources/airline/139

Create a new airline using POST:

curl -v -H "Content-Type: application/json" -X POST -d '{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","type":"airline"}' http://192.168.99.100:8080/couchbase-javaee/resources/airline

curl -v -H "Content-Type: application/json" -X POST -d '{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","type":"airline"}' http://192.168.99.100:8080/couchbase-javaee/resources/airline

Update an existing airline using PUT:

curl -v -H "Content-Type: application/json" -X PUT -d '{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlin Air","icao":"RLA","type":"airline","id": "19810"}' http://192.168.99.100:8080/couchbase-javaee/resources/airline/19810

curl -v -H "Content-Type: application/json" -X PUT -d '{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlin Air","icao":"RLA","type":"airline","id": "19810"}' http://192.168.99.100:8080/couchbase-javaee/resources/airline/19810

Delete an existing airline using DELETE:

curl -v -X DELETE http://192.168.99.100:8080/couchbase-javaee/resources/airline/19810

curl -v -X DELETE http://192.168.99.100:8080/couchbase-javaee/resources/airline/19810

Enjoy!

CRUD Java Application with Couchbase, Java EE and WildFly

November 18, 2015containers, couchbase, javaee, wildflycouchbase, crud, docker, javaee, wildflyarungupta

Couchbase is an open-source, NoSQL, document database. It allows to access, index, and query JSON documents while taking advantage of integrated distributed caching for high performance data access.

REST with Couchbase

The application will use curl to issue REST commands to a JAX-RS endpoint deployed on WildFly. These commands will then perform CRUD operations on travel-sample bucket in Couchbase. N1QL (SQL query language for JSON) will be used to communicate with Couchbase to retrieve results. Both the “builder pattern” and raw N1QL commands will be used.

TL;DR

Complete source code and instructions for the sample are available at github.com/arun-gupta/couchbase-javaee.

Lets get started!

Run Couchbase Server

Couchbase server can be easily downloaded from Couchbase Server Downloads page. In a containerized world, its a lot easier to fire up a Couchbase server using Docker.

If Docker is configured on your machine then the easiest way is to use Docker Compose for Couchbase:

mycouchbase:
  name: mycouchbase
  image: couchbase/server
  volumes:
    - ~/couchbase:/opt/couchbase/var
  ports:
    - 8091:8091
    - 8092:8092 
    - 8093:8093 
    - 11210:11210

mycouchbase:

image: couchbase/server

volumes:

- ~/couchbase:/opt/couchbase/var

ports:

- 8091:8091

- 8092:8092

- 8093:8093

- 11210:11210

Starting up the application server shows:

> docker-compose up -d
Creating couchbaseserver_mycouchbase_1

> docker-compose up -d

Creating couchbaseserver_mycouchbase_1

And then the logs can be seen as:

> docker-compose logs
Attaching to couchbaseserver_mycouchbase_1
mycouchbase_1 | Starting Couchbase Server -- Web UI available at http://<ip>:8091

> docker-compose logs

Attaching to couchbaseserver_mycouchbase_1

mycouchbase_1 | Starting Couchbase Server -- Web UI available at http://<ip>:8091

The database needs to be configured and is explained at Configure Couchbase Server. Make sure to install travel-sample bucket.

Deploy the Java EE Application on WildFly

Download WildFly 9.0.2 , unzip, and start WildFly application server as ./wildfly-9.0.0.Final/bin/standalone.sh.
Git clone the repo: git clone https://github.com/arun-gupta/couchbase-javaee.git
Change directory cd couchbase-javaee
Deploy the application to WildFly: mvn install -Pwildfly.

The application uses Java SDK for Couchbase by importing the following Maven coordinates:

<dependency>
    <groupId>com.couchbase.client</groupId>
    <artifactId>java-client</artifactId>
    <version>2.2.1</version>
</dependency>

<groupId>com.couchbase.client</groupId>

<artifactId>java-client</artifactId>

</dependency>

Invoke the REST Endpoints Using cURL

GET Airline resources (limit to 10)

Lets query the database to list 10 Airline resources.

Request

~ > curl -v http://localhost:8080/couchbase-javaee/resources/airline
* Hostname was NOT found in DNS cache
*   Trying ::1...
* connect to ::1 port 8080 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /couchbase-javaee/resources/airline HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:8080
> Accept: */*
>

~ > curl -v http://localhost:8080/couchbase-javaee/resources/airline

* Hostname was NOT found in DNS cache

* Trying ::1...

* connect to ::1 port 8080 failed: Connection refused

* Trying 127.0.0.1...

* Connected to localhost (127.0.0.1) port 8080 (#0)

> GET /couchbase-javaee/resources/airline HTTP/1.1

> User-Agent: curl/7.37.1

> Host: localhost:8080

> Accept: */*

Response

< HTTP/1.1 200 OK
< Connection: keep-alive
< X-Powered-By: Undertow/1
* Server WildFly/9 is not blacklisted
< Server: WildFly/9
< Content-Type: application/octet-stream
< Content-Length: 1415
< Date: Wed, 18 Nov 2015 21:19:15 GMT
< 
* Connection #0 to host localhost left intact
[{"travel-sample":{"country":"France","iata":"SB","callsign":"AIRCALIN","name":"Air Caledonie International","icao":"ACI","id":139,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"WQ","callsign":null,"name":"PanAm World Airways","icao":"PQW","id":13633,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":"BA","callsign":"SPEEDBIRD","name":"British Airways","icao":"BAW","id":1355,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"FL","callsign":"CITRUS","name":"AirTran Airways","icao":"TRS","id":1316,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"-+","callsign":null,"name":"U.S. Air","icao":"--+","id":13391,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"Q5","callsign":"MILE-AIR","name":"40-Mile Air","icao":"MLA","id":10,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"AF","callsign":"AIRFRANS","name":"Air France","icao":"AFR","id":137,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"K5","callsign":"SASQUATCH","name":"SeaPort Airlines","icao":"SQH","id":10765,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","id":1203,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":"5W","callsign":"FLYSTAR","name":"Astraeus","icao":"AEU","id":112,"type":"airline"}}]

< HTTP/1.1 200 OK

< Connection: keep-alive

< X-Powered-By: Undertow/1

* Server WildFly/9 is not blacklisted

< Server: WildFly/9

< Content-Type: application/octet-stream

< Content-Length: 1415

< Date: Wed, 18 Nov 2015 21:19:15 GMT

* Connection #0 to host localhost left intact

[{"travel-sample":{"country":"France","iata":"SB","callsign":"AIRCALIN","name":"Air Caledonie International","icao":"ACI","id":139,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"WQ","callsign":null,"name":"PanAm World Airways","icao":"PQW","id":13633,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":"BA","callsign":"SPEEDBIRD","name":"British Airways","icao":"BAW","id":1355,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"FL","callsign":"CITRUS","name":"AirTran Airways","icao":"TRS","id":1316,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"-+","callsign":null,"name":"U.S. Air","icao":"--+","id":13391,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"Q5","callsign":"MILE-AIR","name":"40-Mile Air","icao":"MLA","id":10,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"AF","callsign":"AIRFRANS","name":"Air France","icao":"AFR","id":137,"type":"airline"}}, {"travel-sample":{"country":"United States","iata":"K5","callsign":"SASQUATCH","name":"SeaPort Airlines","icao":"SQH","id":10765,"type":"airline"}}, {"travel-sample":{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","id":1203,"type":"airline"}}, {"travel-sample":{"country":"United Kingdom","iata":"5W","callsign":"FLYSTAR","name":"Astraeus","icao":"AEU","id":112,"type":"airline"}}]

The N1QL query for this is written as:

N1qlQuery query = N1qlQuery
                .simple(select("*")
                .from(i(database.getBucket().name()))
                .limit(10));

N1qlQuery query = N1qlQuery

.simple(select("*")

.from(i(database.getBucket().name()))

.limit(10));

And can also be alternatively written as:

SELECT * FROM `travel-sample` LIMIT 10

SELECT * FROM `travel-sample` LIMIT 10

You may optionally update the code to include ORDER BY clause as shown in N1QL Tutorial.

GET one Airline resource

Use id attribute to query a single Airline resource

Request

~ > curl -v http://localhost:8080/couchbase-javaee/resources/airline/139
* Hostname was NOT found in DNS cache
*   Trying ::1...
* connect to ::1 port 8080 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /couchbase-javaee/resources/airline/139 HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:8080
> Accept: */*
>

~ > curl -v http://localhost:8080/couchbase-javaee/resources/airline/139

* Hostname was NOT found in DNS cache

* Trying ::1...

* connect to ::1 port 8080 failed: Connection refused

* Trying 127.0.0.1...

* Connected to localhost (127.0.0.1) port 8080 (#0)

> GET /couchbase-javaee/resources/airline/139 HTTP/1.1

> User-Agent: curl/7.37.1

> Host: localhost:8080

> Accept: */*

Response

< HTTP/1.1 200 OK
< Connection: keep-alive
< X-Powered-By: Undertow/1
* Server WildFly/9 is not blacklisted
< Server: WildFly/9
< Content-Type: application/octet-stream
< Content-Length: 148
< Date: Wed, 18 Nov 2015 21:23:34 GMT
< 
* Connection #0 to host localhost left intact
{"travel-sample":{"country":"France","iata":"SB","callsign":"AIRCALIN","name":"Air Caledonie International","icao":"ACI","id":139,"type":"airline"}}

< HTTP/1.1 200 OK

< Connection: keep-alive

< X-Powered-By: Undertow/1

* Server WildFly/9 is not blacklisted

< Server: WildFly/9

< Content-Type: application/octet-stream

< Content-Length: 148

< Date: Wed, 18 Nov 2015 21:23:34 GMT

* Connection #0 to host localhost left intact

{"travel-sample":{"country":"France","iata":"SB","callsign":"AIRCALIN","name":"Air Caledonie International","icao":"ACI","id":139,"type":"airline"}}

POST a new Airline resource

Learn how to run N1QL queries from the CLI using CBQ tool and verify the existing sample data:

bin > ./cbq -engine=http://192.168.99.100:8093
Couchbase query shell connected to http://192.168.99.100:8093/ . Type Ctrl-D to exit.
cbq> select * from `travel-sample` where name="Airlinair" limit 10;
{
    "requestID": "ce2de67b-2c05-47df-afbe-343cb7409d2b",
    "signature": {
        "*": "*"
    },
    "results": [
        {
            "travel-sample": {
                "callsign": "AIRLINAIR",
                "country": "France",
                "iata": "A5",
                "icao": "RLA",
                "id": 1203,
                "name": "Airlinair",
                "type": "airline"
            }
        }
    ],
    "status": "success",
    "metrics": {
        "elapsedTime": "3.418285894s",
        "executionTime": "3.418232688s",
        "resultCount": 1,
        "resultSize": 294
    }
}

bin > ./cbq -engine=http://192.168.99.100:8093

Couchbase query shell connected to http://192.168.99.100:8093/ . Type Ctrl-D to exit.

cbq> select * from `travel-sample` where name="Airlinair" limit 10;

{

"requestID": "ce2de67b-2c05-47df-afbe-343cb7409d2b",

"signature": {

"*": "*"

"results": [

{

"travel-sample": {

"callsign": "AIRLINAIR",

"country": "France",

"iata": "A5",

"icao": "RLA",

"id": 1203,

"name": "Airlinair",

"type": "airline"

}

"status": "success",

"metrics": {

"elapsedTime": "3.418285894s",

"executionTime": "3.418232688s",

"resultCount": 1,

"resultSize": 294

}

This query retrieve documents where airline’s name is Airlinair. The count is shown in metrics.resultCount.

Create a new document using POST.

Request

~ > curl -v -H "Content-Type: application/json" -X POST -d '{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","type":"airline"}' http://localhost:8080/couchbase-javaee/resources/airline
* Hostname was NOT found in DNS cache
*   Trying ::1...
* connect to ::1 port 8080 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /couchbase-javaee/resources/airline HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:8080
> Accept: */*
> Content-Type: application/json
> Content-Length: 104
>

~ > curl -v -H "Content-Type: application/json" -X POST -d '{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","type":"airline"}' http://localhost:8080/couchbase-javaee/resources/airline

* Hostname was NOT found in DNS cache

* Trying ::1...

* connect to ::1 port 8080 failed: Connection refused

* Trying 127.0.0.1...

* Connected to localhost (127.0.0.1) port 8080 (#0)

> POST /couchbase-javaee/resources/airline HTTP/1.1

> User-Agent: curl/7.37.1

> Host: localhost:8080

> Accept: */*

> Content-Type: application/json

> Content-Length: 104

Response

* upload completely sent off: 104 out of 104 bytes
< HTTP/1.1 200 OK
< Connection: keep-alive
< X-Powered-By: Undertow/1
* Server WildFly/9 is not blacklisted
< Server: WildFly/9
< Content-Type: application/octet-stream
< Content-Length: 117
< Date: Wed, 18 Nov 2015 21:42:51 GMT
< 
* Connection #0 to host localhost left intact
{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","id":"19810","type":"airline"}

* upload completely sent off: 104 out of 104 bytes

< HTTP/1.1 200 OK

< Connection: keep-alive

< X-Powered-By: Undertow/1

* Server WildFly/9 is not blacklisted

< Server: WildFly/9

< Content-Type: application/octet-stream

< Content-Length: 117

< Date: Wed, 18 Nov 2015 21:42:51 GMT

* Connection #0 to host localhost left intact

{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlinair","icao":"RLA","id":"19810","type":"airline"}

Query again using CBQ and now the results are shown as:

cbq> select * from `travel-sample` where name="Airlinair" limit 10;
{
    "requestID": "5e79031a-f7ee-4cc9-8c87-4e3b7484f09f",
    "signature": {
        "*": "*"
    },
    "results": [
        {
            "travel-sample": {
                "callsign": "AIRLINAIR",
                "country": "France",
                "iata": "A5",
                "icao": "RLA",
                "id": 1203,
                "name": "Airlinair",
                "type": "airline"
            }
        },
        {
            "travel-sample": {
                "callsign": "AIRLINAIR",
                "country": "France",
                "iata": "A5",
                "icao": "RLA",
                "id": "19810",
                "name": "Airlinair",
                "type": "airline"
            }
        }
    ],
    "status": "success",
    "metrics": {
        "elapsedTime": "3.342391947s",
        "executionTime": "3.342343455s",
        "resultCount": 2,
        "resultSize": 591
    }
}

cbq> select * from `travel-sample` where name="Airlinair" limit 10;

{

"requestID": "5e79031a-f7ee-4cc9-8c87-4e3b7484f09f",

"signature": {

"*": "*"

"results": [

{

"travel-sample": {

"callsign": "AIRLINAIR",

"country": "France",

"iata": "A5",

"icao": "RLA",

"id": 1203,

"name": "Airlinair",

"type": "airline"

}

{

"travel-sample": {

"callsign": "AIRLINAIR",

"country": "France",

"iata": "A5",

"icao": "RLA",

"id": "19810",

"name": "Airlinair",

"type": "airline"

}

"status": "success",

"metrics": {

"elapsedTime": "3.342391947s",

"executionTime": "3.342343455s",

"resultCount": 2,

"resultSize": 591

}

Note that two JSON documents are returned instead of one as before the POST command was issued.

PUT an existing Airline resource

Update an existing resource using HTTP POST.

The data model for travel-sample bucket requires to include “id” attribute in the payload and in the URI as well.

Request

~ > curl -v -H "Content-Type: application/json" -X PUT -d '{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlin Air","icao":"RLA","type":"airline","id": "19810"}' http://localhost:8080/couchbase-javaee/resources/airline/19810
* Hostname was NOT found in DNS cache
*   Trying ::1...
* connect to ::1 port 8080 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> PUT /couchbase-javaee/resources/airline/19810 HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:8080
> Accept: */*
> Content-Type: application/json
> Content-Length: 118
> 
* upload completely sent off: 118 out of 118 bytes

~ > curl -v -H "Content-Type: application/json" -X PUT -d '{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlin Air","icao":"RLA","type":"airline","id": "19810"}' http://localhost:8080/couchbase-javaee/resources/airline/19810

* Hostname was NOT found in DNS cache

* Trying ::1...

* connect to ::1 port 8080 failed: Connection refused

* Trying 127.0.0.1...

* Connected to localhost (127.0.0.1) port 8080 (#0)

> PUT /couchbase-javaee/resources/airline/19810 HTTP/1.1

> User-Agent: curl/7.37.1

> Host: localhost:8080

> Accept: */*

> Content-Type: application/json

> Content-Length: 118

* upload completely sent off: 118 out of 118 bytes

Name of the airline is updated from “Airlinair” to “Airlin Air”, all other attributes stay the same.

Response

< HTTP/1.1 200 OK
< Connection: keep-alive
< X-Powered-By: Undertow/1
* Server WildFly/9 is not blacklisted
< Server: WildFly/9
< Content-Type: application/octet-stream
< Content-Length: 117
< Date: Wed, 18 Nov 2015 21:46:16 GMT
< 
* Connection #0 to host localhost left intact
{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlin Air","icao":"RLA","id":"19810","type":"airline"}

< HTTP/1.1 200 OK

< Connection: keep-alive

< X-Powered-By: Undertow/1

* Server WildFly/9 is not blacklisted

< Server: WildFly/9

< Content-Type: application/octet-stream

< Content-Length: 117

< Date: Wed, 18 Nov 2015 21:46:16 GMT

* Connection #0 to host localhost left intact

{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlin Air","icao":"RLA","id":"19810","type":"airline"}

The updated record is shown in the response.

Querying for Airlinair gives:

cbq> select * from `travel-sample` where name="Airlinair" limit 10;
{
    "requestID": "a8d72427-9f4b-49ab-a77a-17cd99cdce5f",
    "signature": {
        "*": "*"
    },
    "results": [
        {
            "travel-sample": {
                "callsign": "AIRLINAIR",
                "country": "France",
                "iata": "A5",
                "icao": "RLA",
                "id": 1203,
                "name": "Airlinair",
                "type": "airline"
            }
        }
    ],
    "status": "success",
    "metrics": {
        "elapsedTime": "3.372603693s",
        "executionTime": "3.37256091s",
        "resultCount": 1,
        "resultSize": 294
    }
}

cbq> select * from `travel-sample` where name="Airlinair" limit 10;

{

"requestID": "a8d72427-9f4b-49ab-a77a-17cd99cdce5f",

"signature": {

"*": "*"

"results": [

{

"travel-sample": {

"callsign": "AIRLINAIR",

"country": "France",

"iata": "A5",

"icao": "RLA",

"id": 1203,

"name": "Airlinair",

"type": "airline"

}

"status": "success",

"metrics": {

"elapsedTime": "3.372603693s",

"executionTime": "3.37256091s",

"resultCount": 1,

"resultSize": 294

}

So the previously added record is now updated and thus does not appear in query results. Querying for Airlin Airgives:

cbq> select * from `travel-sample` where name="Airlin Air" limit 10;
{
    "requestID": "a3797a73-d879-4ca1-be90-e07179aae118",
    "signature": {
        "*": "*"
    },
    "results": [
        {
            "travel-sample": {
                "callsign": "AIRLINAIR",
                "country": "France",
                "iata": "A5",
                "icao": "RLA",
                "id": "19810",
                "name": "Airlin Air",
                "type": "airline"
            }
        }
    ],
    "status": "success",
    "metrics": {
        "elapsedTime": "3.393649025s",
        "executionTime": "3.393530368s",
        "resultCount": 1,
        "resultSize": 298
    }
}

cbq> select * from `travel-sample` where name="Airlin Air" limit 10;

{

"requestID": "a3797a73-d879-4ca1-be90-e07179aae118",

"signature": {

"*": "*"

"results": [

{

"travel-sample": {

"callsign": "AIRLINAIR",

"country": "France",

"iata": "A5",

"icao": "RLA",

"id": "19810",

"name": "Airlin Air",

"type": "airline"

}

"status": "success",

"metrics": {

"elapsedTime": "3.393649025s",

"executionTime": "3.393530368s",

"resultCount": 1,

"resultSize": 298

}

This shows the newly updated document.

DELETE an existing Airline resource

Query for a unique id:

cbq> select * from `travel-sample` where id="19810" limit 10;
{
    "requestID": "47a315cd-afe4-45a8-8814-5ab3034e0d0f",
    "signature": {
        "*": "*"
    },
    "results": [
        {
            "travel-sample": {
                "callsign": "AIRLINAIR",
                "country": "France",
                "iata": "A5",
                "icao": "RLA",
                "id": "19810",
                "name": "Airlin Air",
                "type": "airline"
            }
        }
    ],
    "status": "success",
    "metrics": {
        "elapsedTime": "3.006863656s",
        "executionTime": "3.006821997s",
        "resultCount": 1,
        "resultSize": 298
    }
}

cbq> select * from `travel-sample` where id="19810" limit 10;

{

"requestID": "47a315cd-afe4-45a8-8814-5ab3034e0d0f",

"signature": {

"*": "*"

"results": [

{

"travel-sample": {

"callsign": "AIRLINAIR",

"country": "France",

"iata": "A5",

"icao": "RLA",

"id": "19810",

"name": "Airlin Air",

"type": "airline"

}

"status": "success",

"metrics": {

"elapsedTime": "3.006863656s",

"executionTime": "3.006821997s",

"resultCount": 1,

"resultSize": 298

}

Notice that one document is returned.

Lets delete this document.

Request

~ > curl -v -X DELETE http://localhost:8080/couchbase-javaee/resources/airline/19810
* Hostname was NOT found in DNS cache
*   Trying ::1...
* connect to ::1 port 8080 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> DELETE /couchbase-javaee/resources/airline/19810 HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:8080
> Accept: */*
>

~ > curl -v -X DELETE http://localhost:8080/couchbase-javaee/resources/airline/19810

* Hostname was NOT found in DNS cache

* Trying ::1...

* connect to ::1 port 8080 failed: Connection refused

* Trying 127.0.0.1...

* Connected to localhost (127.0.0.1) port 8080 (#0)

> DELETE /couchbase-javaee/resources/airline/19810 HTTP/1.1

> User-Agent: curl/7.37.1

> Host: localhost:8080

> Accept: */*

Response

< HTTP/1.1 200 OK
< Connection: keep-alive
< X-Powered-By: Undertow/1
* Server WildFly/9 is not blacklisted
< Server: WildFly/9
< Content-Type: application/octet-stream
< Content-Length: 136
< Date: Wed, 18 Nov 2015 21:52:47 GMT
< 
* Connection #0 to host localhost left intact
{"travel-sample":{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlin Air","icao":"RLA","id":"19810","type":"airline"}}

< HTTP/1.1 200 OK

< Connection: keep-alive

< X-Powered-By: Undertow/1

* Server WildFly/9 is not blacklisted

< Server: WildFly/9

< Content-Type: application/octet-stream

< Content-Length: 136

< Date: Wed, 18 Nov 2015 21:52:47 GMT

* Connection #0 to host localhost left intact

{"travel-sample":{"country":"France","iata":"A5","callsign":"AIRLINAIR","name":"Airlin Air","icao":"RLA","id":"19810","type":"airline"}}

The deleted document is shown in the response.

Query again for the deleted id:

cbq> select * from `travel-sample` where id="19810" limit 10;
{
    "requestID": "972b0bbd-ba25-4f6c-a30e-ed188bf43588",
    "signature": {
        "*": "*"
    },
    "results": [
    ],
    "status": "success",
    "metrics": {
        "elapsedTime": "3.261481199s",
        "executionTime": "3.261431917s",
        "resultCount": 0,
        "resultSize": 0
    }
}

cbq> select * from `travel-sample` where id="19810" limit 10;

{

"requestID": "972b0bbd-ba25-4f6c-a30e-ed188bf43588",

"signature": {

"*": "*"

"results": [

"status": "success",

"metrics": {

"elapsedTime": "3.261481199s",

"executionTime": "3.261431917s",

"resultCount": 0,

"resultSize": 0

}

And no results are returned!

As mentioned earlier, the complete code base is at github.com/arun-gupta/couchbase-javaee.

Enjoy!

Couchbase at JavaOne 2015

October 16, 2015conferences, couchbase, javacouchbase, javaonearungupta

JavaOne 2015 is just a couple of weeks away. Why you should attend?

450+ sessions over a wide variety of topics around Java
5 days of geekgasm with toys, technology, and discussions
Best of the best developers gather here
Special 20th edition of the event is going to be cherished for ever
Venue is the most lovely city in the world – San Francisco!

Couchbase, one of the leading open-source, NoSQL, document datastore company is going to be there with a few talks. Here is where you’ll find us:

Saturday, Oct 24

JavaOne4Kids Day – Arun Gupta (@arungupta)
5pm – 8pm: Chinascaria (invite only)
7pm – 11:30pm: NetBeans Party (invite only)

Sunday, Oct 25

8am – 9:30am: Java Champions/JUG Leaders Brunch – Arun Gupta (@arungupta)
6:30pm – 7:30pm: WildFly, Hadoop, JavaFX and HTML5 in the Enterprise (UGF10306) – Arun Gupta (@arungupta)
8pm – 10pm: NetBeans, GlassFish, and Payara Party (Thirsty Bear)

Monday, Oct 26

8:30am – 10:30am: Docker and Kubernetes Recipes for Java Developers (TUT1708) – Arun Gupta (@arungupta)
12:30pm – 1:30pm: Refactor your Java EE Applications with Microservices and Containers (CON1700) – Arun Gupta (@arungupta)

Tuesday, Oct 27

12:30pm – 1:30pm: Build Scalable And Secure Mobiles with Java That Work Offline (CON11281) – Wayne Carter (@waynecarter), Ali LeClerc (@ali_leclerc)
5:15pm – 7:15pm: JavaOne Ignite – Arun Gupta (@arungupta)

Wednesday, Oct 28

4:30pm – 5:30pm: SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications (CON11282) – Keshav Murthy (@rkeshavmurthy), Gerald Sangudi (@sangudi)

Of course, we’ll also be running the “hallway track” and so feel free to meet us there.

8 Things About Couchbase is a good place to start learning about Couchbase!

If you’ve any doubts about San Francisco being the most beautiful city, go visit Golden Gate, Fisherman’s Wharf, Crooked Street, Alcatraz Island, Cable cars, Chinatown, MOMA, and much more

Kubernetes Application – Package Multiple Resources Together

August 18, 2015containers, javaee, techtipcontainers, docker, javaee, kubernetes, mysql, wildflyarungupta

Deploying an application in Kubernetes require to create multiple resources such as Pods, Services, Replication Controllers, and others. Typically each resource is define in a configuration file and created using kubectl script. But if multiple resources need to be created then you need to invoke kubectl multiple times. So if you need to create the following resources:

MySQL Pod
MySQL Service
WildFly Replication Controller

Then the commands would look like:

kubectl.sh create -f ~/workspaces/kubernetes-java-sample/app-mysql-pod.yaml
kubectl.sh create -f ~/workspaces/kubernetes-java-sample/app-mysql-service.yaml
kubectl.sh create -f ~/workspaces/kubernetes-java-sample/app-wildfly-rc.yaml

kubectl.sh create -f ~/workspaces/kubernetes-java-sample/app-mysql-pod.yaml

kubectl.sh create -f ~/workspaces/kubernetes-java-sample/app-mysql-service.yaml

kubectl.sh create -f ~/workspaces/kubernetes-java-sample/app-wildfly-rc.yaml

Or for convenience, wrap these invocations in a shell script. But that is not very intuitive! There is a better, and more natural and intuitive way.

Kubernetes allow multiple resources to be specified in a single configuration file. This allows to create a “Kubernetes Application” that can consists of multiple resources easily.

Previous section showed how to deploy the Java EE application using multiple configuration files. This application can be delpoyed using a single configuration file as well.

An application, as discussed above, consisting of MySQL Pod, MySQL Service, and WildFly Replication Controller can be created using the following configuration file:

apiVersion: v1
kind: Pod
metadata:
  name: mysql-pod
  labels:
    name: mysql-pod
    context: docker-k8s-lab
spec:
  containers:
    -
      name: mysql
      image: mysql:latest
      env:
        -
          name: "MYSQL_USER"
          value: "mysql"
        -
          name: "MYSQL_PASSWORD"
          value: "mysql"
        -
          name: "MYSQL_DATABASE"
          value: "sample"
        -
          name: "MYSQL_ROOT_PASSWORD"
          value: "supersecret"
      ports:
        -
          containerPort: 3306
----
apiVersion: v1
kind: Service
metadata:
  name: mysql-service
  labels:
    name: mysql-pod
    context: docker-k8s-lab
spec:
  ports:
    # the port that this service should serve on
    - port: 3306
  # label keys and values that must match in order to receive traffic for this service
  selector:
    name: mysql-pod
    context: docker-k8s-lab
----
apiVersion: v1
kind: ReplicationController
metadata:
  name: wildfly-rc
  labels:
    name: wildfly
    context: docker-k8s-lab
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: wildfly
    spec:
      containers:
      - name: wildfly-rc-pod
        image: arungupta/wildfly-mysql-javaee7:k8s
        ports:
        - containerPort: 8080

apiVersion: v1

kind: Pod

metadata:

labels:

context: docker-k8s-lab

spec:

containers:

image: mysql:latest

env:

value: "mysql"

value: "mysql"

value: "sample"

value: "supersecret"

ports:

containerPort: 3306

----

apiVersion: v1

kind: Service

metadata:

labels:

context: docker-k8s-lab

spec:

ports:

# the port that this service should serve on

- port: 3306

# label keys and values that must match in order to receive traffic for this service

selector:

context: docker-k8s-lab

----

apiVersion: v1

kind: ReplicationController

metadata:

labels:

context: docker-k8s-lab

spec:

replicas: 1

template:

metadata:

labels:

spec:

containers:

- name: wildfly-rc-pod

image: arungupta/wildfly-mysql-javaee7:k8s

ports:

- containerPort: 8080

Notice that each section, one each for MySQL Pod, MySQL Service, and WildFly Replication Controller, is separated by ----.

Such an application can be created as:

kubectl.sh create -f ~/workspaces/kubernetes-java-sample/app.yaml
pods/mysql-pod
services/mysql-service
replicationcontrollers/wildfly-rc

kubectl.sh create -f ~/workspaces/kubernetes-java-sample/app.yaml

pods/mysql-pod

services/mysql-service

replicationcontrollers/wildfly-rc

Complete details about how to setup Kubernetes and run this application are available at github.com/arun-gupta/kubernetes-java-sample/#kubernetes-application.

More details about creating a Kubernetes application with multiple resources can be found in #12104.

You can learn about how to create Kubernetes resources for a Java application, or otherwise, at github.com/arun-gupta/kubernetes-java-sample/.