Monthly Archives: July 2015

Getting Started with ELK Stack on WildFly

July 30, 2015containers, javaee, microservices, techtip, wildflydocker, elasticsearch, elk, kibana, logstash, microservices, wildflyarungupta

Your typical business application would consist of a variety of servers such as WildFly, MySQL, Apache, ActiveMQ, and others. They each have a log format, with minimal to no consistency across them. The log statement typically consist of some sort of timestamp (could be widely varied) and some text information. Logs could be multi-line. If you are running a cluster of servers then these logs are decentralized, in different directories.

How do you aggregate these logs? Provide a consistent visualization over them? Make this data available to business users?

This blog will:

Introduce ELK stack
Explain how to start it
Start a WildFly instance to send log messages to the ELK stack (Logstash)
View the messages using ELK stack (Kibana)

What is ELK Stack?

ELK stack provides a powerful platform to index, search and analyze your data. It uses Logstash for log aggregation, Elasticsearch for searching, and Kibana for visualizing and analyzing data. In short, ELK stack:

Collect logs and events data (Logstash)
Make it searchable in fast and meaningful ways (Elasticsearch)
Use powerful analytics to summarize data across many dimensions (Kibana)

Logstash is a flexible, open source data collection, enrichment, and transportation pipeline.

Logstash

Elasticsearch is a distributed, open source search and analytics engine, designed for horizontal scalability, reliability, and easy management.

Elasticsearch

Kibana is an open source data visualization platform that allows you to interact with your data through stunning, powerful graphics.

Kibana

How does ELK Stack work?

Logstash can collect logs from a variety of sources (using input plugins), process the data into a common format using filters, and stream data to a variety of sources (using output plugins). Multiple filters can be chained to parse the data into a common format. Together, they build a Logstash Processing Pipeline.

Inputs and outputs support codecs that enable you to encode or decode the data as it enters or exits the pipeline without having to use a separate filter.

Logstash can then store the data in Elasticsearch and Kibana provides a visualization of that data. Here is a sample pipeline that can collect logs from different servers and run it through the ELK stack.

Start ELK Stack

You can download individual components of ELK stack and start that way. There is plenty of advise on how to configure these components. But I like to start with a KISS, and Docker makes it easy to KISS!

All the source code on this blog is at github.com/arun-gupta/elk.

Clone the repo:

git clone https://github.com/arun-gupta/elk.git
cd elk

git clone https://github.com/arun-gupta/elk.git

cd elk

Run the ELK stack:

docker-compose up -d
Creating elk_elasticsearch_1...
Creating elk_logstash_1...
Creating elk_kibana_1...

docker-compose up -d

Creating elk_elasticsearch_1...

Creating elk_logstash_1...

Creating elk_kibana_1...

This will use the pre-built Elasticsearch, Logstack, and Kibana images. It is built upon the work done in github.com/nathanleclaire/elk.

docker ps will show the output as:

elk> docker ps
CONTAINER ID        IMAGE                           COMMAND                CREATED              STATUS              PORTS                                            NAMES
cdc61acc1623        kibana:4.1.1                    "/docker-entrypoint.   59 seconds ago       Up 58 seconds       0.0.0.0:80->5601/tcp                             elk_kibana_1          
e184f2efcf95        arungupta/logstash:1.5.3        "/opt/logstash/bin/l   About a minute ago   Up 58 seconds       0.0.0.0:5000->5000/tcp, 0.0.0.0:5000->5000/udp   elk_logstash_1        
0622b55e8645        arungupta/elasticsearch:1.7.1   "/opt/elasticsearch/   About a minute ago   Up 59 seconds       0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   elk_elasticsearch_1

elk> docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

cdc61acc1623 kibana:4.1.1 "/docker-entrypoint. 59 seconds ago Up 58 seconds 0.0.0.0:80->5601/tcp elk_kibana_1

e184f2efcf95 arungupta/logstash:1.5.3 "/opt/logstash/bin/l About a minute ago Up 58 seconds 0.0.0.0:5000->5000/tcp, 0.0.0.0:5000->5000/udp elk_logstash_1

0622b55e8645 arungupta/elasticsearch:1.7.1 "/opt/elasticsearch/ About a minute ago Up 59 seconds 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp elk_elasticsearch_1

It shows all the containers running.

WildFly and ELK

James (@the_jamezp) blogged about Centralized Logging for WildFly with ELK Stack. The blog explains how to configure WildFly to send log messages to Logstash. It uses the highly modular nature of WildFly to install jboss-logmanager-ext library and install it as a module. The configured logmanager includes @timestamp field to the log messages sent to logstash. These log messages are then sent to Elasticsearch.

Instead of following the steps, lets Docker KISS and use a pre-configured image to get you started.

Start the image as:

docker run -p 8080:8080 -it -e HOST=<DOCKER_HOST_IP> arungupta/wildfly-logstash

docker run -p 8080:8080 -it -e HOST=<DOCKER_HOST_IP> arungupta/wildfly-logstash

Make sure to substitute <DOCKER_HOST_IP> with the IP address of the host where your Docker host is running. This can be easily found using docker-machine ip <MACHINE_NAME>.

View Logs using ELK Stack

Kibana runs on an embedded nginx and is configured to run on port 80 in docker-compose.yml. Lets view the logs using that.

Access http://<DOCKER_HOST_IP> in your machine and it should show the default page as:The @timestamp field was created by logmanager configured in WildFly.
Click on Create to create an index pattern and select Discover tab to view the logs as:

Try connecting other sources and enjoy the power of distributed consolidated by ELK!

Some more references …

Logstash docs
Kibana docs
Elasticsearch The Definitive Guide

Distributed logging and visualization is a critical component in a microservices world where multiple services would come and go at a given time. A future blog will show how to use ELK stack with a microservices architecture based application.

Enjoy!

Kubernetes Design Patterns

July 21, 2015containers, microservices, redhatdocker, javaee, kubernetes, wildflyarungupta

14,000 commits and 400 contributors (including one tiny commit from me!) is what build Kubernetes 1.0. It is now available!

Download here
API Docs
Kubectl command tool
Getting Started Guide
Kubernetes Introduction Slides

This blog discusses some of the Kubernetes design patterns. All source code for the design patterns discussed below are available at kubernetes-java-sample.

Key Concepts of Kubernetes

At a very high level, there are three key concepts:

Pods are the smallest deployable units that can be created, scheduled, and managed. Its a logical collection of containers that belong to an application.
Master is the central control point that provides a unified view of the cluster. There is a single master node that control multiple minions.
Node is a worker node that run tasks as delegated by the master. Minions can run one or more pods. It provides an application-specific “virtual host” in a containerized environment.

Some other concepts to be aware of:

Replication Controller is a resource at Master that ensures that requested number of pods are running on nodes at all times.
Service is an object on master that provides load balancing across a replicated group of pods.
Label is an arbitrary key/value pair in a distributed watchable storage that the Replication Controller uses for service discovery.

Start Kubernetes Cluster

Easiest way to start a Kubernetes cluster on a Mac OS is using Vagrant:

export KUBERNETES_PROVIDER=vagrant
curl -sS https://get.k8s.io | bash

export KUBERNETES_PROVIDER=vagrant

curl -sS https://get.k8s.io | bash

Alternatively, Kubernetes can be downloaded from github.com/GoogleCloudPlatform/kubernetes/releases/download/v1.0.0/kubernetes.tar.gz, and cluster can be started as:

cd kubernetes
export KUBERNETES_PROVIDER=vagrant
cluster/kube-up.sh

cd kubernetes

export KUBERNETES_PROVIDER=vagrant

cluster/kube-up.sh

Kubernetes Cluster Vagrant

A Pod with One Container

This section will explain how to start a Pod with one Container. WildFly base Docker image will be used as the Container.

Pod, Replication Controller, Service, etc are all resources in Kubernetes. They can be created using the kubectl by using a configuration file.

The configuration file in this case:

apiVersion: v1
kind: Pod
metadata:
  name: wildfly-pod
  labels: 
    name: wildfly
spec: 
  containers: 
    - image: jboss/wildfly
      name: wildfly-pod
      ports: 
        - containerPort: 8080

apiVersion: v1

kind: Pod

metadata:

labels:

spec:

containers:

- image: jboss/wildfly

ports:

- containerPort: 8080

Complete details on how to create a Pod are explained at github.com/arun-gupta/kubernetes-java-sample#a-pod-with-one-container

Java EE Application Deployed in a Pod with One Container

This section will show how to deploy a Java EE application in a Pod with one Container. WildFly, with an in-memory H2 database, will be used as the container.

Configuration file is:

apiVersion: v1
kind: ReplicationController
metadata:
  name: javaee7-hol
  labels:
    name: javaee7-hol
spec:
  replicas: 1
  selector:
    name: javaee7-hol
  template:
    metadata:
      labels:
        name: javaee7-hol
    spec:
      containers:
      - name: master
        image: arungupta/javaee7-hol
        ports:
        - containerPort: 8080
          hostPort: 8080

apiVersion: v1

kind: ReplicationController

metadata:

labels:

spec:

replicas: 1

selector:

template:

metadata:

labels:

spec:

containers:

- name: master

image: arungupta/javaee7-hol

ports:

- containerPort: 8080

hostPort: 8080

Complete details at github.com/arun-gupta/kubernetes-java-sample#java-ee-application-deployed-in-a-pod-with-one-container-wildfly–h2-in-memory-database.

A Replication Controller with Two Replicas of a Pod

This section will explain how to start a Replication Controller with two replicas of a Pod. Each Pod will have one WildFly container.

Configuration file is:

apiVersion: v1
kind: ReplicationController
metadata:
  name: wildfly-rc
  labels:
    name: wildfly
    context: docker-k8s-lab
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: wildfly
    spec:
      containers:
      - name: wildfly-rc-pod
        image: arungupta/wildfly-mysql-javaee7:k8s
        ports:
        - containerPort: 8080

apiVersion: v1

kind: ReplicationController

metadata:

labels:

context: docker-k8s-lab

spec:

replicas: 1

template:

metadata:

labels:

spec:

containers:

- name: wildfly-rc-pod

image: arungupta/wildfly-mysql-javaee7:k8s

ports:

- containerPort: 8080

Complete details at github.com/arun-gupta/kubernetes-java-sample#a-replication-controller-with-two-replicas-of-a-pod-wildfly

Rescheduling Pods

Replication Controller ensures that specified number of pod “replicas” are running at any one time. If there are too many, the replication controller kills some pods. If there are too few, it starts more.

Complete details at github.com/arun-gupta/kubernetes-java-sample#rescheduling-pods.

Scaling Pods

Replication Controller allows dynamic scaling up and down of Pods.

Complete details at github.com/arun-gupta/kubernetes-java-sample#scaling-pods.

Kubernetes Service

Pods are ephemeral. IP address assigned to a Pod cannot be relied upon. Kubernetes, Replication Controller in particular, create and destroy Pods dynamically. A consumer Pod cannot rely upon the IP address of a producer Pod.

Kubernetes Service is an abstraction which defines a set of logical Pods. The set of Pods targeted by a Service are determined by labels associated with the Pods.

This section will show how to run a WildFly and MySQL containers in separate Pods. WildFly Pod will talk to the MySQL Pod using a Service.

Complete details at github.com/arun-gupta/kubernetes-java-sample#kubernetes-service.

Here are couple of blogs that will help you get started:

The complete set of Kubernetes blog entries provide more details.

Enjoy!

Scaling Kubernetes Cluster

July 15, 2015containers, redhatclustering, kubernetes, wildflyarungupta

Automatic Restarting of Pods inside Replication Controller of Kubernetes Cluster shows how Kubernetes reschedule pods in the cluster if one or more of existing Pods disappear for some reason. This is a common usage pattern and one of the key features of Kubernetes.

Another common usage pattern of Replication Controller is scaling:

The replication controller makes it easy to scale the number of replicas up or down, either manually or by an auto-scaling control agent, by simply updating the replicas field.

Replication Controller#Scaling

This blog will show how a Kubernetes cluster can be easily scaled up and down.

All the code used in this blog is available at kubernetes-java-sample.

Start Replication Controller and Verify

Start a Replication Controller as:

./cluster/kubectl.sh create -f ~/workspaces/kubernetes-java-sample/wildfly-rc.yaml 
replicationcontrollers/wildfly-rc

./cluster/kubectl.sh create -f ~/workspaces/kubernetes-java-sample/wildfly-rc.yaml

replicationcontrollers/wildfly-rc

Get status of the Pods:

./cluster/kubectl.sh get -w po
NAME      READY     STATUS    RESTARTS   AGE
NAME               READY     STATUS    RESTARTS   AGE
wildfly-rc-bgtkg   0/1       Pending   0          1s
wildfly-rc-l8fqv   0/1       Pending   0         1s
wildfly-rc-bgtkg   0/1       Pending   0         1s
wildfly-rc-l8fqv   0/1       Pending   0         1s
wildfly-rc-bgtkg   0/1       Pending   0         1s
wildfly-rc-l8fqv   0/1       Pending   0         1s
wildfly-rc-bgtkg   0/1       Running   0         1m
wildfly-rc-l8fqv   0/1       Running   0         1m
wildfly-rc-bgtkg   1/1       Running   0         1m
wildfly-rc-l8fqv   1/1       Running   0         1m

./cluster/kubectl.sh get -w po

NAME READY STATUS RESTARTS AGE

wildfly-rc-bgtkg 0/1 Pending 0 1s

wildfly-rc-l8fqv 0/1 Pending 0 1s

wildfly-rc-bgtkg 0/1 Pending 0 1s

wildfly-rc-l8fqv 0/1 Pending 0 1s

wildfly-rc-bgtkg 0/1 Pending 0 1s

wildfly-rc-l8fqv 0/1 Pending 0 1s

wildfly-rc-bgtkg 0/1 Running 0 1m

wildfly-rc-l8fqv 0/1 Running 0 1m

wildfly-rc-bgtkg 1/1 Running 0 1m

wildfly-rc-l8fqv 1/1 Running 0 1m

Make sure to wait for the status to change to Running.

Note down name of the Pods as wildfly-rc-bgtkg” and wildfly-rc-bgtkg”.

Get status of the Replication Controller:

./cluster/kubectl.sh get rc
CONTROLLER   CONTAINER(S)     IMAGE(S)        SELECTOR       REPLICAS
wildfly-rc   wildfly-rc-pod   jboss/wildfly   name=wildfly   2

./cluster/kubectl.sh get rc

CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS

wildfly-rc wildfly-rc-pod jboss/wildfly name=wildfly 2

If multiple Replication Controllers are running then you can query for this specific one using the label:

./cluster/kubectl.sh get rc -l name=wildfly
CONTROLLER   CONTAINER(S)     IMAGE(S)        SELECTOR       REPLICAS
wildfly-rc   wildfly-rc-pod   jboss/wildfly   name=wildfly   2

./cluster/kubectl.sh get rc -l name=wildfly

CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS

wildfly-rc wildfly-rc-pod jboss/wildfly name=wildfly 2

Scaling Kubernetes Cluster Up

Replication Controller allows dynamic scaling up and down of Pods.

Scale up the number of Pods:

./cluster/kubectl.sh scale --replicas=3 rc wildfly-rc
scaled

./cluster/kubectl.sh scale --replicas=3 rc wildfly-rc

scaled

Status of the Pods can be seen in another shell:

./cluster/kubectl.sh get -w po
NAME               READY     STATUS    RESTARTS   AGE
wildfly-rc-k6pk2   1/1       Running   0          47s
wildfly-rc-wez29   1/1       Running   0          47s
NAME               READY     STATUS    RESTARTS   AGE
wildfly-rc-aqaqn   0/1       Pending   0          0s
wildfly-rc-aqaqn   0/1       Pending   0         0s
wildfly-rc-aqaqn   0/1       Pending   0         0s
wildfly-rc-aqaqn   0/1       Running   0         2s
wildfly-rc-aqaqn   1/1       Running   0         11s

./cluster/kubectl.sh get -w po

NAME READY STATUS RESTARTS AGE

wildfly-rc-k6pk2 1/1 Running 0 47s

wildfly-rc-wez29 1/1 Running 0 47s

NAME READY STATUS RESTARTS AGE

wildfly-rc-aqaqn 0/1 Pending 0 0s

wildfly-rc-aqaqn 0/1 Running 0 2s

wildfly-rc-aqaqn 1/1 Running 0 11s

Notice a new Pod with the name wildfly-rc-aqaqn is created.

Scale Kubernetes Cluster Down

Scale down the number of Pods:

./cluster/kubectl.sh scale --replicas=1 rc wildfly-rc
scaled

./cluster/kubectl.sh scale --replicas=1 rc wildfly-rc

scaled

Status of the Pods using -w is not correctly updated (#11338). But status of the Pods can be seen correctly as:

./cluster/kubectl.sh get po
NAME               READY     STATUS    RESTARTS   AGE
wildfly-rc-k6pk2   1/1       Running   0          9m

./cluster/kubectl.sh get po

NAME READY STATUS RESTARTS AGE

wildfly-rc-k6pk2 1/1 Running 0 9m

Notice only one Pod is now running.

Kubernetes dynamically scales the Pods up and down using the scale --replicas command.

All code used in this blog is available at kubernetes-java-sample.

Enjoy!

Automatic Restarting of Pods inside Replication Controller of Kubernetes Cluster

July 14, 2015containers, redhat, techtipclustering, docker, kubernetesarungupta

A key feature of Kubernetes is its ability to maintain the “desired state” using declared primitives. Replication Controllers is a key concept that helps achieve this state.

A replication controller ensures that a specified number of pod “replicas” are running at any one time. If there are too many, it will kill some. If there are too few, it will start more.

Replication Controller

Lets take a look on how to spin up a Replication Controller with two replicas of a Pod. Then we’ll kill one pod and see how Kubernetes will start another Pod automatically.

Start Kubernetes Cluster

Easiest way to start a Kubernetes cluster on a Mac OS is using Vagrant:

export KUBERNETES_PROVIDER=vagrant
curl -sS https://get.k8s.io | bash

export KUBERNETES_PROVIDER=vagrant

curl -sS https://get.k8s.io | bash

Alternatively, Kubernetes can be downloaded from github.com/GoogleCloudPlatform/kubernetes/releases/download/v1.0.0/kubernetes.tar.gz, and cluster can be started as:

cd kubernetes
export KUBERNETES_PROVIDER=vagrant
cluster/kube-up.sh

cd kubernetes

export KUBERNETES_PROVIDER=vagrant

cluster/kube-up.sh

Start and Verify Replication Controller and Pods

All configuration files required by Kubernetes to start Replication Controller are in kubernetes-java-sample project. Clone the workspace:

git clone https://github.com/arun-gupta/kubernetes-java-sample.git

git clone https://github.com/arun-gupta/kubernetes-java-sample.git

Start a Replication Controller that has two replicas of a pod, each with a WildFly container:

./cluster/kubectl.sh create -f ~/workspaces/kubernetes-java-sample/wildfly-rc.yaml
replicationcontrollers/wildfly-rc

./cluster/kubectl.sh create -f ~/workspaces/kubernetes-java-sample/wildfly-rc.yaml

replicationcontrollers/wildfly-rc

The configuration file used is shown:

apiVersion: v1
kind: ReplicationController
metadata:
  name: wildfly-rc
  labels:
    name: wildfly
spec:
  replicas: 2
  template:
    metadata:
      labels:
        name: wildfly
    spec:
      containers:
      - name: wildfly-rc-pod
        image: jboss/wildfly
        ports:
        - containerPort: 8080

apiVersion: v1

kind: ReplicationController

metadata:

labels:

spec:

replicas: 2

template:

metadata:

labels:

spec:

containers:

- name: wildfly-rc-pod

image: jboss/wildfly

ports:

- containerPort: 8080

Default WildFly Docker image is used here.

Get status of the Pods:

./cluster/kubectl.sh get -w po
NAME               READY     STATUS    RESTARTS   AGE
wildfly-rc-15xg5   0/1       Pending   0          2s
wildfly-rc-d5fbs   0/1       Pending   0          2s
NAME               READY     STATUS    RESTARTS   AGE
wildfly-rc-15xg5   0/1       Pending   0          5s
wildfly-rc-d5fbs   0/1       Pending   0         5s
wildfly-rc-d5fbs   0/1       Running   0         8s
wildfly-rc-15xg5   0/1       Running   0         8s
wildfly-rc-d5fbs   1/1       Running   0         15s
wildfly-rc-15xg5   1/1       Running   0         15s

./cluster/kubectl.sh get -w po

NAME READY STATUS RESTARTS AGE

wildfly-rc-15xg5 0/1 Pending 0 2s

wildfly-rc-d5fbs 0/1 Pending 0 2s

NAME READY STATUS RESTARTS AGE

wildfly-rc-15xg5 0/1 Pending 0 5s

wildfly-rc-d5fbs 0/1 Pending 0 5s

wildfly-rc-d5fbs 0/1 Running 0 8s

wildfly-rc-15xg5 0/1 Running 0 8s

wildfly-rc-d5fbs 1/1 Running 0 15s

wildfly-rc-15xg5 1/1 Running 0 15s

Notice -w refreshes the status whenever there is a change. The status changes from Pending to Running and then Ready to receive requests.

Get status of the Replication Controller:

./cluster/kubectl.sh get rc
CONTROLLER   CONTAINER(S)     IMAGE(S)        SELECTOR       REPLICAS
wildfly-rc   wildfly-rc-pod   jboss/wildfly   name=wildfly   2

./cluster/kubectl.sh get rc

CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS

wildfly-rc wildfly-rc-pod jboss/wildfly name=wildfly 2

If multiple Replication Controllers are running then you can query for this specific one using the label:

./cluster/kubectl.sh get rc -l name=wildfly
CONTROLLER   CONTAINER(S)     IMAGE(S)        SELECTOR       REPLICAS
wildfly-rc   wildfly-rc-pod   jboss/wildfly   name=wildfly   2

./cluster/kubectl.sh get rc -l name=wildfly

CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS

wildfly-rc wildfly-rc-pod jboss/wildfly name=wildfly 2

Get name of the running Pods:

./cluster/kubectl.sh get po
NAME               READY     STATUS    RESTARTS   AGE
wildfly-rc-15xg5   1/1       Running   0          36m
wildfly-rc-d5fbs   1/1       Running   0          36m

./cluster/kubectl.sh get po

NAME READY STATUS RESTARTS AGE

wildfly-rc-15xg5 1/1 Running 0 36m

wildfly-rc-d5fbs 1/1 Running 0 36m

Find IP address of each Pod (using the name):

./cluster/kubectl.sh get -o template po wildfly-rc-15xg5 --template={{.status.podIP}}
10.246.1.18

./cluster/kubectl.sh get -o template po wildfly-rc-15xg5 --template={{.status.podIP}}

10.246.1.18

And of the other Pod as well:

./cluster/kubectl.sh get -o template po wildfly-rc-d5fbs --template={{.status.podIP}}
10.246.1.19

./cluster/kubectl.sh get -o template po wildfly-rc-d5fbs --template={{.status.podIP}}

10.246.1.19

Pod’s IP address is accessible only inside the cluster. Login to the minion to access WildFly’s main page hosted by the containers:

kubernetes> vagrant ssh minion-1
Last login: Tue Jul 14 21:35:12 2015 from 10.0.2.2
[vagrant@kubernetes-minion-1 ~]$ curl http://10.246.1.18:8080
<!--
  ~ JBoss, Home of Professional Open Source.

. . .

</div>
</body>
</html>
[vagrant@kubernetes-minion-1 ~]$ curl http://10.246.1.19:8080
<!--
  ~ JBoss, Home of Professional Open Source.
  ~ Copyright (c) 2014, Red Hat, Inc., and individual contributors

. . .

</div>
</body>
</html>

kubernetes> vagrant ssh minion-1

Last login: Tue Jul 14 21:35:12 2015 from 10.0.2.2

[vagrant@kubernetes-minion-1 ~]$ curl http://10.246.1.18:8080

<!--

~ JBoss, Home of Professional Open Source.

. . .

</div>

</body>

</html>

[vagrant@kubernetes-minion-1 ~]$ curl http://10.246.1.19:8080

<!--

~ JBoss, Home of Professional Open Source.

. . .

</div>

</body>

</html>

Automatic Restart of Pods

Lets delete a Pod and see how a new Pod is automatically created.

kubernetes> ./cluster/kubectl.sh delete po wildfly-rc-15xg5
pods/wildfly-rc-15xg5
kubernetes> ./cluster/kubectl.sh get -w po
NAME               READY     STATUS    RESTARTS   AGE
wildfly-rc-0xoms   0/1       Pending   0          2s
wildfly-rc-d5fbs   1/1       Running   0          48m
NAME               READY     STATUS    RESTARTS   AGE
wildfly-rc-0xoms   0/1       Pending   0          11s
wildfly-rc-0xoms   0/1       Running   0         13s
wildfly-rc-0xoms   1/1       Running   0         21s

kubernetes> ./cluster/kubectl.sh delete po wildfly-rc-15xg5

pods/wildfly-rc-15xg5

kubernetes> ./cluster/kubectl.sh get -w po

NAME READY STATUS RESTARTS AGE

wildfly-rc-0xoms 0/1 Pending 0 2s

wildfly-rc-d5fbs 1/1 Running 0 48m

NAME READY STATUS RESTARTS AGE

wildfly-rc-0xoms 0/1 Pending 0 11s

wildfly-rc-0xoms 0/1 Running 0 13s

wildfly-rc-0xoms 1/1 Running 0 21s

Notice how the Pod with name wildfly-rc-15xg5 was deleted and a new Pod with the name wildfly-rc-0xoms was created.

Finally, delete the Replication Controller:

kubectl.sh create -f ~/workspaces/kubernetes-java-sample/wildfly-rc.yaml

kubectl.sh create -f ~/workspaces/kubernetes-java-sample/wildfly-rc.yaml

The latest configuration files and detailed instructions are at kubernetes-java-sample.

In real world, you’ll typically wrap this Replication Controller in a Service and front-end with a Load Balancer. But that’s a topic for another blog!

Enjoy!

Minecon 2015 Wrapup

July 12, 2015conferences, kidsminecon, minecraft, mojangarungupta

From a gathering of ~50 people in 2010, Minecon 2015 with 10,000 attendees created a new world record for the biggest convention for a single game.

Minecon 2015 Experience

Do you want to know what what it feels like to be at Minecon?

Minecraft Modding Workshop

Devoxx4Kids was fortunate to give Minecraft Modding workshops to ~200 kids at Minecon 2015. Feedback from all the parents and kids was quite outstanding. Glad we were able to ignite spark in some kids and get them excited in programming, and open source tools like Java, Eclipse, and Minecraft Forge.

Here are a couple of tweets:

My son’s first Java program is a Minecraft mod. Thanks @arungupta. #Minecon2015 pic.twitter.com/dKfeXDZcNv

— Alberto Escarlate (@cacheop) July 7, 2015

Thanks @arungupta for a great intro to modding! #MINECON2015 pic.twitter.com/CcNJSEQUbK

— Jim Sewell (@tinyit) July 4, 2015

All the instructions for minecraft modding are at minecraftmodding.org.

Many thanks to Mark Little and his son Adam, and my son for helping with a successful workshop. Its very important that kids feel comfortable to play with open source tools, and be willing to hack!

Using Mods for Teaching Panel

I also got the opportunity to lead a panel on Using Mods for Teaching with @DorineFlies, @YouthDigital, and @_moonlapse.

Here are some of the questions we addressed:

How are you involved with modding?
How many students/kids have you reached out so far?
What languages/platforms do you use?
Can modding be the right medium for first introduction to programming?
What is an appropriate age to start modding?
What can be done to fundamentally change STEM education in schools?
What would you like from Mojang to improve the modding experience?

The panel was recorded and should be made available at youtube.com/user/TeamMojang/videos. I’ll update this blog when the exact link is available.

Minecraft Youtubers

One of the big craze, and genuine one, in the Minecraft world is about youtubers who produce video of game plays and most of them have 1m+ subscribers. Several of them were attending Minecon and we were fortunate to meet a few of them:

As Lydia walked around the main hall, most of the kids were super excited to meet their favorite youtubers!

Minecon 2015 Cape

Every Minecon attendee also get a cape that their in-game character can wear it and show-off the fact you attended a big celebration! Theme for this year was Iron Golem and it looks like as shown:

Minecraft Characters with Snaak

We also met the team behind Snaak and played around with creating some of the Minecraft characters using it.

Minecraft and HoloLens

A re-run of HoloLens and Minecraft video was also shown during one of the keynotes, and a preview is available here:

Here is the complete opening ceremony animation:

Minecon 2015 Photo Album

Check out some pictures from our trip:

And the complete photo album:

To me the highlight of the conference was meeting @SeargeDP. If there is one name that is responsible for starting modding in Minecraft, that would be him! Many thanks to him for giving us a chance to deliver minecraft modding workshops at Minecon.

And then, of course, meeting @lexmanos who is the lead developer for Minecraft Forge. We’ve authored an O’Reilly book (targeted at 8+ years old kid) and video on this topic. Several Devoxx4Kids chapters around the world have delivered workshops using the instructions based on Minecraft Forge and explained at minecraftmodding.org.

Check out a nice credential about book from one of the parents we met:

And last, but not the least, many thanks to the Mojang team for keeping the release cadence and supporting different modding communities.

Minecraft is truly a revolutionary game and allows to introduce Java programming to kids at a very early age!

Hopefully we get invited to Minecon 2016 again

Multi-container Applications using Docker Compose and Swarm

July 10, 2015containers, javaee, techtip, wildflyclustering, docker, javaee7, mysql, swarm, wildflyarungupta

Docker Compose to Orchestrate Containers shows how to run two linked Docker containers using Docker Compose. Clustering Using Docker Swarm shows how to configure a Docker Swarm cluster.

This blog will show how to run a multi-container application created using Docker Compose in a Docker Swarm cluster.

Updated version of Docker Compose and Docker Swarm are released with Docker 1.7.0.

Docker 1.7.0 CLI

Get the latest Docker CLI:

curl https://get.docker.com/builds/Darwin/x86_64/docker-latest > /usr/local/bin/docker

curl https://get.docker.com/builds/Darwin/x86_64/docker-latest > /usr/local/bin/docker

and check the version as:

docker -v
Docker version 1.7.0, build 0baf609

docker -v

Docker version 1.7.0, build 0baf609

Docker Machine 0.3.0

Get the latest Docker Machine as:

curl -L https://github.com/docker/machine/releases/download/v0.3.0/docker-machine_darwin-amd64 > /usr/local/bin/docker-machine

curl -L https://github.com/docker/machine/releases/download/v0.3.0/docker-machine_darwin-amd64 > /usr/local/bin/docker-machine

and check the version as:

docker-machine -v
docker-machine version 0.3.0 (0a251fe)

docker-machine -v

docker-machine version 0.3.0 (0a251fe)

Docker Compose 1.3.0

Get the latest Docker Compose as:

curl -L https://github.com/docker/compose/releases/download/1.3.0/docker-compose-`uname -s`-`uname -m` &gt; /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

curl -L https://github.com/docker/compose/releases/download/1.3.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

chmod +x /usr/local/bin/docker-compose

and verify the version as:

docker-compose -v
docker-compose version: 1.3.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1j 15 Oct 2014

docker-compose -v

docker-compose version: 1.3.0

CPython version: 2.7.9

OpenSSL version: OpenSSL 1.0.1j 15 Oct 2014

Docker Swarm 0.3.0

Swarm is run as a Docker container and can be downloaded as:

docker pull swarm

docker pull swarm

You can learn about Docker Swarm at docs.docker.com/swarm or Clustering using Docker Swarm.

Create Docker Swarm Cluster

The key components of Docker Swarm are shown below:

and explained in Clustering Using Docker Swarm.

The easiest way of getting started with Swarm is by using the official Docker image:

docker run swarm create

docker run swarm create

This command returns a discovery token, referred as <TOKEN> in this document, and is the unique cluster id. It will be used when creating master and nodes later. This cluster id is returned by the hosted discovery service on Docker Hub.

It shows the output as:

docker run swarm create
Unable to find image 'swarm:latest' locally
latest: Pulling from swarm
55b38848634f: Pull complete
fd7bc7d11a30: Pull complete
db039e91413f: Pull complete
1e5a49ab6458: Pull complete
5d9ce3cdadc7: Pull complete
1f26e949f933: Pull complete
e08948058bed: Already exists
swarm:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.
Digest: sha256:0e417fe3f7f2c7683599b94852e4308d1f426c82917223fccf4c1c4a4eddb8ef
Status: Downloaded newer image for swarm:latest
1d528bf0568099a452fef5c029f39b85

docker run swarm create

Unable to find image 'swarm:latest' locally

latest: Pulling from swarm

55b38848634f: Pull complete

fd7bc7d11a30: Pull complete

db039e91413f: Pull complete

1e5a49ab6458: Pull complete

5d9ce3cdadc7: Pull complete

1f26e949f933: Pull complete

e08948058bed: Already exists

swarm:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.

Digest: sha256:0e417fe3f7f2c7683599b94852e4308d1f426c82917223fccf4c1c4a4eddb8ef

Status: Downloaded newer image for swarm:latest

1d528bf0568099a452fef5c029f39b85

The last line is the <TOKEN>.

Make sure to note this cluster id now as there is no means to list it later. This should be fixed with#661.

Swarm is fully integrated with Docker Machine, and so is the easiest way to get started. Let’s create a Swarm Master next:

docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery token://&lt;TOKEN&gt; swarm-master

docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery token://<TOKEN> swarm-master

Replace <TOKEN> with the cluster id obtained in the previous step.

--swarm configures the machine with Swarm, --swarm-master configures the created machine to be Swarm master. Swarm master creation talks to the hosted service on Docker Hub and informs that a master is created in the cluster.

Connect to this newly created master and find some more information about it:

eval "$(docker-machine env swarm-master)"
docker info

eval "$(docker-machine env swarm-master)"

docker info

This will show the output as:

&gt; docker info
Containers: 2
Images: 7
Storage Driver: aufs
 Root Dir: /mnt/sda1/var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 11
 Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.0.5-boot2docker
Operating System: Boot2Docker 1.7.0 (TCL 6.3); master : 7960f90 - Thu Jun 18 18:31:45 UTC 2015
CPUs: 1
Total Memory: 996.2 MiB
Name: swarm-master
ID: DLFR:OQ3E:B5P6:HFFD:VKLI:IOLU:URNG:HML5:UHJF:6JCL:ITFH:DS6J
Debug mode (server): true
File Descriptors: 22
Goroutines: 36
System Time: 2015-07-11T00:16:34.29965306Z
EventsListeners: 1
Init SHA1:
Init Path: /usr/local/bin/docker
Docker Root Dir: /mnt/sda1/var/lib/docker
Username: arungupta
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox

> docker info

Containers: 2

Images: 7

Storage Driver: aufs

Root Dir: /mnt/sda1/var/lib/docker/aufs

Backing Filesystem: extfs

Dirs: 11

Dirperm1 Supported: true

Execution Driver: native-0.2

Logging Driver: json-file

Kernel Version: 4.0.5-boot2docker

Operating System: Boot2Docker 1.7.0 (TCL 6.3); master : 7960f90 - Thu Jun 18 18:31:45 UTC 2015

CPUs: 1

Total Memory: 996.2 MiB

Name: swarm-master

ID: DLFR:OQ3E:B5P6:HFFD:VKLI:IOLU:URNG:HML5:UHJF:6JCL:ITFH:DS6J

Debug mode (server): true

File Descriptors: 22

Goroutines: 36

System Time: 2015-07-11T00:16:34.29965306Z

EventsListeners: 1

Init SHA1:

Init Path: /usr/local/bin/docker

Docker Root Dir: /mnt/sda1/var/lib/docker

Username: arungupta

Registry: https://index.docker.io/v1/

Labels:

provider=virtualbox

Create a Swarm node

docker-machine create -d virtualbox --swarm --swarm-discovery token://&lt;TOKEN&gt; swarm-node-01

docker-machine create -d virtualbox --swarm --swarm-discovery token://<TOKEN> swarm-node-01

Replace <TOKEN> with the cluster id obtained in an earlier step.

Node creation talks to the hosted service at Docker Hub and joins the previously created cluster. This is specified by --swarm-discovery token://... and specifying the cluster id obtained earlier.

To make it a real cluster, let’s create a second node:

docker-machine create -d virtualbox --swarm --swarm-discovery token://&lt;TOKEN&gt; swarm-node-02

docker-machine create -d virtualbox --swarm --swarm-discovery token://<TOKEN> swarm-node-02

Replace <TOKEN> with the cluster id obtained in the previous step.

List all the nodes created so far:

docker-machine ls

docker-machine ls

This shows the output similar to the one below:

docker-machine ls
NAME            ACTIVE   DRIVER       STATE     URL                         SWARM
lab                      virtualbox   Running   tcp://192.168.99.101:2376
summit2015               virtualbox   Running   tcp://192.168.99.100:2376
swarm-master    *        virtualbox   Running   tcp://192.168.99.102:2376   swarm-master (master)
swarm-node-01            virtualbox   Running   tcp://192.168.99.103:2376   swarm-master
swarm-node-02            virtualbox   Running   tcp://192.168.99.104:2376   swarm-master

docker-machine ls

NAME ACTIVE DRIVER STATE URL SWARM

lab virtualbox Running tcp://192.168.99.101:2376

summit2015 virtualbox Running tcp://192.168.99.100:2376

swarm-master * virtualbox Running tcp://192.168.99.102:2376 swarm-master (master)

swarm-node-01 virtualbox Running tcp://192.168.99.103:2376 swarm-master

swarm-node-02 virtualbox Running tcp://192.168.99.104:2376 swarm-master

The machines that are part of the cluster have the cluster’s name in the SWARM column, blank otherwise. For example, “lab” and “summit2015” are standalone machines where as all other machines are part of the “swarm-master” cluster. The Swarm master is also identified by (master) in the SWARM column.

Connect to the Swarm cluster and find some information about it:

eval "$(docker-machine env --swarm swarm-master)"
docker info

eval "$(docker-machine env --swarm swarm-master)"

docker info

This shows the output as:

&gt; docker info
Containers: 4
Images: 3
Role: primary
Strategy: spread
Filters: affinity, health, constraint, port, dependency
Nodes: 3
 swarm-master: 192.168.99.102:2376
  └ Containers: 2
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.022 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.0.5-boot2docker, operatingsystem=Boot2Docker 1.7.0 (TCL 6.3); master : 7960f90 - Thu Jun 18 18:31:45 UTC 2015, provider=virtualbox, storagedriver=aufs
 swarm-node-01: 192.168.99.103:2376
  └ Containers: 1
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.022 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.0.5-boot2docker, operatingsystem=Boot2Docker 1.7.0 (TCL 6.3); master : 7960f90 - Thu Jun 18 18:31:45 UTC 2015, provider=virtualbox, storagedriver=aufs
 swarm-node-02: 192.168.99.104:2376
  └ Containers: 1
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.022 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.0.5-boot2docker, operatingsystem=Boot2Docker 1.7.0 (TCL 6.3); master : 7960f90 - Thu Jun 18 18:31:45 UTC 2015, provider=virtualbox, storagedriver=aufs
CPUs: 3
Total Memory: 3.065 GiB

> docker info

Containers: 4

Images: 3

Role: primary

Strategy: spread

Filters: affinity, health, constraint, port, dependency

Nodes: 3

swarm-master: 192.168.99.102:2376

└ Containers: 2

└ Reserved CPUs: 0 / 1

└ Reserved Memory: 0 B / 1.022 GiB

└ Labels: executiondriver=native-0.2, kernelversion=4.0.5-boot2docker, operatingsystem=Boot2Docker 1.7.0 (TCL 6.3); master : 7960f90 - Thu Jun 18 18:31:45 UTC 2015, provider=virtualbox, storagedriver=aufs

swarm-node-01: 192.168.99.103:2376

└ Containers: 1

└ Reserved CPUs: 0 / 1

└ Reserved Memory: 0 B / 1.022 GiB

swarm-node-02: 192.168.99.104:2376

└ Containers: 1

└ Reserved CPUs: 0 / 1

└ Reserved Memory: 0 B / 1.022 GiB

CPUs: 3

Total Memory: 3.065 GiB

There are 3 nodes – one Swarm master and 2 Swarm nodes. There is a total of 4 containers running in this cluster – one Swarm agent on master and each node, and there is an additional swarm-agent-master running on the master.

List nodes in the cluster with the following command:

docker run swarm list token://&lt;TOKEN&gt;

docker run swarm list token://<TOKEN>

This shows the output as:

&gt; docker run swarm list token://1d528bf0568099a452fef5c029f39b85
192.168.99.103:2376
192.168.99.104:2376
192.168.99.102:2376

> docker run swarm list token://1d528bf0568099a452fef5c029f39b85

192.168.99.103:2376

192.168.99.104:2376

192.168.99.102:2376

Deploy Java EE Application to Docker Swarm Cluster using Docker Compose

Docker Compose to Orchestrate Containers explains how multi container applications can be easily started using Docker Compose.

Use the docker-compose.yml file explained in that blog to start the containers as:

docker-compose up -d
Creating wildflymysqljavaee7_mysqldb_1...
Creating wildflymysqljavaee7_mywildfly_1...

docker-compose up -d

Creating wildflymysqljavaee7_mysqldb_1...

Creating wildflymysqljavaee7_mywildfly_1...

The docker-compose.yml file looks like:

mysqldb:
  image: mysql:latest
  environment:
    MYSQL_DATABASE: sample
    MYSQL_USER: mysql
    MYSQL_PASSWORD: mysql
    MYSQL_ROOT_PASSWORD: supersecret
mywildfly:
  image: arungupta/wildfly-mysql-javaee7
  links:
    - mysqldb:db
  ports:
    - 8080:8080

mysqldb:

image: mysql:latest

environment:

MYSQL_DATABASE: sample

MYSQL_USER: mysql

MYSQL_PASSWORD: mysql

MYSQL_ROOT_PASSWORD: supersecret

mywildfly:

image: arungupta/wildfly-mysql-javaee7

links:

- mysqldb:db

ports:

- 8080:8080

Check the containers running in the cluster as:

eval "$(docker-machine env --swarm swarm-master)"
docker info

eval "$(docker-machine env --swarm swarm-master)"

docker info

to see the output as:

docker info
Containers: 7
Images: 5
Role: primary
Strategy: spread
Filters: affinity, health, constraint, port, dependency
Nodes: 3
 swarm-master: 192.168.99.102:2376
  └ Containers: 2
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.022 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.0.5-boot2docker, operatingsystem=Boot2Docker 1.7.0 (TCL 6.3); master : 7960f90 - Thu Jun 18 18:31:45 UTC 2015, provider=virtualbox, storagedriver=aufs
 swarm-node-01: 192.168.99.103:2376
  └ Containers: 2
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.022 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.0.5-boot2docker, operatingsystem=Boot2Docker 1.7.0 (TCL 6.3); master : 7960f90 - Thu Jun 18 18:31:45 UTC 2015, provider=virtualbox, storagedriver=aufs
 swarm-node-02: 192.168.99.104:2376
  └ Containers: 3
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.022 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.0.5-boot2docker, operatingsystem=Boot2Docker 1.7.0 (TCL 6.3); master : 7960f90 - Thu Jun 18 18:31:45 UTC 2015, provider=virtualbox, storagedriver=aufs
CPUs: 3
Total Memory: 3.065 GiB

docker info

Containers: 7

Images: 5

Role: primary

Strategy: spread

Filters: affinity, health, constraint, port, dependency

Nodes: 3

swarm-master: 192.168.99.102:2376

└ Containers: 2

└ Reserved CPUs: 0 / 1

└ Reserved Memory: 0 B / 1.022 GiB

swarm-node-01: 192.168.99.103:2376

└ Containers: 2

└ Reserved CPUs: 0 / 1

└ Reserved Memory: 0 B / 1.022 GiB

swarm-node-02: 192.168.99.104:2376

└ Containers: 3

└ Reserved CPUs: 0 / 1

└ Reserved Memory: 0 B / 1.022 GiB

CPUs: 3

Total Memory: 3.065 GiB

“swarm-node-02” is running three containers and so lets look at the list of containers running there:

eval "$(docker-machine env swarm-node-02)"

eval "$(docker-machine env swarm-node-02)"

and see the list of running containers as:

docker ps -a
CONTAINER ID        IMAGE                             COMMAND                CREATED             STATUS              PORTS                    NAMES
b1e7d9bd2c09        arungupta/wildfly-mysql-javaee7   "/opt/jboss/wildfly/   38 seconds ago      Up 37 seconds       0.0.0.0:8080-&gt;8080/tcp   wildflymysqljavaee7_mywildfly_1
ac9c967e4b1d        mysql:latest                      "/entrypoint.sh mysq   38 seconds ago      Up 38 seconds       3306/tcp                 wildflymysqljavaee7_mysqldb_1
45b015bc79f4        swarm:latest                      "/swarm join --addr    20 minutes ago      Up 20 minutes       2375/tcp                 swarm-agent

docker ps -a

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

b1e7d9bd2c09 arungupta/wildfly-mysql-javaee7 "/opt/jboss/wildfly/ 38 seconds ago Up 37 seconds 0.0.0.0:8080->8080/tcp wildflymysqljavaee7_mywildfly_1

ac9c967e4b1d mysql:latest "/entrypoint.sh mysq 38 seconds ago Up 38 seconds 3306/tcp wildflymysqljavaee7_mysqldb_1

45b015bc79f4 swarm:latest "/swarm join --addr 20 minutes ago Up 20 minutes 2375/tcp swarm-agent

Application can then be accessed again using:

curl http://$(docker-machine ip swarm-node-02):8080/employees/resources/employees

curl http://$(docker-machine ip swarm-node-02):8080/employees/resources/employees

and shows the output as:

&lt;?<span class="pl-ent">xml</span><span class="pl-e"> version</span>=<span class="pl-s"><span class="pl-pds">"</span>1.0<span class="pl-pds">"</span></span><span class="pl-e"> encoding</span>=<span class="pl-s"><span class="pl-pds">"</span>UTF-8<span class="pl-pds">"</span></span><span class="pl-e"> standalone</span>=<span class="pl-s"><span class="pl-pds">"</span>yes<span class="pl-pds">"</span></span>?&gt;&lt;<span class="pl-ent">collection</span>&gt;&lt;<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">id</span>&gt;1&lt;/<span class="pl-ent">id</span>&gt;&lt;<span class="pl-ent">name</span>&gt;Penny&lt;/<span class="pl-ent">name</span>&gt;&lt;/<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">id</span>&gt;2&lt;/<span class="pl-ent">id</span>&gt;&lt;<span class="pl-ent">name</span>&gt;Sheldon&lt;/<span class="pl-ent">name</span>&gt;&lt;/<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">id</span>&gt;3&lt;/<span class="pl-ent">id</span>&gt;&lt;<span class="pl-ent">name</span>&gt;Amy&lt;/<span class="pl-ent">name</span>&gt;&lt;/<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">id</span>&gt;4&lt;/<span class="pl-ent">id</span>&gt;&lt;<span class="pl-ent">name</span>&gt;Leonard&lt;/<span class="pl-ent">name</span>&gt;&lt;/<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">id</span>&gt;5&lt;/<span class="pl-ent">id</span>&gt;&lt;<span class="pl-ent">name</span>&gt;Bernadette&lt;/<span class="pl-ent">name</span>&gt;&lt;/<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">id</span>&gt;6&lt;/<span class="pl-ent">id</span>&gt;&lt;<span class="pl-ent">name</span>&gt;Raj&lt;/<span class="pl-ent">name</span>&gt;&lt;/<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">id</span>&gt;7&lt;/<span class="pl-ent">id</span>&gt;&lt;<span class="pl-ent">name</span>&gt;Howard&lt;/<span class="pl-ent">name</span>&gt;&lt;/<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">employee</span>&gt;&lt;<span class="pl-ent">id</span>&gt;8&lt;/<span class="pl-ent">id</span>&gt;&lt;<span class="pl-ent">name</span>&gt;Priya&lt;/<span class="pl-ent">name</span>&gt;&lt;/<span class="pl-ent">employee</span>&gt;&lt;/<span class="pl-ent">collection</span>&gt;

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><collection><employee><id>1</id><name>Penny</name></employee><employee><id>2</id><name>Sheldon</name></employee><employee><id>3</id><name>Amy</name></employee><employee><id>4</id><name>Leonard</name></employee><employee><id>5</id><name>Bernadette</name></employee><employee><id>6</id><name>Raj</name></employee><employee><id>7</id><name>Howard</name></employee><employee><id>8</id><name>Priya</name></employee></collection>

Latest instructions for this setup are always available at: github.com/javaee-samples/docker-java/blob/master/chapters/docker-swarm.adoc.

Enjoy!

Microservices and DevOps Journey at Wix

July 1, 2015microservicesmicroservices, wixarungupta

Wix.com started their journey on DevOps and Microservices about two years ago and recently switched from a monolithic application to a microservices-based application. Yes, it took them full two years to complete the transition from monolith to microservices!

I got connected with Aviran Mordo (@aviranm), head of backend engineering at Wix on twitter.

@arungupta @Valdarez we at @WixEng are doing microservices for over 4 years now with over 100 microservices in production

— Aviran Mordo (@aviranm) June 26, 2015

They migrated to microservices because the “system could not scale” and the requirements for functional components were varied. The journey took their WAR-based deployment on Tomcat to fat JAR with embedded Jetty. On a side note, take a look at WildFly Swarm if you are interested in a similar approach for Java EE applications.

Video Interview

I discussed some points with him about this journey and you can watch the same too.

In this discussion, you’ll learn:

Why Continuous Delivery and DevOps are important requirements for microservices?
How they migrated from a big monolith to smaller monoliths and then a full-blown microservices architecture
How database referential integrity constraints were moved from database to application?
“micro” in microservices refers to the area of responsibility, nothing to do with LOC
Neither REST nor messaging was used for communication between different services. Which protocol was used? JSON-RPC
How do services register and discover each other? Is that required during early phases?
Why YAGNI and KISS are important?
Chef for configuration management and how to make it accessible for massive deployments
TeamCity for CI
Is 100% automation a requirement? Can 100% automation be achieved? Learn about Petri, Wix’s open source framework for A/B testing
Relevance of hybrid cloud (Google, Amazon, Private data center) and redundancy
Hardest part of migrating from monolith to microservice
How much code was repurposed during refactoring?
Where was the most effort spent during the two years of migration?
Distributed transactions
What was the biggest challenge in DevOps journey? Look out for a nice story towards the end that could be motivating for your team as well 😉

Additional Material

Watch the slides from DevoxxUK:

Scaling wix with microservices architecture devoxx London 2015 from Aviran Mordo

You can also learn more about their architecture in Scaling Wix to 60m Users.

Enjoy!