Tag Archives: mysql

Deploying Java EE Application to Docker Swarm Cluster (Tech Tip #88)

containers, javaee, techtip, wildfly, , , , ,

What is Docker Swarm?

Docker Swarm provides native clustering to Docker. Clustering using Docker Swarm 0.2.0 provide a basic introduction to Docker Swarm, and how to create a simple three node cluster. As a refresher, the key components of Docker Swarm are shown below:

In short, Swarm Manager is a pre-defined Docker Host, and is a single point for all administration. Additional Docker hosts are identified as Nodes and communicate with the Manager using TCP. By default, Swarm uses hosted Discovery Service, based on Docker Hub, using tokens to discover nodes that are part of a cluster. Each node runs a Node Agent that registers the referenced Docker daemon, monitors it, and updates the Discovery Service with the node’s status. The containers run on a node.

That blog provide complete details, but a quick summary to create the cluster is shown below:

Listing the cluster shows:

It has one master and two nodes.

Deploy a Java EE application to Docker Swarm

All hosts in the cluster are accessible using a single, virtual host. Swarm serves the standard Docker API, so any tool that communicates with a single Docker host communicate can scale to multiple Docker hosts by communicating to this virtual host.

Docker Container Linking Across Multiple Hosts explains how to link containers across multiple Docker hosts. It deploys a Java EE 7 application to WildFly on one Docker host, and connects it with a MySQL container running on a different Docker host. We can deploy both of these containers using the virtual host, and they will then be deployed to the Docker Swarm cluster.

Lets get started!

MySQL on Docker Swarm

  1. Start the MySQL container
  2. Status of the container can be seen as:

    It shows the container is running on swarm-node-01.

    Make sure you are connected to the Docker Swarm cluster using eval $(docker-machine env --swarm swarm-master).

  3. Find IP address of the host where this container is started:

    Note IP address of the node where MySQL server is running. This will be used when starting WildFly application server later.

    ps: Filtering by name seem to not return accurate results (#10897).

WildFly on Docker Swarm

  1. Start WildFly application server by passing the IP address of the host and the port on which MySQL server is running:

  2. Status of the container can be seen as:

    It shows the container is running on swarm-node-02. IP address of the host is also shown in the PORTS column.

    As explained in Tech Tip #69, JDBC URL of the data source uses the specified IP address and port for connecting with the MySQL server. However passing IP address is very brittle as the MySQL server may restart on a different Docker host. This is filed as #773.

  3. Access the application at:

    This is using the IP address of the host where the container is started.

Enjoy!

 

Docker MySQL Persistence (Tech Tip #83)

containers, techtip, , ,

One of the recipes in 9 Docker recipes for Java developers  is using MySQL container with WildFly. Docker containers are ephemeral, and so any state stored in them is gone after they are terminated and removed. So even though MySQL container can be used as explained in the recipe, DDL/DML commands can be used to persist data, but that state is lost, or at least not accessible, after the container is terminated and removed.

This blog shows different approaches of Docker MySQL Persistence – across container restarts and accessible from multiple containers.

Default Data Location of MySQL Docker Container

Lets see the default location where MySQL Docker container stores the data.

Start a MySQL container as:

And inspect as:

Then it shows the anonymous volumes:

If you are using Boot2Docker, then /mnt/sda1 directory is used for storing images, containers, and data. This directory is from the Boot2Docker virtual machine filesystem. This is clarified in Docker docs as well and worth repeating here:

Note: If you are using Boot2Docker, your Docker daemon only has limited access to your OSX/Windows filesystem. Boot2Docker tries to auto-share your /Users (OSX) or C:\Users (Windows) directory – and so you can mount files or directories using docker run -v /Users/<path>:/<container path> ... (OSX) or docker run -v /c/Users/<path>:/<container path ... (Windows). All other paths come from the Boot2Docker virtual machine’s filesystem.

You can view this mounted directory on Boot2Docker by logging into the VM as:

And then view the directory listing as:

MySQL Data Across Container Restart – Anonymous Volumes

Anonymous volumes, i.e. volumes created by a container and which are not explicitly mounted, are container specific. They stay around unless explicitly deleted using docker remove -v command. This means a new anonymous volume is mounted for a new container even though the previous volume may not be deleted. The volume still lives on the Docker host even after the container is terminated and removed. Anonymous volume created by one MySQL container is not accessible to another MySQL container. This means data cannot be shared between different data containers.

Lets understand this using code.

Start a MySQL container as:

Login to the container:

Connect to the MySQL instance, and create a table, as:

Stop the container:

Restart the container:

Now when you connect to the MySQL container, the database table is shown correctly. This shows that anonymous volumes can persist state across container restarts.

Inspect the container:

And it correctly shows the same anonymous volume from /mnt/sda1 directory.

Now lets delete the container, and start a new MySQL container. First remove the container:

And start a new container using the same command as earlier:

Now when you try to see the list of tables, its shown as empty:

This is because anonymous volumes are visible across container restarts, but not visible to different containers. A new volume is mounted for a new run of the container. This is also verified by inspecting the container again:

A different directory is used to mount the anonymous volume.

So effectively, any data stored in the MySQL database by one container is not available to another MySQL container.

Docker Volume to Store MySQL Data

One option to share data between different MySQL containers is to mount directories on your Docker host as volume in the containers using -v switch when running the Docker image. If you are using Boot2Docker, then there are two options:

  • Mount a directory from the Boot2Docker VM filesystem. This directory, if does not exist already, would need to be created.
  • Mount a directory from your Mac host. For convenience, this need to exist in /Users/arungupta or whatever your corresponding directory is.

The first approach ties to the specific Boot2Docker VM image, and the second approach ties to a specific Mac host. We’ll look at how this can be fixed later.

We’ll discuss the first approach only here. Start the MySQL container as:

/var/lib/mysql is the default directory where MySQL container writes its files. This directory is not persisted after a Boot2Docker reboot. So the recommended option is to create a directory in /mnt/sda1 and map that instead. Make sure to create the directory /mnt/sda1/var/mysql_data, as is the case above.

Now inspecting the container as:

Now any additional runs of the container can mount the same volume and will have access to the data.

Remember, multiple MySQL containers cannot access this shared mount together and instead will give the error:

So you need to make sure to stop an existing MySQL container, start a new MySQL container using the same volume, and the data would still be accessible.

This might be configured using master/slave configuration, where the master and slave have access to same volume. It’ll be great if somebody who has tried that configuration can share that recipe.

But as mentioned before, this approach is host-centric. It restricts MySQL to a particular Boot2Docker VM image. That means, you once again loose the big benefit of portability as offered by Docker.

Meet Docker data-only containers!

Docker Data-only Containers

Docker follows Single Responsibility Principle (SRP) really well. Docker Data-only containers are NoOp containers that perform a command that is not really relevant, and instead mount volumes that are used for storing data. These containers don’t even need to start or run, and so the command really is irrelevant, just creating them is enough.

Create the container as:

If you plan to use a MySQL container later, its recommended to use the mysql image to save bandwidth and space from downloading another random image. You can adjust this command for whatever database container you are using.

If you intend to use MySQL, then this data-only container can be created as:

Dockerfile for this container is pretty simple and can be adopted for a database server of your choice.

Since this container is not running, it will not be visible with just docker ps. Instead you’ll need to use docker ps -a to view the container:

Docker allows to mount, or pull in, volumes from other containers using --volumes-from switch specified when running the container.

Lets start our MySQL container to use this data-only container as:

Boot2Docker VM has /var/lib/mysql directory now populated:

If you stop this container, and run another container then the data will be accessible there.

Docker Data Containers

In a simple scenario, application server, database, and data-only container can all live on the same host. Alternatively, application server can live on a separate host and database server and data-only container can stay on the same host.

Hopefully this would be more extensive when Docker volumes can work across multiple hosts.

It would be nice if all of this, i.e. creating the data-only container and starting the MySQL container that uses the volume from data-only container can be easily done using Docker Compose. #1284 should fix this.

Usual mysqldump and mysql commands can be used to backup and restore from the volume. This can be achieved by connecting to the MySQL using CLI as explained here.

You can also look at docker-volumes to manage volumes on your host.

You can also read more about volumes may evolve in future at #6496.

Enjoy!

Docker Compose to Orchestrate Containers – Tech Tip #77

containers, javaee, , , ,

Docker Orchestration using Fig showed how to defining and control a multi-container service using Fig. Since then, Fig has been renamed to Docker Compose, or Compose for short.

First release of Compose was announced recently

From github.com/docker/compose

Compose is a tool for defining and running complex applications with Docker. With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.

Docker Compose uses the same API used by other Docker commands and tools.

Docker Compose

This Tech Tip will rewrite Docker Orchestration using Fig blog to use Docker Compose. In other words, it will show how to run a Java EE 7 application that is deployed using MySQL and WildFly.

Lets get started!

Install Docker Compose

Install Compose as:

Docker Compose Configuration File

Entry point to Compose is docker-compose.yml. To begin with, docker-compose tool also recognizes fig.yml file name but shows the following message:

And if both fig.yml and docker-compose.yml are available in the directory then the following message is shown:

Use the same configuration file from the previous blog and rename to docker-compose.yml:

This YML-based configuration file has:

  1. Two containers defined by the name “mysqldb” and “mywildfly”
  2. Image names are defined using “image”
  3. Environment variables for the MySQL container are defined in “environment”
  4. MySQL container is linked with WildFly container using “links”
  5. Port forwarding is achieved using “ports”

Start, Verify, Stop Docker Containers

  1. All the containers can be started, in detached mode, by giving the command:

    And that shows the output as:
  2. Verify the containers as:
  3. Logs for the containers can be seen as:

    And shows the output as:
  4. Find the IP address of the host as:

    And access the application as:

    To see the output as:

    Or in the browser as:

    Docker Compose Output

  5. Stop the containers as:

    to see the output as:

Docker Compose Commands

Complete list of Docker Compose commands can be seen by typing docker-compose and shows the output as:

A subsequent blog will likely play with scale command.

Help for each command is shown by typing -h after the command name. For example, help for run command is shown as:

Enjoy!

MySQL as Kubernetes Service, Access from WildFly Pod (Tech Tip #72)

containers, javaee, techtip, , , ,

Java EE 7 and WildFly on Kubernetes using Vagrant (Tech Tip #71) explained how to run a trivial Java EE 7 application on WildFly hosted using Kubernetes and Docker. The Java EE 7 application was the hands-on lab that have been delivered around the world. It uses an in-memory database that is bundled with WildFly and allows to understand the key building blocks of Kubernetes. This is good to get you started with initial development efforts but quickly becomes a bottleneck as the database is lost when the application server goes down. This tech tip will show how to run another trivial Java EE 7 application and use MySQL as the database server. It will use Kubernetes Services to explain how MySQL and WildFly can be easily decoupled.

Lets get started!

Make sure to have a working Kubernetes setup as explained in Kubernetes using Vagrant.

The complete source code used in this blog is available at github.com/arun-gupta/kubernetes-java-sample.

Start MySQL Kubernetes pod

First step is to start the MySQL pod. This can be started by using the MySQL Kubernetes configuration file:

The configuration file used is at github.com/arun-gupta/kubernetes-java-sample/blob/master/mysql.json.

Check the status of MySQL pod:

Wait till the status changes to “Running”. It will look like:

It takes a few minutes for MySQL server to be in that state, so grab a coffee or a quick fast one miler!

Start MySQL Kubernetes service

Pods, and the IP addresses assigned to them, are ephemeral. If a pod dies then Kubernetes will recreate that pod because of its self-healing features, but it might recreate it on a different host. Even if it is on the same host, a different IP address could be assigned to it. And so any application cannot rely upon the IP address of the pod.

Kubernetes services is an abstraction which defines a logical set of pods. A service is typically back-ended by one or more physical pods (associated using labels), and it has a permanent IP address that can be used by other pods/applications. For example, WildFly pod can not directly connect to a MySQL pod but can connect to MySQL service. In essence, Kubernetes service offers clients an IP and port pair which, when accessed, redirects to the appropriate backends.

Kubernetes Services

Lets start MySQL service.

The configuration file used is at github.com/arun-gupta/kubernetes-java-sample/blob/master/mysql-service.json. In this case, only a single MySQL instance is started. But multiple MySQL instances can be easily started and WildFly Pod will continue to refer to all of them using MySQL Service.

Check the status/IP of the MySQL service:

Start WildFly Kubernetes Pod

WildFly Pod must be started after MySQL service has started. This is because the environment variables used for creating JDBC resource in WildFly are only available after the service is up and running. Specifically, the JDBC resource is created as:

$MYSQL_SERVICE_HOST and $MYSQL_SERVICE_PORT environment variables are populated by Kubernetes as explained here.

This is shown at github.com/arun-gupta/docker-images/blob/master/wildfly-mysql-javaee7/customization/execute.sh#L44.

Start WildFly pod:

The configuration file used is at github.com/arun-gupta/kubernetes-java-sample/blob/master/wildfly.json.

Check the status of pods:

Wait until WildFly pod’s status is changed to Running. This could be a few minutes, so may be time to grab another quick miler!

Once the container is up and running, you can check /opt/jboss/wildfly/standalone/configuration/standalone.xml in the WildFly container and verify that the connection URL indeed contains the correct IP address. Here is how it looks on my machine:

The updated status (after the container is running) would look like as shown:

Access the Java EE 7 Application

Note down the HOST IP address of the WildFly container and access the application as:

to see the output as:

Or viewed in the browser as:

Java EE 7 Application using WildFly, MySQL, and Kubernetes

Debugging Kubernetes and Docker

Login to the Minion-1 VM:

Log in as root:

Default root password for VM images created by Vagrant is “vagrant”.

List of Docker containers running on this VM can be seen as:

Last 10 lines of the WildFly log (after application has been accessed a few times) can be seen as:

Similarly, MySQL log is seen as:

Enjoy!

 

Docker container linking across multiple hosts (Tech Tip #69)

containers, techtip, wildfly, , ,

Docker container linking is important concept to understand since any application in production will typically run on a cluster of containers across multiple hosts. But simple container linking does not allow cross-host communication.

Whats the issue with Docker container linking?

Docker containers can communicate with each other be manually linking as shown in Tech Tip #66 or orchestrated using Fig as shown in Tech Tip #68. Both of these using container linking but that has an inherent disadvantage that it is restricted to a single host. Linking does not work if containers are running across multiple hosts.

What is the solution?

This Tech Tip will evolve the sample built in Tech Tip #66 and #68 and show the containers can be connected if they are running across multiple hosts.

Docker container linking across multiple hosts can be easily done by explicitly publishing the host/port and using it from a container on a different host.

Lets get started!

  1. Start MySQL container as:

    The MySQL container is explicitly forwarding the port 3306 to port 5506.
  2. Git repo has customization/execute.sh that creates the MySQL data source. The command looks like:

    This command creates the JDBC resource for WildFly using jboss-cli. It is using $DB_PORT_3306_TCP_ADDR and $DB_PORT_3306_TCP_PORT variables which are defined per Container Linking Environment Variables. The scheme by which the environment variables for containers are created is rather weird. It exposes the port number in the variable name itself. I hope this improves in subsequent releases.

    This command needs to be updated such that an explicit host/port can be used instead.

    So update the command to:

    The only change in the command is to use $MYSQL_HOST and $MYSQL_PORT variables. This command already exists in the file but is commented. So just comment the previous one and uncomment this one.

  3. Build the image and run it as:

    Make sure to substitute <IP_ADDRESS> with the IP address of your host. For convenience, I ran it on the same host. The IP address in this case can be easily obtained using boot2docker ip.

  4. A quick verification of the deployment can be done by accessing the REST endpoint:

With this, your WildFly and MySQL can run on two separate hosts, no special configuration required.

Enjoy!

Docker allows cross-host container linking using Ambassador Containers but that adds a redundant hop for the service to be accessed. A cleaner solution would to use Kubernetes or Swarm, more on that later.

Marek also blogged about a more elaborate solution in Connecting Docker Containers on Multiple Hosts.

 

Docker orchestration using Fig (Tech Tip #68)

containers, techtip, wildfly, , , ,

Tech Tip #66 showed how to run a Java EE 7 application using WildFly and MySQL in two separate containers. It required to explicitly start the two containers, and link them using --link. This defining and controlling a multi-container service is a common design pattern in order to get an application up and going.

Meet Fig – Docker Orchestration Tool.

Fig allows to:

  • Define multiple containers in a single configuration file
  • Create dependencies between two containers by creating links between them
  • Start containers in the right sequence

Let’s get started!

  1. Install Fig as:
  2. Entry point to Fig is a configuration file that defines the containers and their dependencies. The equivalent configuration file from Tech Tip #65 is:

    This YML-based configuration file has:

    1. Two containers defined by the name “mysqldb” and “mywildfly”
    2. Image names are defined using “image”
    3. Environment variables for the MySQL container are defined in “environment”
    4. MySQL container is linked with WildFly container using “links”
    5. Port forwarding is achieved using “ports”
  3. All the containers can be started, in detached mode, by giving the command:

    The output is shown as:

    Fig commands allow to monitor and update the status of the containers:

    1. Logs can be seen as:
    2. Container status can be seen by giving the command:

      to show the output as:
    3. Containers can be stopped as:
    4. Alternatively, containers can be started in foreground by giving the command:

      and the output is seen as:
  4. Find out the IP address using boot2docker ip and access the app as:

Complete list of Fig commands can be seen by typing fig:

Particularly interesting is scale command, and we’ll take a look at it in a subsequent blog.

File issues on github.

Enjoy!

WildFly/JavaEE7 and MySQL linked on two Docker containers (Tech Tip #66)

containers, techtip, wildfly, , ,

Tech Tip #61 showed how to run Java EE 7 hands-on lab on WildFly Docker container. A couple of assumptions were made in that case:

  • WildFly bundles H2 in-memory database. The Java EE 7 application uses the default database resource, which in case of WildFly, gets resolved to a JDBC connection to that in-memory database. This is a good way to start building your application but pretty soon you want to start using a real database, like MySQL.
  • Typically, Application Server and Database may not be residing on the same host. This reduces the risk by avoiding a single point of failure. And so WildFly and MySQL would be on to separate host.

There is plethora of material available to show how to configure WildFly and MySQL on separate hosts. What are the design patterns, and anti-patterns, if you were to do that using Docker?

Lets take a look!

In simplified steps:

  1. Run the MySQL container as:
  2. Run the WildFly container, with MySQL JDBC resource pre-configured, as:
  3. Find the IP address of the WildFly container:

    If you are on a Mac, then use boot2docker ip to find the IP address.
  4. Access the application as:

    to see the output as:

    The application is a trivial Java EE 7 application that publishes a REST endpoint. Access it as:

    to see:

If you are interested in nitty gritty, read further details.

Linking Containers

The first concept we need to understand is how Docker allows linking containers. Creating a link between two containers creates a conduit between a source container and a target container and securely transfer information about source container to target container. In our case, target container (WildFly) can see information about source container (MySQL). The important part to understand here is that none of this information needs to be publicly exposed by the source container, and is only made available to the target container.

The magic switch to enable link is, intuitively, --link. So for example, if MySQL and WildFly containers are run as shown above, then --link mysqldb:db links the MySQL container named mysqldb with an alias db to the WildFly target container. This defines some environment variables, following the defined protocol, in the target container which can then be used to access information about the source container. For example, IP address, exposed ports, username, passwords, etc. The complete list of environment variables can be seen as:

So you can see there are DB_* environment variables providing plenty of information about source container.

Linking only works if all the containers are running on the same host. A better solution will be shown in the subsequent blog, stay tuned.

Override default Docker command

Dockerfile for this image inherits from jboss/wildfly:latest and starts the WildFly container. Docker containers can only run one command but we need to install JDBC driver, create JDBC resource using the correct IP address and port, and deploy the WAR file. So we will override the command by inheriting from jboss/wildfly:latest and use a custom command. This command will do everything that we want to do, and then start WildFly as well.

The custom command does the following:

  • Add MySQL module
  • Add MySQL JDBC driver
  • Add the JDBC data source using IP address and port of the linked MySQL container
  • Deploy the WAR file
  • Start WildFly container

Note, WildFly is starting with -b 0.0.0.0 that allows it to be bound to any IP address. Also, the command needs to run in foreground so that the container stays active.

Customizing security

Ideally, you’ll poke holes in the firewall to enable connection to specific host/ports. But these instructions were tried on Fedora 20 running in Virtual Box. So for convenience, the complete firewall was disabled as:

In addition, a Host-only adapter was added using Virtual Box settings and looks like:

techtip65-host-only-adapter

That’s it, that should get you going to to use WildFly and MySQL on two separate containers.

Also verified the steps on boot2docker, and it worked seamlessly there too:

Source code for the image is at github.com/arun-gupta/docker-images/tree/master/wildfly-mysql-javaee7.

Enjoy!