MySQL as Kubernetes Service, Access from WildFly Pod (Tech Tip #72)

Java EE 7 and WildFly on Kubernetes using Vagrant (Tech Tip #71) explained how to run a trivial Java EE 7 application on WildFly hosted using Kubernetes and Docker. The Java EE 7 application was the hands-on lab that have been delivered around the world. It uses an in-memory database that is bundled with WildFly and allows to understand the key building blocks of Kubernetes. This is good to get you started with initial development efforts but quickly becomes a bottleneck as the database is lost when the application server goes down. This tech tip will show how to run another trivial Java EE 7 application and use MySQL as the database server. It will use Kubernetes Services to explain how MySQL and WildFly can be easily decoupled.

Lets get started!

Make sure to have a working Kubernetes setup as explained in Kubernetes using Vagrant.

The complete source code used in this blog is available at github.com/arun-gupta/kubernetes-java-sample.

Start MySQL Kubernetes pod

First step is to start the MySQL pod. This can be started by using the MySQL Kubernetes configuration file:

The configuration file used is at github.com/arun-gupta/kubernetes-java-sample/blob/master/mysql.json.

Check the status of MySQL pod:

Wait till the status changes to “Running”. It will look like:

It takes a few minutes for MySQL server to be in that state, so grab a coffee or a quick fast one miler!

Start MySQL Kubernetes service

Pods, and the IP addresses assigned to them, are ephemeral. If a pod dies then Kubernetes will recreate that pod because of its self-healing features, but it might recreate it on a different host. Even if it is on the same host, a different IP address could be assigned to it. And so any application cannot rely upon the IP address of the pod.

Kubernetes services is an abstraction which defines a logical set of pods. A service is typically back-ended by one or more physical pods (associated using labels), and it has a permanent IP address that can be used by other pods/applications. For example, WildFly pod can not directly connect to a MySQL pod but can connect to MySQL service. In essence, Kubernetes service offers clients an IP and port pair which, when accessed, redirects to the appropriate backends.

Kubernetes Services

Lets start MySQL service.

The configuration file used is at github.com/arun-gupta/kubernetes-java-sample/blob/master/mysql-service.json. In this case, only a single MySQL instance is started. But multiple MySQL instances can be easily started and WildFly Pod will continue to refer to all of them using MySQL Service.

Check the status/IP of the MySQL service:

Start WildFly Kubernetes Pod

WildFly Pod must be started after MySQL service has started. This is because the environment variables used for creating JDBC resource in WildFly are only available after the service is up and running. Specifically, the JDBC resource is created as:

$MYSQL_SERVICE_HOST and $MYSQL_SERVICE_PORT environment variables are populated by Kubernetes as explained here.

This is shown at github.com/arun-gupta/docker-images/blob/master/wildfly-mysql-javaee7/customization/execute.sh#L44.

Start WildFly pod:

The configuration file used is at github.com/arun-gupta/kubernetes-java-sample/blob/master/wildfly.json.

Check the status of pods:

Wait until WildFly pod’s status is changed to Running. This could be a few minutes, so may be time to grab another quick miler!

Once the container is up and running, you can check /opt/jboss/wildfly/standalone/configuration/standalone.xml in the WildFly container and verify that the connection URL indeed contains the correct IP address. Here is how it looks on my machine:

The updated status (after the container is running) would look like as shown:

Access the Java EE 7 Application

Note down the HOST IP address of the WildFly container and access the application as:

to see the output as:

Or viewed in the browser as:

Java EE 7 Application using WildFly, MySQL, and Kubernetes

Debugging Kubernetes and Docker

Login to the Minion-1 VM:

Log in as root:

Default root password for VM images created by Vagrant is “vagrant”.

List of Docker containers running on this VM can be seen as:

Last 10 lines of the WildFly log (after application has been accessed a few times) can be seen as:

Similarly, MySQL log is seen as:

Enjoy!

 

Be Sociable, Share!
  • Tweet

8 thoughts on “MySQL as Kubernetes Service, Access from WildFly Pod (Tech Tip #72)

  1. Thanks for your post!

    So you start the WildFly Pod after the MySQL service has started, in order to get the environment variables (IP+port) populated by Kubernetes.

    Somethings remains unclear, if the MySQL pod is recreated, do you have to manually restart the WildFly pod (or at least, force re-access the service)?

  2. OK, I think i have found my answer here : https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/services.md

    The IP provided by the service itself does not change, but the service redirect to the correct database IP if automatically restarted.

  3. Hy Arun

    I asked myself what would be a good solution if I try to replicate the POD with the Database in a multi-node Kubernetes cluster.

    There is the possibility of shared volumes. HostDir could not work in a multi-node environment, GCEPersistentDisk would be a vendor lock-in and therefore I think the best fit at the moment would be with NFS. Do you agree with that?

    Greets Rob

  4. Maybe somebody did not notice because of the great article but the names on the web browser appear to be characters of #BBT.Good one Arun.

    Ramit Surana

  5. Hi Arun,

    Why did you use host IP address 10.245.1.3:8080 to access the WildFly? Can you use the WildFly’s pod IP address 10.246.4:8080 to access? If using pod’s IP address to access container is not possible, then why adding a layer of POD IP address? It seems very puzzling. Appreciate your answer.

  6. Sherry,

    Pod’s IP address is accessible within the Kubernetes cluster only. 10.245.1.3 is the IP address of the host where WildFly pod is started and thus that IP address is accessible for Mac host.

  7. Hello Arun,

    Thanks for the tutorial. But I am facing one issue, after running all the pods in sequence, I am not able to hit the wildfly url from the respective minion. After checking the wildfly container logs, I got the following error, can you please provide the necessary guidance;

    07:14:01,328 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0019: Stopped Driver service with driver-name = mysql
    07:14:01,331 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0019: Stopped Driver service with driver-name = h2
    07:14:01,460 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow HTTP listener default suspending
    07:14:01,463 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow HTTP listener default stopped, was bound to /0.0.0.0:8080
    07:14:01,465 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0004: Undertow 1.2.8.Final stopping
    07:14:01,466 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0028: Stopped deployment employees.war (runtime-name: employees.war) in 190ms
    07:14:01,498 INFO [org.jboss.as] (MSC service thread 1-2) WFLYSRV0050: WildFly Full 9.0.0.Final (WildFly Core 1.0.0.Final) stopped in 209ms
    =========================================================================

    JBoss Bootstrap Environment

    JBOSS_HOME: /opt/jboss/wildfly

    JAVA: /usr/lib/jvm/java/bin/java

    JAVA_OPTS: -server -XX:+UseCompressedOops -server -XX:+UseCompressedOops -Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

    =========================================================================

    OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
    07:14:02,425 INFO [org.jboss.modules] (main) JBoss Modules version 1.4.3.Final
    07:14:03,008 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
    07:14:03,539 INFO [org.jboss.as] (MSC service thread 1-1) WFLYSRV0049: WildFly Full 9.0.0.Final (WildFly Core 1.0.0.Final) starting
    07:14:05,884 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool — 23) WFLYCTL0028: Attribute ‘job-repository-type’ in the resource at address ‘/subsystem=batch’ is deprecated, and may be removed in future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
    07:14:05,914 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool — 21) WFLYCTL0028: Attribute ‘enabled’ in the resource at address ‘/subsystem=datasources/data-source=ExampleDS’ is deprecated, and may be removed in future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
    07:14:05,922 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool — 21) WFLYCTL0028: Attribute ‘enabled’ in the resource at address ‘/subsystem=datasources/data-source=mysqlDS’ is deprecated, and may be removed in future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
    07:14:05,959 INFO [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads – 1) WFLYDS0015: Re-attempting failed deployment employees.war
    07:14:06,074 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
    07:14:06,132 INFO [org.xnio] (MSC service thread 1-1) XNIO version 3.3.1.Final
    07:14:06,164 INFO [org.xnio.nio] (MSC service thread 1-1) XNIO NIO Implementation Version 3.3.1.Final
    07:14:06,272 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool — 38) WFLYCLINF0001: Activating Infinispan subsystem.
    07:14:06,336 INFO [org.jboss.as.jsf] (ServerService Thread Pool — 44) WFLYJSF0007: Activated the following JSF Implementations: [main]
    07:14:06,358 INFO [org.jboss.remoting] (MSC service thread 1-1) JBoss Remoting version 4.0.9.Final
    07:14:06,383 INFO [org.jboss.as.naming] (ServerService Thread Pool — 46) WFLYNAM0001: Activating Naming Subsystem
    07:14:06,373 INFO [org.wildfly.extension.io] (ServerService Thread Pool — 37) WFLYIO001: Worker ‘default’ has auto-configured to 2 core threads with 16 task threads based on your 1 available processors
    07:14:06,416 INFO [org.jboss.as.security] (ServerService Thread Pool — 53) WFLYSEC0002: Activating Security Subsystem
    07:14:06,414 WARN [org.jboss.as.txn] (ServerService Thread Pool — 54) WFLYTX0013: Node identifier property is set to the default value. Please make sure it is unique.
    07:14:06,465 INFO [org.jboss.as.webservices] (ServerService Thread Pool — 56) WFLYWS0002: Activating WebServices Extension
    07:14:06,520 INFO [org.jboss.as.security] (MSC service thread 1-2) WFLYSEC0001: Current PicketBox version=4.9.2.Final
    07:14:06,598 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool — 55) WFLYUT0003: Undertow 1.2.8.Final starting
    07:14:06,627 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool — 33) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
    07:14:06,628 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0003: Undertow 1.2.8.Final starting
    07:14:06,702 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool — 33) WFLYJCA0005: Deploying non-JDBC-compliant driver class com.mysql.jdbc.Driver (version 5.1)
    07:14:06,717 INFO [org.jboss.as.connector] (MSC service thread 1-2) WFLYJCA0009: Starting JCA Subsystem (IronJacamar 1.2.4.Final)
    07:14:06,939 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0018: Started Driver service with driver-name = h2
    07:14:06,959 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0018: Started Driver service with driver-name = mysql
    07:14:06,965 INFO [org.jboss.as.naming] (MSC service thread 1-1) WFLYNAM0003: Starting Naming Service
    07:14:06,969 INFO [org.jboss.as.mail.extension] (MSC service thread 1-2) WFLYMAIL0001: Bound mail session [java:jboss/mail/Default]
    07:14:07,103 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool — 55) WFLYUT0014: Creating file handler for path /opt/jboss/wildfly/welcome-content
    07:14:07,192 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0012: Started server default-server.
    07:14:07,213 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0018: Host default-host starting
    07:14:07,616 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0006: Undertow HTTP listener default listening on /0.0.0.0:8080
    07:14:08,063 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleMySQLDS]
    07:14:08,064 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS]
    07:14:08,295 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-1) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/jboss/wildfly/standalone/deployments
    07:14:08,314 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0027: Starting deployment of “employees.war” (runtime-name: “employees.war”)
    07:14:08,431 INFO [org.jboss.ws.common.management] (MSC service thread 1-2) JBWS022052: Starting JBoss Web Services – Stack CXF Server 5.0.0.Final
    07:14:08,701 INFO [org.jboss.as.jpa] (MSC service thread 1-1) WFLYJPA0002: Read persistence.xml for MyPU
    07:14:08,845 INFO [org.jboss.as.jpa] (ServerService Thread Pool — 58) WFLYJPA0010: Starting Persistence Unit (phase 1 of 2) Service ’employees.war#MyPU’
    07:14:08,877 INFO [org.hibernate.jpa.internal.util.LogHelper] (ServerService Thread Pool — 58) HHH000204: Processing PersistenceUnitInfo [
    name: MyPU
    …]
    07:14:08,914 INFO [org.jboss.weld.deployer] (MSC service thread 1-2) WFLYWELD0003: Processing weld deployment employees.war
    07:14:09,063 INFO [org.hibernate.Version] (ServerService Thread Pool — 58) HHH000412: Hibernate Core {4.3.10.Final}
    07:14:09,079 INFO [org.hibernate.cfg.Environment] (ServerService Thread Pool — 58) HHH000206: hibernate.properties not found
    07:14:09,082 INFO [org.hibernate.cfg.Environment] (ServerService Thread Pool — 58) HHH000021: Bytecode provider name : javassist
    07:14:09,103 INFO [org.hibernate.validator.internal.util.Version] (MSC service thread 1-2) HV000001: Hibernate Validator 5.1.3.Final
    07:14:09,651 INFO [org.jboss.weld.deployer] (MSC service thread 1-1) WFLYWELD0006: Starting Services for CDI deployment: employees.war
    07:14:09,748 INFO [org.jboss.weld.Version] (MSC service thread 1-1) WELD-000900: 2.2.14 (Final)
    07:14:09,840 INFO [org.jboss.weld.deployer] (MSC service thread 1-1) WFLYWELD0009: Starting weld service for deployment employees.war
    07:14:10,283 INFO [org.jboss.as.jpa] (ServerService Thread Pool — 58) WFLYJPA0010: Starting Persistence Unit (phase 2 of 2) Service ’employees.war#MyPU’
    07:14:10,547 INFO [org.hibernate.annotations.common.Version] (ServerService Thread Pool — 58) HCANN000001: Hibernate Commons Annotations {4.0.5.Final}
    07:14:10,859 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (ServerService Thread Pool — 58) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection
    at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:336)
    at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:343)
    at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:278)
    at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:1224)
    at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:484)
    at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:622)
    at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:594)
    at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:579)
    at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:430)
    at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:737)
    at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:138)
    at org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:67)
    at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:139)
    at org.hibernate.engine.jdbc.internal.JdbcServicesImpl$ConnectionProviderJdbcConnectionAccess.obtainConnection(JdbcServicesImpl.java:279)
    at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:124)
    at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:111)
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:234)
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:206)
    at org.hibernate.cfg.Configuration.buildTypeRegistrations(Configuration.java:1887)
    at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1845)
    at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:857)
    at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl$4.perform(EntityManagerFactoryBuilderImpl.java:850)
    at org.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl.withTccl(ClassLoaderServiceImpl.java:425)
    at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:849)
    at org.jboss.as.jpa.hibernate4.TwoPhaseBootstrapImpl.build(TwoPhaseBootstrapImpl.java:44)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:154)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:117)
    at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:665)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:182)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:320)
    Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

Leave a Reply

Your email address will not be published. Required fields are marked *