This blog will show how a simple Java application can talk to a database using service discovery in DC/OS.
Why Service Discovery?
An application typically consist of multiple components such as an application server, a database, a web server, caching and messaging server. Typically, multiple replicas of each component would run based upon the needs of your application. Deploying this application using a container orchestration framework means that each replica would run as a container. So, an application is typically deployed as multi-container application.
Each container is assigned a unique IP address for its lifetime. But containers are ephemeral and may terminate and rescheduled on a different host by the orchestration framework. A container is typically assigned a different IP address in that case. This means an application deployed in application server cannot rely upon the IP address of the database. This is where service discovery is required.
So, multiple replicas of a component are assigned a logical name. For example, web for all the application server containers and db for all the database containers. Now, an application can talk to the database containers using the logical service name. This allows the database containers to be rescheduled anywhere in the cluster, and also scale up and down dynamically.
Let’s see how this can be accomplished in DC/OS with a single instance of application server and database server. This blog will use WildFly for application server and Couchbase for database.
Couchbase Cluster on Mesos with DC/OS provide more details on how to setup a Couchbase cluster on DC/OS.
This blog will use the following main steps:
- Setup DC/OS Cluster
- Marathon application definition
- Deploy the application
The complete source code used in this blog is at github.com/arun-gupta/dcos-java-database.
Many thanks to @unterstein for creating the Maven plugin and helping me understand the inner workings of DC/OS.
Setup DC/OS Cluster
DC/OS cluster can be easily created using the CloudFormation template. Detailed instructions, including system requirements and screenshots and setup, are available at Installing DC/OS on AWS.
CloudFormation output looks as shown:
Note down the value shown for the keys DnsAddress and PublicSlaveDnsAddress. The value of the first key can be used to access DC/OS GUI and looks like:
Configure DC/OS CLI as explained in CLI. In short, the following commands are used:
dcos config set core.dcos_url http://${DnsAddress}Replace${DnsAddress}with the corresponding value from the CloudFormation output.dcos auth logindcos config show core.dcos_acs_token. If not already done, clone the repo from github.com/arun-gupta/dcos-java-database. Create a new file.dcos-tokenand copy the output from the command in this file.dcos package install marathon-lb
Marathon Application Definition
Marathon framework is used to schedule containers in DC/OS. A marathon application can be defined by providing an application definition.
As mentioned earlier, this blog will show how a simple Java application can talk to a database. We’ll use a Java EE application deployed in WildFly and use Couchbase as the database. The application definition looks like:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
|
{
"id":"/webapp",
"apps":[
{
"id":"database",
"cpus":4,
"mem":4096,
"instances":1,
"container":{
"type":"DOCKER",
"docker":{
"image":"arungupta/couchbase:travel",
"network":"USER"
}
},
"ipAddress":{
"networkName":"dcos"
}
},
{
"id":"web",
"dependencies":[
"/webapp/database"
],
"cpus":2,
"mem":4096,
"instances":1,
"container":{
"type":"DOCKER",
"docker":{
"image":"arungupta/wildfly-couchbase-javaee:travel",
"network":"USER",
"portMappings":[
{
"hostPort":0,
"containerPort":8080,
"protocol":"tcp"
}
]
}
},
"ipAddress":{
"networkName":"dcos"
},
"env":{
"COUCHBASE_URI":"database-webapp.marathon.containerip.dcos.thisdcos.directory"
},
"labels":{
"HAPROXY_0_VHOST":"DCOS-PublicSlaveLo-DD2EGGFVCJA0-1604955948.us-west-1.elb.amazonaws.com",
"HAPROXY_GROUP":"external"
}
}
]
}
|
What are the key points in this application definition?
- Application has two containers:
databaseandweb. The web container has a dependency on the database container defined usingdependenciesattribute. databasecontainer usesarungupta/couchbase:travelDocker image. This image is created from github.com/arun-gupta/couchbase-javaee/tree/master/couchbase. It uses Couchbase base image and uses Couchbase REST API to pre-configure the database. A sample bucket is also loaded in the database as well.webcontainer usesarungupta/wildfly-couchbase-javaee:travelimage. This image is created from github.com/arun-gupta/couchbase-javaee/blob/master/Dockerfile. This is a Java EE 7 application bundled in WildFly. The app usesCOUCHBASE_URIas an environment variable to connect to the Couchbase database. The value of this environment variable is configured to use DNS service discovery and is derived as explained in Virtual Networks.
Make sure to change the value of HAPROXY_0_VHOST to match the value of ${PublicSlaveDnsAddress} from CloudFormation output. The label HAPROXY_0_VHOST instructs Marathon-LB to expose the Docker container, the WildFly application server in our case, on the external load balancer with a virtual host. The 0 in the label key corresponds to the servicePort index, beginning from 0. If you had multiple servicePort definitions, you would iterate them as 0, 1, 2, and so on. Deploying an internally and externally load-balanced app with marathon-lb provide more details about how to configure marathon-lb.
Service Discovery and Load Balancing provide more details about service discovery and load balancing in DC/OS.
Deploy the Application using Maven
The application can be deployed using dcos-maven-plugin.
Plugin looks like:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
<plugin>
<groupId>dcos</groupId>
<artifactId>dcos-maven-plugin</artifactId>
<version>0.2</version>
<configuration>
<dcosUrl>http://DCOS-ElasticLoadBa-1TH4TXIU5P783-1163025470.us-west-1.elb.amazonaws.com/</dcosUrl>
<deployable>group</deployable>
<ignoreSslCertificate>true</ignoreSslCertificate>
</configuration>
<executions>
<execution>
<id>dcos:deploy</id>
<phase>install</phase>
<goals>
<goal>deploy</goal>
</goals>
</execution>
</executions>
</plugin>
|
Main points in this fragment are:
- Plugin version is 0.2. That indicates the plugin is still in early stages of development.
dcosUrlis the value of${DnsAddress}key from CloudFormation output. This address is used for deployment of the application.<deployable>element enables different types of deployment – app, group or pods. This element is a hint for the plugin and should likely go away in a future version as Marathon API consolidates. Follow #11 for more details.
Other details and configuration about the plugin are at dcos-maven-plugin.
Deploy the application:
|
1
2
3
|
mvn install
|
The following output is shown:
|
1
2
3
4
5
6
7
8
9
10
|
[INFO] --- dcos-maven-plugin:0.2:deploy (dcos:deploy) @ dcos-java-database ---
[INFO] About to execute DC/OS deploy
[INFO] app definition: /Users/arungupta/workspaces/dcos-java-database/app-definition.json
[INFO] dcos token: /Users/arungupta/workspaces/dcos-java-database/.dcos-token
[INFO] dcos url: http://DCOS-ElasticLoadBa-1TH4TXIU5P783-1163025470.us-west-1.elb.amazonaws.com/
[INFO] ignore ssl certificate: true
[INFO] deployable: group
[INFO] Response from DC/OS [200] {"version":"2017-03-07T13:27:49.970Z","deploymentId":"dbc80f96-28cb-4040-8d0d-78452e461ec1"}
|
Here are some of the updated output from DC/OS console.
First updated Services tab:
Two applications in the service:
Database application has one task:
Status of database task:
Logs from the database task:
It shows the output from Couchbase REST API for configuring the server.
Status of web task:
Logs from web task:
It shows that the Java EE application is deployed successfully.
Access the application:
|
1
2
3
|
curl http://DCOS-PublicSlaveLo-DD2EGGFVCJA0-1604955948.us-west-1.elb.amazonaws.com/airlines/resources/airline
|
The address is the value of the key ${PublicSlaveDnsAddress} from CloudFormation output. A formatted output, for example with jq, looks like:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
[
{
"travel-sample": {
"country": "United States",
"iata": "Q5",
"callsign": "MILE-AIR",
"name": "40-Mile Air",
"icao": "MLA",
"id": 10,
"type": "airline"
}
},
{
"travel-sample": {
"country": "United States",
. . .
"icao": "RLA",
"id": 1203,
"type": "airline"
}
}
]
|
That’s it!
As mentioned earlier, the complete source code used in this blog is at github.com/arun-gupta/dcos-java-database.
This blog showed how a simple Java application can talk to a database using service discovery in DC/OS.
For further information check out:
- DC/OS Docs
- Couchbase on Containers
- Couchbase Developer Portal
- Ask questions on Couchbase Forums or Stack Overflow
- Download Couchbase
Source: blog.couchbase.com/service-discovery-java-database-dcos/
Thanks for sharing! It’s very useful!
Hi,
I wondered why we can’t just use a load balancer like haproxy to do the service registration. It publishes a logical service name that would forward the request to one of the services available on cs/os. Why the fuss with service registration? It’s dynamic, it’s scalable and it’s technology transparent.
Best regards,
Stevenson
I just need this information , really thanks for sharing this.
Valuable post.I found this is more informative,Thank you so much to share.
nice post!!
filmywap 2018
hey guys if you are a gamer and want some amazing adventures game then try roblox robux its amazing open my link to get it for free
I like this post, it is very useful for us. Thank you!