One of the key updates as part of Docker 1.6 is Docker Swarm 0.2.0. Docker Swarm solves one of the fundamental limitations of Docker where the containers could only run on a single Docker host. Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single, virtual host.

This Tech Tip will show how to create a cluster across multiple hosts with Docker Swarm.

A good introduction to Docker Swarm is by @aluzzardi and @vieux from Container Camp:

Key Components of Docker Swarm

Swarm Manager: Docker Swarm has a Master or Manager, that is a pre-defined Docker Host, and is a single point for all administration. Currently only a single instance of manager is allowed in the cluster. This is a SPOF for high availability architectures and additional managers will be allowed in a future version of Swarm with #598.

Swarm Nodes: The containers are deployed on Nodes that are additional Docker Hosts. Each Swarm Node  must be accessible by the manager, each node must listen to the same network interface (TCP port). Each node runs a node agent that registers the referenced Docker daemon, monitors it, and updates the discovery backend with the node’s status. The containers run on a node.

Scheduler Strategy: Different scheduler strategies (binpack, spread, and random) can be applied to pick the best node to run your container. The default strategy is spread which optimizes the node for least number of running containers. There are multiple kinds of filters, such as constraints and affinity.  This should allow for a decent scheduling algorithm.

Node Discovery Service: By default, Swarm uses hosted discovery service, based on Docker Hub, using tokens to discover nodes that are part of a cluster. However etcd, consul, and zookeeper can be also be used for service discovery as well. This is particularly useful if there is no access to Internet, or you are running the setup in a closed network. A new discovery backend can be created as explained here. It would be useful to have the hosted Discovery Service inside the firewall and #660 will discuss this.

Standard Docker API: Docker Swarm serves the standard Docker API and thus any tool that talks to a single Docker host will seamlessly scale to multiple hosts now. That means if you were using shell scripts using Docker CLI to configure multiple Docker hosts, the same CLI would can now talk to Swarm cluster and Docker Swarm will then act as proxy and run it on the cluster.

There are lots of other concepts but these are the main ones.

TL;DR Here is a simple script that will create a boilerplate cluster with a master and two nodes:

Lets dig into the details now!

Create Swarm Cluster

Create a Swarm cluster as:

This command returns a token and is the unique cluster id. It will be used when creating master and nodes later. As mentioned earlier, this cluster id is returned by the hosted discovery service on Docker Hub.

Make sure to note this cluster id now as there is no means to list it later. #661 should fix this.

Create Swarm Master

Swarm is fully integrated with Docker Machine, and so is the easiest way to get started on OSX.

1. Create Swarm master as:
   
   ~> docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery token://117c8d19ba140d7ba3259aae9012e22f swarm-master INFO[0000] Creating SSH key... INFO[0000] Creating VirtualBox VM... INFO[0005] Starting VirtualBox VM... INFO[0006] Waiting for VM to start... INFO[0060] "swarm-master" has been created and is now the active machine. INFO[0060] To point your Docker client at it, run this in your shell: eval "$(docker-machine env swarm-master)"

   1 2 3 4 5 6 7 8 9

   ~> docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery token://117c8d19ba140d7ba3259aae9012e22f swarm-master INFO[0000] Creating SSH key...                           INFO[0000] Creating VirtualBox VM...                     INFO[0005] Starting VirtualBox VM...                     INFO[0006] Waiting for VM to start...                   INFO[0060] "swarm-master" has been created and is now the active machine. INFO[0060] To point your Docker client at it, run this in your shell: eval "$(docker-machine env swarm-master)"

   --swarm configures the machine with Swarm, --swarm-master configures the created machine to be Swarm master. Make sure to replace cluster id after token:// with that obtained in the previous step. Swarm master creation talks to the hosted service on Docker Hub and informs that a master is created in the cluster.

   There should be an option to make an existing machine as Swarm master. This is reported as #1017.

2. List all the running machines as:
   ~> docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM swarm-master * virtualbox Running tcp://192.168.99.108:2376 swarm-master (master)

   1 2 3 4 5

   ~> docker-machine ls NAME           ACTIVE   DRIVER       STATE     URL                         SWARM swarm-master   *        virtualbox   Running   tcp://192.168.99.108:2376   swarm-master (master)

   Notice, how swarm-master is marked as master.

   Seems like the cluster name is derived from the master’s name. There should be an option to specify the cluster name, likely during cluster creation. This is reported as #1018.

3. Connect to this newly created master and find some more information about it:
   ~> eval "$(docker-machine env swarm-master)" ~> docker info Containers: 2 Images: 8 Storage Driver: aufs Root Dir: /mnt/sda1/var/lib/docker/aufs Backing Filesystem: extfs Dirs: 12 Dirperm1 Supported: true Execution Driver: native-0.2 Kernel Version: 3.18.11-tinycore64 Operating System: Boot2Docker 1.6.0 (TCL 5.4); master : a270c71 - Thu Apr 16 19:50:36 UTC 2015 CPUs: 8 Total Memory: 999.4 MiB Name: swarm-master ID: UAEO:HLG6:2XOF:QQH7:GTGW:XW6K:ZILW:RY57:JSEY:2PHI:4OHE:QMVW Debug mode (server): true Debug mode (client): false Fds: 25 Goroutines: 38 System Time: Thu Apr 23 02:15:55 UTC 2015 EventsListeners: 1 Init SHA1: 9145575052383dbf64cede3bac278606472e027c Init Path: /usr/local/bin/docker Docker Root Dir: /mnt/sda1/var/lib/docker Username: arungupta Registry: [https://index.docker.io/v1/] Labels: provider=virtualbox

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

   ~> eval "$(docker-machine env swarm-master)" ~> docker info Containers: 2 Images: 8 Storage Driver: aufs Root Dir: /mnt/sda1/var/lib/docker/aufs Backing Filesystem: extfs Dirs: 12 Dirperm1 Supported: true Execution Driver: native-0.2 Kernel Version: 3.18.11-tinycore64 Operating System: Boot2Docker 1.6.0 (TCL 5.4); master : a270c71 - Thu Apr 16 19:50:36 UTC 2015 CPUs: 8 Total Memory: 999.4 MiB Name: swarm-master ID: UAEO:HLG6:2XOF:QQH7:GTGW:XW6K:ZILW:RY57:JSEY:2PHI:4OHE:QMVW Debug mode (server): true Debug mode (client): false Fds: 25 Goroutines: 38 System Time: Thu Apr 23 02:15:55 UTC 2015 EventsListeners: 1 Init SHA1: 9145575052383dbf64cede3bac278606472e027c Init Path: /usr/local/bin/docker Docker Root Dir: /mnt/sda1/var/lib/docker Username: arungupta Registry: [https://index.docker.io/v1/] Labels: provider=virtualbox

Create Swarm Nodes

1. Create a swarm node as:
   ~> docker-machine create -d virtualbox --swarm --swarm-discovery token://117c8d19ba140d7ba3259aae9012e22f swarm-node-01 INFO[0000] Creating SSH key... INFO[0000] Creating VirtualBox VM... INFO[0006] Starting VirtualBox VM... INFO[0006] Waiting for VM to start... INFO[0070] "swarm-node-01" has been created and is now the active machine. INFO[0070] To point your Docker client at it, run this in your shell: eval "$(docker-machine env swarm-node-01)"

   1 2 3 4 5 6 7 8 9

   ~> docker-machine create -d virtualbox --swarm --swarm-discovery token://117c8d19ba140d7ba3259aae9012e22f swarm-node-01 INFO[0000] Creating SSH key...                           INFO[0000] Creating VirtualBox VM...                     INFO[0006] Starting VirtualBox VM...                     INFO[0006] Waiting for VM to start...                   INFO[0070] "swarm-node-01" has been created and is now the active machine. INFO[0070] To point your Docker client at it, run this in your shell: eval "$(docker-machine env swarm-node-01)"

   Once again, node creation talks to the hosted service at Docker Hub and joins the previously created cluster. This is specified by --swarm-discovery token://... and specifying the cluster id obtained earlier.

2.  Create another Swarm node as:
   ~> docker-machine create -d virtualbox --swarm --swarm-discovery token://117c8d19ba140d7ba3259aae9012e22f swarm-node-02 INFO[0000] Creating SSH key... INFO[0000] Creating VirtualBox VM... INFO[0006] Starting VirtualBox VM... INFO[0006] Waiting for VM to start... INFO[0061] "swarm-node-02" has been created and is now the active machine. INFO[0061] To point your Docker client at it, run this in your shell: eval "$(docker-machine env swarm-node-02)"

   1 2 3 4 5 6 7 8 9

   ~> docker-machine create -d virtualbox --swarm --swarm-discovery token://117c8d19ba140d7ba3259aae9012e22f swarm-node-02 INFO[0000] Creating SSH key...                           INFO[0000] Creating VirtualBox VM...                     INFO[0006] Starting VirtualBox VM...                     INFO[0006] Waiting for VM to start...                   INFO[0061] "swarm-node-02" has been created and is now the active machine. INFO[0061] To point your Docker client at it, run this in your shell: eval "$(docker-machine env swarm-node-02)"

3. List all the existing Docker machines:
   ~> docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM mydocker virtualbox Running tcp://192.168.99.107:2376 swarm-master virtualbox Running tcp://192.168.99.108:2376 swarm-master (master) swarm-node-01 virtualbox Running tcp://192.168.99.109:2376 swarm-master swarm-node-02 * virtualbox Running tcp://192.168.99.110:2376 swarm-master

   1 2 3 4 5 6 7 8

   ~> docker-machine ls NAME            ACTIVE   DRIVER       STATE     URL                         SWARM mydocker                 virtualbox   Running   tcp://192.168.99.107:2376   swarm-master             virtualbox   Running   tcp://192.168.99.108:2376   swarm-master (master) swarm-node-01            virtualbox   Running   tcp://192.168.99.109:2376   swarm-master swarm-node-02   *        virtualbox   Running   tcp://192.168.99.110:2376   swarm-master

   The machines that are part of the cluster have the cluster’s name in the SWARM column, blank otherwise. For example, mydocker is a standalone machine where as all other machines are part of swarm-master cluster. The Swarm master is also identified by (master) in the SWARM column.

4. Connect to the Swarm cluster and find some information about it:
   ~> eval "$(docker-machine env --swarm swarm-master)" ~> docker info Containers: 4 Strategy: spread Filters: affinity, health, constraint, port, dependency Nodes: 3 swarm-master: 192.168.99.108:2376 └ Containers: 2 └ Reserved CPUs: 0 / 8 └ Reserved Memory: 0 B / 1.025 GiB swarm-node-01: 192.168.99.109:2376 └ Containers: 1 └ Reserved CPUs: 0 / 8 └ Reserved Memory: 0 B / 1.025 GiB swarm-node-02: 192.168.99.110:2376 └ Containers: 1 └ Reserved CPUs: 0 / 8 └ Reserved Memory: 0 B / 1.025 GiB

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

   ~> eval "$(docker-machine env --swarm swarm-master)" ~> docker info Containers: 4 Strategy: spread Filters: affinity, health, constraint, port, dependency Nodes: 3 swarm-master: 192.168.99.108:2376   └ Containers: 2   └ Reserved CPUs: 0 / 8   └ Reserved Memory: 0 B / 1.025 GiB swarm-node-01: 192.168.99.109:2376   └ Containers: 1   └ Reserved CPUs: 0 / 8   └ Reserved Memory: 0 B / 1.025 GiB swarm-node-02: 192.168.99.110:2376   └ Containers: 1   └ Reserved CPUs: 0 / 8   └ Reserved Memory: 0 B / 1.025 GiB

   There are 3 nodes – one Swarm master and 2 Swarm nodes. There is a total of 4 containers running in this cluster – one Swarm agent on master and each node, and there is an additional swarm-agent-master running on the master. This can be verified by connecting to the master and listing all the containers:

   ~> eval "$(docker-machine env swarm-master)" ~> docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 458c58f93a2b swarm:latest "/swarm join --addr 19 minutes ago Up 19 minutes 2375/tcp swarm-agent 0c80a04859ba swarm:latest "/swarm manage --tls 19 minutes ago Up 19 minutes 2375/tcp, 0.0.0.0:3376->3376/tcp swarm-agent-master

   1 2 3 4 5 6 7

   ~> eval "$(docker-machine env swarm-master)" ~> docker ps CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS                              NAMES 458c58f93a2b        swarm:latest        "/swarm join --addr    19 minutes ago      Up 19 minutes       2375/tcp                           swarm-agent           0c80a04859ba        swarm:latest        "/swarm manage --tls   19 minutes ago      Up 19 minutes       2375/tcp, 0.0.0.0:3376->3376/tcp   swarm-agent-master

5. Configure the Docker client to connect to Swarm cluster and check the list of running containers:
   ~> eval "$(docker-machine env --swarm swarm-master)" ~> docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

   1 2 3 4 5

   ~> eval "$(docker-machine env --swarm swarm-master)" ~> docker ps CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

   No application containers are running in the cluster, as expected.

6. List the nodes in the cluster as:
   ~> docker run swarm list token://117c8d19ba140d7ba3259aae9012e22f 192.168.99.108:2376 192.168.99.109:2376 192.168.99.110:2376

   1 2 3 4 5 6

   ~> docker run swarm list token://117c8d19ba140d7ba3259aae9012e22f 192.168.99.108:2376 192.168.99.109:2376 192.168.99.110:2376

A subsequent blog will show how to run multiple containers across hosts on this cluster, and also look into different scheduling strategies.

Scaling Docker with Swarm has good details.

Swarm is not fully integrated with Docker Compose yet. But what would be really cool is when I can specify all the Docker Machine descriptions in docker-compose.yml, in addition to the containers. Then docker-compose up -d would setup the cluster and run the containers in that cluster.

Be Sociable, Share!

• 

•