There are multiple ways to monitor Docker containers.
- Docker CLI provides the
docker container statsAPI that gives basic information about the running containers. - Docker Remote API provides more detailed information about the containers.
- Starting with Docker 1.13, there is an experimental feature with a Prometheus endpoint
- cAdvisor is an open source tool that provides last container usage and performance characteristics. This data can be stored in a time series database, such as InfluxDB. This data can then be shown in a fancy graph using a Kibana dashboard.
These options were covered in detail in an earlier blog.
There are other commercial options like Docker EE, Sysdig, Datadog, New Relic, App Dynamics and others. If you are running containers on AWS, then CloudWatch can provide integrated monitoring.
OSX is my primary development platform. But recently, I needed a way to monitor Docker containers using the Remote API (aka REST API) on a Windows machine. The output of the REST API is exactly same independent of the operating system. But the way to access the Docker REST API using curl is different on an OSX and a Windows machine. This blog will explain how to exactly access this API on these two operating systems.
Check out 1.27 swagger specification to learn more about the capabilities of the REST API. A nicer and a more readable version of the API can be seen using Swagger UI. This is broken until #32649 is fixed.
Complete details about how the REST API corresponds to different Docker versions is explained in Docker REST API Versioning.
We’ll dig into this a bit later but first let’s take a look on how this API can be accessed.
Docker Remote API on OSX
On OSX, curl connects using a Unix domain socket as shown:
|
1
2
3
|
curl --unix-sock /var/run/docker.sock http://localhost/containers/<name-or-id>/stats
|
A WildFly container can be started as:
|
1
2
3
|
docker run --name web -d jboss/wildfly
|
Stats can then be obtained using the command:
|
1
2
3
|
curl --unix-socket /var/run/docker.sock http://localhost/containers/web/stats
|
This will start printing stats as:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
|
{
"read":"2017-04-16T21:25:06.99499178Z",
"preread":"0001-01-01T00:00:00Z",
"pids_stats":{
"current":40
},
"blkio_stats":{
"io_service_bytes_recursive":[
{
"major":8,
"minor":0,
"op":"Read",
"value":0
},
{
"major":8,
"minor":0,
"op":"Write",
"value":4096
},
{
"major":8,
"minor":0,
"op":"Sync",
"value":0
},
{
"major":8,
"minor":0,
"op":"Async",
"value":4096
},
{
"major":8,
"minor":0,
"op":"Total",
"value":4096
}
],
"io_serviced_recursive":[
{
"major":8,
"minor":0,
"op":"Read",
"value":0
},
{
"major":8,
"minor":0,
"op":"Write",
"value":1
},
{
"major":8,
"minor":0,
"op":"Sync",
"value":0
},
{
"major":8,
"minor":0,
"op":"Async",
"value":1
},
{
"major":8,
"minor":0,
"op":"Total",
"value":1
}
],
"io_queue_recursive":[
],
"io_service_time_recursive":[
],
"io_wait_time_recursive":[
],
"io_merged_recursive":[
],
"io_time_recursive":[
],
"sectors_recursive":[
]
},
"num_procs":0,
"storage_stats":{
},
"cpu_stats":{
"cpu_usage":{
"total_usage":9569101779,
"percpu_usage":[
4920699125,
4648402654
],
"usage_in_kernelmode":1050000000,
"usage_in_usermode":7730000000
},
"system_cpu_usage":5431500000000,
"online_cpus":2,
"throttling_data":{
"periods":0,
"throttled_periods":0,
"throttled_time":0
}
},
"precpu_stats":{
"cpu_usage":{
"total_usage":0,
"usage_in_kernelmode":0,
"usage_in_usermode":0
},
"throttling_data":{
"periods":0,
"throttled_periods":0,
"throttled_time":0
}
},
"memory_stats":{
"usage":266448896,
"max_usage":285159424,
"stats":{
"active_anon":241647616,
"active_file":90112,
"cache":225280,
"dirty":20480,
"hierarchical_memory_limit":9223372036854772000,
"hierarchical_memsw_limit":9223372036854772000,
"inactive_anon":20971520,
"inactive_file":135168,
"mapped_file":32768,
"pgfault":81529,
"pgmajfault":1,
"pgpgin":76795,
"pgpgout":12624,
"rss":262619136,
"rss_huge":0,
"swap":57344,
"total_active_anon":241647616,
"total_active_file":90112,
"total_cache":225280,
"total_dirty":20480,
"total_inactive_anon":20971520,
"total_inactive_file":135168,
"total_mapped_file":32768,
"total_pgfault":81529,
"total_pgmajfault":1,
"total_pgpgin":76795,
"total_pgpgout":12624,
"total_rss":262619136,
"total_rss_huge":0,
"total_swap":57344,
"total_unevictable":0,
"total_writeback":0,
"unevictable":0,
"writeback":0
},
"limit":2096181248
},
"name":"/web",
"id":"b6ad4af36f05ce100877d2d60c72e01b87f3627682e4492e7abe2a099b709953",
"networks":{
"eth0":{
"rx_bytes":2764,
"rx_packets":50,
"rx_errors":0,
"rx_dropped":0,
"tx_bytes":0,
"tx_packets":0,
"tx_errors":0,
"tx_dropped":0
}
}
}
|
These stats are refreshed every second.
Any other REST API can be invoked very easily with this. This is simple and straight forward.
Install Docker on Windows
Docker installation on Windows depends on the flavor of your operating system.
Are you using Windows 10+ Pro 64-bit, then use Docker for Windows.
If using any older version of Windows, then you need to use Docker Toolbox.
Are you installing Windows in a virtual machine? Then Virtual Box cannot be used to create the VM. This is because Virtual Box does not support nested virtualization. This is required as Docker Toolbox uses Virtual Box to create and start Docker Machine. VMWare Fusion seems to work fine here though.
Now that you know that VMWare Fusion needs to be used, make sure you enable nested virtualization before starting the virtual machine.
Many thanks to Stefan Scherer (@stefscherer) for helping me understand the Windows configuration details.
Let’s see how the Docker REST API can now be invoked.
Docker Remote API on Windows 7/8
This section shows how to invoke the REST API using curl on Windows 7/8.
The REST API can be invoked using curl as shown:
|
1
2
3
|
curl --cert <CERT> --cacert <CA_CERT> --key <KEY> https://localhost/containers/<name-or-id>/stats
|
First of all, the REST API, /containers/<name-or-id>/stats, is exactly the same. The way this API can be invoked is a bit different.
There are four differences: The first three parameters specify security credentials for the Docker Machine generated by Docker Toolbox on your Windows box:
<CERT>is the SSL certificate for Docker Machine<CA_CERT>is the Certificate Authority certificate for the Docker Machine<KEY>is the key generated for the Docker Machine
The values of these configuration parameters is those generated by docker-machine CLI.
The last change is that the protocol is changed from http to https.
Here is the exact command that worked on Windows 7 VM:
|
1
2
3
|
$ curl --cert /c/Users/Arun\ Gupta/.docker/machine/machines/default/cert.pem --cacert /c/Users/Arun\ Gupta/.docker/machine/machines/default/ca.pem --key /c/Users/Arun\ Gupta/.docker/machine/machines/default/key.pem https://192.168.99.100:2376/containers/web/stats
|
This invocation will print the exact same stats output on Windows 7 VM.
Now that you know how to use this API on OSX and Windows, you can also this API to do everything that Docker CLI. This is because the Docker CLI is just a convenient wrapper over the REST API. So a docker container run command is invoking the appropriate REST API on the Docker Host.
Docker Remote API on Windows 10
If you are using Windows 10, then use Docker for Windows. After that, you need to figure out which curl command to be used. There are couple of options:
- Use Bash shell on Windows. It has
curlcommand that works like Unix command that we all know pretty well. In this case, the REST API can be invoked as:
Docker for Windows listens on port123curl http://localhost:2375/containers/web/stats2375on Windows. - If you are Powershell user, then install
curlcommand as:
Now the command to invoke the REST API is:123choco install -u curl
Note, there is a123curl.exe http://localhost:2375/containers/web/statscurlalias in Powershell and that is an alias forInvoke-WebRequest. So make sure to usecurl.exeto invoke the REST API as this is the command installed using Chocolatey.
This blog provided different options of how to invoke the Docker Remote API using curl on Windows and OSX.