Yearly Archives: 2016

Microservice using AWS API Gateway, AWS Lambda and Couchbase

December 31, 2016couchbaseamazon, apigateway, aws, couchbase, ec2, lambda, serverlessarungupta

This blog has explained the following concepts for serverless applications so far:

Serverless FaaS with AWS Lambda and Java
AWS IoT Button, Lambda and Couchbase

The third blog in serverless series will explain how to create a simple microservice using Amazon API Gateway, AWS Lambda and Couchbase.

Read previous blogs for more context on AWS Lambda.

Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. Amazon API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, authorization and access control, monitoring, and API version management.

Here are the key components in this architecture:

Client could be curl, AWS CLI, Postman client or any other tool/API that can invoke a REST endpoint.
API Gateway is used to provision APIs. The top level resource is available at path /books. HTTP GET and POST methods are published for the resource.
Each API triggers a Lambda function. Two Lambda functions are created, book-list function for listing all the books available and book-create function to create a new book.
Couchbase is used as a persistence store in EC2. All the JSON documents are stored and retrieved from this database.

Let’s get started!

Create IAM Role

IAM roles will have policies and trust relationships that will allow this role to be used in API Gateway and execute Lambda function.

Let’s create a new IAM role:

aws iam create-role \
--role-name microserviceRole \
--assume-role-policy-document file://./trust.json

aws iam create-role \

--role-name microserviceRole \

--assume-role-policy-document file://./trust.json

--assume-role-policy-document defines the trust relationship policy document that grants an entity permission to assume the role. trust.json is at github.com/arun-gupta/serverless/blob/master/aws/microservice/trust.json and looks like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "lambda.amazonaws.com",
          "apigateway.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

{

"Version": "2012-10-17",

"Statement": [

{

"Sid": "",

"Effect": "Allow",

"Principal": {

"Service": [

"lambda.amazonaws.com",

"apigateway.amazonaws.com"

]

"Action": "sts:AssumeRole"

}

]

}

This trust relationship allows Lambda functions and API Gateway to assume this role during execution.

Associate policies with this role as:

aws iam put-role-policy \
--role-name microserviceRole \
--policy-name microPolicy \
--policy-document file://./policy.json

aws iam put-role-policy \

--role-name microserviceRole \

--policy-name microPolicy \

--policy-document file://./policy.json

policy.json is at github.com/arun-gupta/serverless/blob/master/aws/microservice/policy.json and looks like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:*"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "apigateway:*"
      ],
      "Resource": "arn:aws:apigateway:*::/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "execute-api:Invoke"
      ],
      "Resource": "arn:aws:execute-api:*:*:*"
    },
    {
      "Effect": "Allow",
      "Action": [
          "lambda:*"
      ],
      "Resource": "*"
    }
  ]
}

{

"Version": "2012-10-17",

"Statement": [

{

"Effect": "Allow",

"Action": [

"logs:*"

"Resource": "arn:aws:logs:*:*:*"

{

"Effect": "Allow",

"Action": [

"apigateway:*"

"Resource": "arn:aws:apigateway:*::/*"

{

"Effect": "Allow",

"Action": [

"execute-api:Invoke"

"Resource": "arn:aws:execute-api:*:*:*"

{

"Effect": "Allow",

"Action": [

"lambda:*"

"Resource": "*"

}

]

}

This generous policy allows any permissions over logs generated in CloudWatch for all resources. In addition it allows all Lambda and API Gateway permissions to all resources. In general, only required policy would be given to specific resources.

Create Lambda Functions

Detailed steps to create Lambda functions are explained in Serverless FaaS with AWS Lambda and Java. Let’s create the two Lambda functions as required in our case:

aws lambda create-function \
--function-name MicroserviceGetAll \
--role arn:aws:iam::598307997273:role/microserviceRole \
--handler org.sample.serverless.aws.couchbase.BucketGetAll \
--zip-file fileb:///Users/arungupta/workspaces/serverless/aws/microservice/microservice-http-endpoint/target/microservice-http-endpoint-1.0-SNAPSHOT.jar \
--description "Microservice HTTP Endpoint - Get All" \
--runtime java8 \
--region us-west-1 \
--timeout 30 \
--memory-size 1024 \
--environment Variables={COUCHBASE_HOST=ec2-52-53-193-176.us-west-1.compute.amazonaws.com} \
--publish

aws lambda create-function \

--function-name MicroserviceGetAll \

--role arn:aws:iam::598307997273:role/microserviceRole \

--handler org.sample.serverless.aws.couchbase.BucketGetAll \

--zip-file fileb:///Users/arungupta/workspaces/serverless/aws/microservice/microservice-http-endpoint/target/microservice-http-endpoint-1.0-SNAPSHOT.jar \

--description "Microservice HTTP Endpoint - Get All" \

--runtime java8 \

--region us-west-1 \

--timeout 30 \

--memory-size 1024 \

--environment Variables={COUCHBASE_HOST=ec2-52-53-193-176.us-west-1.compute.amazonaws.com} \

--publish

Couple of key items to note in this function are:

IAM role microserviceRole created in previous step is explicitly specified here
Handler is org.sample.serverless.aws.couchbase.BucketGetAll class. This class queries the Couchbase database defined using the COUCHBASE_HOST environment variable.

Create the second Lambda function:

aws lambda create-function \
--function-name MicroservicePost \
--role arn:aws:iam::598307997273:role/microserviceRole \
--handler org.sample.serverless.aws.couchbase.BucketPost \
--zip-file fileb:///Users/arungupta/workspaces/serverless/aws/microservice/microservice-http-endpoint/target/microservice-http-endpoint-1.0-SNAPSHOT.jar \
--description "Microservice HTTP Endpoint - Post" \
--runtime java8 \
--region us-west-1 \
--timeout 30 \
--memory-size 1024 \
--environment Variables={COUCHBASE_HOST=ec2-52-53-193-176.us-west-1.compute.amazonaws.com} \
--publish

aws lambda create-function \

--function-name MicroservicePost \

--role arn:aws:iam::598307997273:role/microserviceRole \

--handler org.sample.serverless.aws.couchbase.BucketPost \

--zip-file fileb:///Users/arungupta/workspaces/serverless/aws/microservice/microservice-http-endpoint/target/microservice-http-endpoint-1.0-SNAPSHOT.jar \

--description "Microservice HTTP Endpoint - Post" \

--runtime java8 \

--region us-west-1 \

--timeout 30 \

--memory-size 1024 \

--environment Variables={COUCHBASE_HOST=ec2-52-53-193-176.us-west-1.compute.amazonaws.com} \

--publish

The handler for this function is org.sample.serverless.aws.couchbase.BucketPost class. This class creates a new JSON document in the Couchbase database identified by COUCHBASE_HOST environment variable.

The complete source code for these classes is at github.com/arun-gupta/serverless/tree/master/aws/microservice/microservice-http-endpoint.

API Gateway Resource

Create an API using Amazon API Gateway and Test It and Build an API to Expose a Lambda Function provide detailed steps and explanation on how to use API Gateway and Lambda Functions to build powerful backend systems. This blog will do a quick run down of the steps in case you want to cut the chase.

Let’s create API Gateway resources.

The first step is to create an API:

aws apigateway \
create-rest-api \
--name Book

aws apigateway \

create-rest-api \

--name Book

This shows the output as:

{
    "name": "Book", 
    "id": "lb2qgujjif", 
    "createdDate": 1482998945
}

{

"name": "Book",

"id": "lb2qgujjif",

"createdDate": 1482998945

}

The value of id attribute is API ID. In our case, this is lb2qgujjif.

Find ROOT ID of the created API as this is required for the next AWS CLI invocation:

aws apigateway get-resources --rest-api-id lb2qgujjif

aws apigateway get-resources --rest-api-id lb2qgujjif

This shows the output:

{
    "items": [
        {
            "path": "/", 
            "id": "hgxogdkheg"
        }
    ]
}

{

"items": [

{

"path": "/",

"id": "hgxogdkheg"

}

]

}

Value of id attribute is ROOT ID. This is also the PARENT ID for the top level resource.

Create a resource

aws apigateway create-resource \
--rest-api-id lb2qgujjif \
--parent-id hgxogdkheg \
--path-part books

aws apigateway create-resource \

--rest-api-id lb2qgujjif \

--parent-id hgxogdkheg \

--path-part books

This shows the output:

{
    "path": "/books", 
    "pathPart": "books", 
    "id": "vrpkod", 
    "parentId": "hgxogdkheg"
}

{

"path": "/books",

"pathPart": "books",

"id": "vrpkod",

"parentId": "hgxogdkheg"

}

Value of id attribute is RESOURCE ID.

API ID and RESOURCE ID are used for subsequent AWS CLI invocations.

API Gateway POST Method

Now that the resource is created, let’s create HTTP POST method on this resource.

Create a POST method

aws apigateway put-method \
--rest-api-id lb2qgujjif \
--resource-id vrpkod \
--http-method POST \
--authorization-type NONE

aws apigateway put-method \

--rest-api-id lb2qgujjif \

--resource-id vrpkod \

--http-method POST \

--authorization-type NONE

to see the response:

{
    "apiKeyRequired": false, 
    "httpMethod": "POST", 
    "authorizationType": "NONE"
}

{

"apiKeyRequired": false,

"httpMethod": "POST",

"authorizationType": "NONE"

}

Set Lambda function as destination of the POST method:

aws apigateway put-integration \
--rest-api-id lb2qgujjif \
--resource-id vrpkod \
--http-method POST \
--type AWS \
--integration-http-method POST \
--uri arn:aws:apigateway:us-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-1:<act-id>:function:MicroservicePost/invocations

aws apigateway put-integration \

--rest-api-id lb2qgujjif \

--resource-id vrpkod \

--http-method POST \

--type AWS \

--integration-http-method POST \

--uri arn:aws:apigateway:us-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-1:<act-id>:function:MicroservicePost/invocations

Make sure to replace <act-id> with your AWS account id. API ID and RESOURCE ID from previous section are used here as well. --uri is used to specify the URI of integration input. The format of the URI is fixed. This CLI will show the result as:

{
    "httpMethod": "POST", 
    "passthroughBehavior": "WHEN_NO_MATCH", 
    "cacheKeyParameters": [], 
    "type": "AWS", 
    "uri": "arn:aws:apigateway:us-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-1:<act-id>:function:MicroservicePost/invocations", 
    "cacheNamespace": "vrpkod"
}

{

"httpMethod": "POST",

"passthroughBehavior": "WHEN_NO_MATCH",

"cacheKeyParameters": [],

"type": "AWS",

"uri": "arn:aws:apigateway:us-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-1:<act-id>:function:MicroservicePost/invocations",

"cacheNamespace": "vrpkod"

}

Set content-type of POST method response:

aws apigateway put-method-response \
--rest-api-id lb2qgujjif \
--resource-id vrpkod \
--http-method POST \
--status-code 200 \
--response-models "{\"application/json\": \"Empty\"}"

aws apigateway put-method-response \

--rest-api-id lb2qgujjif \

--resource-id vrpkod \

--http-method POST \

--status-code 200 \

--response-models "{\"application/json\": \"Empty\"}"

to see the response:

{
    "responseModels": {
        "application/json": "Empty"
    }, 
    "statusCode": "200"
}

{

"responseModels": {

"application/json": "Empty"

"statusCode": "200"

}

Set content-type of POST method integration response:

aws apigateway put-integration-response \
--rest-api-id lb2qgujjif \
--resource-id vrpkod \
--http-method POST \
--status-code 200 \
--response-templates "{\"application/json\": \"Empty\"}"

aws apigateway put-integration-response \

--rest-api-id lb2qgujjif \

--resource-id vrpkod \

--http-method POST \

--status-code 200 \

--response-templates "{\"application/json\": \"Empty\"}"

to see the response:

{
    "statusCode": "200", 
    "responseTemplates": {
        "application/json": "Empty"
    }
}

{

"statusCode": "200",

"responseTemplates": {

"application/json": "Empty"

}

Deploy the API

aws apigateway create-deployment \
--rest-api-id lb2qgujjif \
--stage-name test

aws apigateway create-deployment \

--rest-api-id lb2qgujjif \

--stage-name test

to see the response

{
    "id": "9wi991", 
    "createdDate": 1482999187
}

{

"id": "9wi991",

"createdDate": 1482999187

}

Grant permission to allow API Gateway to invoke Lambda Function:

aws lambda add-permission \
--function-name MicroservicePost \
--statement-id apigateway-test-post-1 \
--action lambda:InvokeFunction \
--principal apigateway.amazonaws.com \
--source-arn "arn:aws:execute-api:us-west-1:<act-id>:lb2qgujjif/*/POST/books"

aws lambda add-permission \

--function-name MicroservicePost \

--statement-id apigateway-test-post-1 \

--action lambda:InvokeFunction \

--principal apigateway.amazonaws.com \

--source-arn "arn:aws:execute-api:us-west-1:<act-id>:lb2qgujjif/*/POST/books"

Also, grant permission to the deployed API:

aws lambda add-permission \
--function-name MicroservicePost \
--statement-id apigateway-test-post-2 \
--action lambda:InvokeFunction \
--principal apigateway.amazonaws.com \
--source-arn "arn:aws:execute-api:us-west-1:<act-id>:lb2qgujjif/test/GET/books"

aws lambda add-permission \

--function-name MicroservicePost \

--statement-id apigateway-test-post-2 \

--action lambda:InvokeFunction \

--principal apigateway.amazonaws.com \

--source-arn "arn:aws:execute-api:us-west-1:<act-id>:lb2qgujjif/test/GET/books"

Test the API method:

aws apigateway test-invoke-method \
--rest-api-id lb2qgujjif \
--resource-id vrpkod \
--http-method POST \
--path-with-query-string "" \
--body "{\"id\": \"1\", \"bookname\": \"test book\", \"isbn\": \"123\", \"cost\": \"1.23\"}"

aws apigateway test-invoke-method \

--rest-api-id lb2qgujjif \

--resource-id vrpkod \

--http-method POST \

--path-with-query-string "" \

--body "{\"id\": \"1\", \"bookname\": \"test book\", \"isbn\": \"123\", \"cost\": \"1.23\"}"

to see the response:

{
    "status": 200, 
    "body": "Empty", 
    "log": "Execution log for request test-request\nThu Dec 29 08:16:05 UTC 2016 : Starting execution for request: test-invoke-request\nThu Dec 29 08:16:05 UTC 2016 : HTTP Method: POST, Resource Path: /books\nThu Dec 29 08:16:05 UTC 2016 : Method request path: {}\nThu Dec 29 08:16:05 UTC 2016 : Method request query string: {}\nThu Dec 29 08:16:05 UTC 2016 : Method request headers: {}\nThu Dec 29 08:16:05 UTC 2016 : Method request body before transformations: {\"id\": \"1\", \"bookname\": \"test book\", \"isbn\": \"123\", \"cost\": \"1.23\"}\nThu Dec 29 08:16:05 UTC 2016 : Endpoint request URI: https://lambda.us-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-1:598307997273:function:MicroservicePost/invocations\nThu Dec 29 08:16:05 UTC 2016 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=****************************************************************************************************************************************************************************************************************************************************************************************************************************************c8bb85, X-Amz-Date=20161229T081605Z, x-amzn-apigateway-api-id=lb2qgujjif, X-Amz-Source-Arn=arn:aws:execute-api:us-west-1:598307997273:lb2qgujjif/null/POST/books, Accept=application/json, User-Agent=AmazonAPIGateway_lb2qgujjif, Host=lambda.us-west-1.amazonaws.com, X-Amz-Content-Sha256=559d0296d96ec5647eef6381602fe5e7f55dd17065864fafb4f581d106aa92f4, X-Amzn-Trace-Id=Root=1-5864c645-8494974a41a3a16c8d2f9929, Content-Type=application/json}\nThu Dec 29 08:16:05 UTC 2016 : Endpoint request body after transformations: {\"id\": \"1\", \"bookname\": \"test book\", \"isbn\": \"123\", \"cost\": \"1.23\"}\nThu Dec 29 08:16:10 UTC 2016 : Endpoint response body before transformations: \"{\\\"cost\\\":\\\"1.23\\\",\\\"id\\\":\\\"1\\\",\\\"bookname\\\":\\\"test book\\\",\\\"isbn\\\":\\\"123\\\"}\"\nThu Dec 29 08:16:10 UTC 2016 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=0b25323b-cd9f-11e6-8bd4-292925ba63a9, Connection=keep-alive, Content-Length=78, Date=Thu, 29 Dec 2016 08:16:10 GMT, Content-Type=application/json}\nThu Dec 29 08:16:10 UTC 2016 : Method response body after transformations: Empty\nThu Dec 29 08:16:10 UTC 2016 : Method response headers: {X-Amzn-Trace-Id=Root=1-5864c645-8494974a41a3a16c8d2f9929, Content-Type=application/json}\nThu Dec 29 08:16:10 UTC 2016 : Successfully completed execution\nThu Dec 29 08:16:10 UTC 2016 : Method completed with status: 200\n", 
    "latency": 5091, 
    "headers": {
        "X-Amzn-Trace-Id": "Root=1-5864c645-8494974a41a3a16c8d2f9929", 
        "Content-Type": "application/json"
    }
}

{

"status": 200,

"body": "Empty",

"log": "Execution log for request test-request\nThu Dec 29 08:16:05 UTC 2016 : Starting execution for request: test-invoke-request\nThu Dec 29 08:16:05 UTC 2016 : HTTP Method: POST, Resource Path: /books\nThu Dec 29 08:16:05 UTC 2016 : Method request path: {}\nThu Dec 29 08:16:05 UTC 2016 : Method request query string: {}\nThu Dec 29 08:16:05 UTC 2016 : Method request headers: {}\nThu Dec 29 08:16:05 UTC 2016 : Method request body before transformations: {\"id\": \"1\", \"bookname\": \"test book\", \"isbn\": \"123\", \"cost\": \"1.23\"}\nThu Dec 29 08:16:05 UTC 2016 : Endpoint request URI: https://lambda.us-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-1:598307997273:function:MicroservicePost/invocations\nThu Dec 29 08:16:05 UTC 2016 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=****************************************************************************************************************************************************************************************************************************************************************************************************************************************c8bb85, X-Amz-Date=20161229T081605Z, x-amzn-apigateway-api-id=lb2qgujjif, X-Amz-Source-Arn=arn:aws:execute-api:us-west-1:598307997273:lb2qgujjif/null/POST/books, Accept=application/json, User-Agent=AmazonAPIGateway_lb2qgujjif, Host=lambda.us-west-1.amazonaws.com, X-Amz-Content-Sha256=559d0296d96ec5647eef6381602fe5e7f55dd17065864fafb4f581d106aa92f4, X-Amzn-Trace-Id=Root=1-5864c645-8494974a41a3a16c8d2f9929, Content-Type=application/json}\nThu Dec 29 08:16:05 UTC 2016 : Endpoint request body after transformations: {\"id\": \"1\", \"bookname\": \"test book\", \"isbn\": \"123\", \"cost\": \"1.23\"}\nThu Dec 29 08:16:10 UTC 2016 : Endpoint response body before transformations: \"{\\\"cost\\\":\\\"1.23\\\",\\\"id\\\":\\\"1\\\",\\\"bookname\\\":\\\"test book\\\",\\\"isbn\\\":\\\"123\\\"}\"\nThu Dec 29 08:16:10 UTC 2016 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=0b25323b-cd9f-11e6-8bd4-292925ba63a9, Connection=keep-alive, Content-Length=78, Date=Thu, 29 Dec 2016 08:16:10 GMT, Content-Type=application/json}\nThu Dec 29 08:16:10 UTC 2016 : Method response body after transformations: Empty\nThu Dec 29 08:16:10 UTC 2016 : Method response headers: {X-Amzn-Trace-Id=Root=1-5864c645-8494974a41a3a16c8d2f9929, Content-Type=application/json}\nThu Dec 29 08:16:10 UTC 2016 : Successfully completed execution\nThu Dec 29 08:16:10 UTC 2016 : Method completed with status: 200\n",

"latency": 5091,

"headers": {

"X-Amzn-Trace-Id": "Root=1-5864c645-8494974a41a3a16c8d2f9929",

"Content-Type": "application/json"

}

Value of status attribute is 200 and indicates this was a successful invocation. Value of log attribute shows the log statement from CloudWatch Logs. Detailed logs can also be obtained using aws logs filter-log-events --log-group /aws/lambda/MicroservicePost.

This command stores a single JSON document in Couchbase. This can be easily verified using the Couchbase CLI Tool cbq.Connect to the Couchbase server as:

cbq -u Administrator -p password -e="http://<COUCHBASE_HOST>:8091"

cbq -u Administrator -p password -e="http://<COUCHBASE_HOST>:8091"

Create a primary index on default bucket as this is required to query the bucket with no clauses:

cbq> create primary index default_index on default;
{
    "requestID": "13b539f9-7fff-4386-92f4-cea161a7aa08",
    "signature": null,
    "results": [
    ],
    "status": "success",
    "metrics": {
        "elapsedTime": "1.917009047s",
        "executionTime": "1.916970061s",
        "resultCount": 0,
        "resultSize": 0
    }
}

cbq> create primary index default_index on default;

{

"requestID": "13b539f9-7fff-4386-92f4-cea161a7aa08",

"signature": null,

"results": [

"status": "success",

"metrics": {

"elapsedTime": "1.917009047s",

"executionTime": "1.916970061s",

"resultCount": 0,

"resultSize": 0

}

Write a N1QL query to access the data:

cbq> select * from default limit 10;
{
    "requestID": "d7b1c3f9-6b4e-4952-9a1e-9faf5169926e",
    "signature": {
        "*": "*"
    },
    "results": [
        {
            "default": {
                "bookname": "test",
                "cost": "1.23",
                "id": "1",
                "isbn": "123"
            }
        }
    ],
    "status": "success",
    "metrics": {
        "elapsedTime": "24.337755ms",
        "executionTime": "24.289796ms",
        "resultCount": 1,
        "resultSize": 175
    }
}

cbq> select * from default limit 10;

{

"requestID": "d7b1c3f9-6b4e-4952-9a1e-9faf5169926e",

"signature": {

"*": "*"

"results": [

{

"default": {

"bookname": "test",

"cost": "1.23",

"id": "1",

"isbn": "123"

}

"status": "success",

"metrics": {

"elapsedTime": "24.337755ms",

"executionTime": "24.289796ms",

"resultCount": 1,

"resultSize": 175

}

The results show the JSON document that was stored by our Lambda function.

API Gateway GET Method

Let’s create HTTP GET method on the resource:

Create a GET method:

aws apigateway put-method \
--rest-api-id lb2qgujjif \
--resource-id vrpkod \
--http-method GET \
--authorization-type NONE

aws apigateway put-method \

--rest-api-id lb2qgujjif \

--resource-id vrpkod \

--http-method GET \

--authorization-type NONE

Set correct Lambda function as destination of GET:

aws apigateway put-integration \
--rest-api-id lb2qgujjif \
--resource-id vrpkod \
--http-method GET \
--type AWS \
--integration-http-method POST \
--uri arn:aws:apigateway:us-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-1:598307997273:function:MicroserviceGetAll/invocations

aws apigateway put-integration \

--rest-api-id lb2qgujjif \

--resource-id vrpkod \

--http-method GET \

--type AWS \

--integration-http-method POST \

--uri arn:aws:apigateway:us-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-1:598307997273:function:MicroserviceGetAll/invocations

Set content-type of GET method response:

aws apigateway put-method-response \
--rest-api-id lb2qgujjif \
--resource-id vrpkod \
--http-method GET \
--status-code 200 \
--response-models "{\"application/json\": \"Empty\"}"

aws apigateway put-method-response \

--rest-api-id lb2qgujjif \

--resource-id vrpkod \

--http-method GET \

--status-code 200 \

--response-models "{\"application/json\": \"Empty\"}"

Set content-type of GET method integration response:

aws apigateway put-integration-response \
--rest-api-id lb2qgujjif \
--resource-id vrpkod \
--http-method GET \
--status-code 200 \
--response-templates "{\"application/json\": \"Empty\"}"

aws apigateway put-integration-response \

--rest-api-id lb2qgujjif \

--resource-id vrpkod \

--http-method GET \

--status-code 200 \

--response-templates "{\"application/json\": \"Empty\"}"

Grant permission to allow API Gateway to invoke Lambda Function

aws lambda add-permission \
--function-name MicroserviceGetAll \
--statement-id apigateway-test-getall-1 \
--action lambda:InvokeFunction \
--principal apigateway.amazonaws.com \
--source-arn "arn:aws:execute-api:us-west-1:598307997273:lb2qgujjif/*/GET/books"

aws lambda add-permission \

--function-name MicroserviceGetAll \

--statement-id apigateway-test-getall-1 \

--action lambda:InvokeFunction \

--principal apigateway.amazonaws.com \

--source-arn "arn:aws:execute-api:us-west-1:598307997273:lb2qgujjif/*/GET/books"

Grant permission to the deployed API:

aws lambda add-permission \
--function-name MicroserviceGetAll \
--statement-id apigateway-test-getall-2 \
--action lambda:InvokeFunction \
--principal apigateway.amazonaws.com \
--source-arn "arn:aws:execute-api:us-west-1:598307997273:lb2qgujjif/test/GET/books"

aws lambda add-permission \

--function-name MicroserviceGetAll \

--statement-id apigateway-test-getall-2 \

--action lambda:InvokeFunction \

--principal apigateway.amazonaws.com \

--source-arn "arn:aws:execute-api:us-west-1:598307997273:lb2qgujjif/test/GET/books"

Test the method:

aws apigateway test-invoke-method \
--rest-api-id lb2qgujjif \
--resource-id vrpkod \
--http-method GET

aws apigateway test-invoke-method \

--rest-api-id lb2qgujjif \

--resource-id vrpkod \

--http-method GET

to see the output:

{
    "status": 200, 
    "body": "Empty", 
    "log": "Execution log for request test-request\nSat Dec 31 09:07:48 UTC 2016 : Starting execution for request: test-invoke-request\nSat Dec 31 09:07:48 UTC 2016 : HTTP Method: GET, Resource Path: /books\nSat Dec 31 09:07:48 UTC 2016 : Method request path: {}\nSat Dec 31 09:07:48 UTC 2016 : Method request query string: {}\nSat Dec 31 09:07:48 UTC 2016 : Method request headers: {}\nSat Dec 31 09:07:48 UTC 2016 : Method request body before transformations: \nSat Dec 31 09:07:48 UTC 2016 : Endpoint request URI: https://lambda.us-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-1:598307997273:function:MicroserviceGetAll/invocations\nSat Dec 31 09:07:48 UTC 2016 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=******************************************************************************************************************************************************************************************************************************************************************************************************6de147, X-Amz-Date=20161231T090748Z, x-amzn-apigateway-api-id=lb2qgujjif, X-Amz-Source-Arn=arn:aws:execute-api:us-west-1:598307997273:lb2qgujjif/null/GET/books, Accept=application/json, User-Agent=AmazonAPIGateway_lb2qgujjif, X-Amz-Security-Token=FQoDYXdzEHEaDEILpsKTo45Ys1LrFCK3A+KOe5HXOSP3GfVAaRYHe1pDUJGHL9MtkFiPjORLFT+UCKjRqE7UFaGscTVG6PZXTuSyQev4XTyROfPylCrtDomGsoZF/iwy4rlJQIJ7elBceyeKu1OVdaT1A99PVeliaCAiDL6Veo1viWOnP+7c72nAaJ5jnyF/nHl/OLhFdFv4t/hnx3JePMk5YM89/6ofxUEVDNfzXxbZHRpTrG/4TPHwjPdoR5i9dEzWMU6Eo5xD4ldQ/m5B3RmrwpaPOuEq39LhJ8k/Vzo+pAfgJTq5ssbNwYOgh0RPSGVNMcoTkCwk0EMMT5vDbmQqZ2dW1a1tmQg9N2xR+QQy+RKMFgO5YY8fMxHnRSdMuuipxl79G1pktc [TRUNCATED]\nSat Dec 31 09:07:48 UTC 2016 : Endpoint request body after transformations: \nSat Dec 31 09:07:53 UTC 2016 : Endpoint response body before transformations: \"[{\\\"default\\\":{\\\"cost\\\":\\\"1.23\\\",\\\"id\\\":\\\"1\\\",\\\"bookname\\\":\\\"test book\\\",\\\"isbn\\\":\\\"123\\\"}}]\"\nSat Dec 31 09:07:53 UTC 2016 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=99ab09b2-cf38-11e6-996f-f5f07af431af, Connection=keep-alive, Content-Length=94, Date=Sat, 31 Dec 2016 09:07:52 GMT, Content-Type=application/json}\nSat Dec 31 09:07:53 UTC 2016 : Method response body after transformations: Empty\nSat Dec 31 09:07:53 UTC 2016 : Method response headers: {X-Amzn-Trace-Id=Root=1-58677564-66f1e96642b16d2db703126e, Content-Type=application/json}\nSat Dec 31 09:07:53 UTC 2016 : Successfully completed execution\nSat Dec 31 09:07:53 UTC 2016 : Method completed with status: 200\n", 
    "latency": 4744, 
    "headers": {
        "X-Amzn-Trace-Id": "Root=1-58677564-66f1e96642b16d2db703126e", 
        "Content-Type": "application/json"
    }
}

{

"status": 200,

"body": "Empty",

"log": "Execution log for request test-request\nSat Dec 31 09:07:48 UTC 2016 : Starting execution for request: test-invoke-request\nSat Dec 31 09:07:48 UTC 2016 : HTTP Method: GET, Resource Path: /books\nSat Dec 31 09:07:48 UTC 2016 : Method request path: {}\nSat Dec 31 09:07:48 UTC 2016 : Method request query string: {}\nSat Dec 31 09:07:48 UTC 2016 : Method request headers: {}\nSat Dec 31 09:07:48 UTC 2016 : Method request body before transformations: \nSat Dec 31 09:07:48 UTC 2016 : Endpoint request URI: https://lambda.us-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-1:598307997273:function:MicroserviceGetAll/invocations\nSat Dec 31 09:07:48 UTC 2016 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=******************************************************************************************************************************************************************************************************************************************************************************************************6de147, X-Amz-Date=20161231T090748Z, x-amzn-apigateway-api-id=lb2qgujjif, X-Amz-Source-Arn=arn:aws:execute-api:us-west-1:598307997273:lb2qgujjif/null/GET/books, Accept=application/json, User-Agent=AmazonAPIGateway_lb2qgujjif, X-Amz-Security-Token=FQoDYXdzEHEaDEILpsKTo45Ys1LrFCK3A+KOe5HXOSP3GfVAaRYHe1pDUJGHL9MtkFiPjORLFT+UCKjRqE7UFaGscTVG6PZXTuSyQev4XTyROfPylCrtDomGsoZF/iwy4rlJQIJ7elBceyeKu1OVdaT1A99PVeliaCAiDL6Veo1viWOnP+7c72nAaJ5jnyF/nHl/OLhFdFv4t/hnx3JePMk5YM89/6ofxUEVDNfzXxbZHRpTrG/4TPHwjPdoR5i9dEzWMU6Eo5xD4ldQ/m5B3RmrwpaPOuEq39LhJ8k/Vzo+pAfgJTq5ssbNwYOgh0RPSGVNMcoTkCwk0EMMT5vDbmQqZ2dW1a1tmQg9N2xR+QQy+RKMFgO5YY8fMxHnRSdMuuipxl79G1pktc [TRUNCATED]\nSat Dec 31 09:07:48 UTC 2016 : Endpoint request body after transformations: \nSat Dec 31 09:07:53 UTC 2016 : Endpoint response body before transformations: \"[{\\\"default\\\":{\\\"cost\\\":\\\"1.23\\\",\\\"id\\\":\\\"1\\\",\\\"bookname\\\":\\\"test book\\\",\\\"isbn\\\":\\\"123\\\"}}]\"\nSat Dec 31 09:07:53 UTC 2016 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=99ab09b2-cf38-11e6-996f-f5f07af431af, Connection=keep-alive, Content-Length=94, Date=Sat, 31 Dec 2016 09:07:52 GMT, Content-Type=application/json}\nSat Dec 31 09:07:53 UTC 2016 : Method response body after transformations: Empty\nSat Dec 31 09:07:53 UTC 2016 : Method response headers: {X-Amzn-Trace-Id=Root=1-58677564-66f1e96642b16d2db703126e, Content-Type=application/json}\nSat Dec 31 09:07:53 UTC 2016 : Successfully completed execution\nSat Dec 31 09:07:53 UTC 2016 : Method completed with status: 200\n",

"latency": 4744,

"headers": {

"X-Amzn-Trace-Id": "Root=1-58677564-66f1e96642b16d2db703126e",

"Content-Type": "application/json"

}

Once again, 200 status code shows a successful invocation. Detailed logs can be obtained using aws logs filter-log-events --log-group /aws/lambda/MicroservicePost.

This blog only shows one simple POST and GET methods. Other HTTP methods can be very easily included in this microservice as well.

API Gateway and Lambda References

Serverless Architectures
AWS API Gateway
Creating a simple Microservice using Lambda and API Gateway
Couchbase Server Docs
Couchbase Forums
Follow us at @couchbasedev

Source: blog.couchbase.com/2016/december/microservice-aws-api-gateway-lambda-couchbase

AWS IoT Button, Lambda and Couchbase

December 25, 2016couchbase, techtipamazon, aws, couchbase, iotarungupta

Getting Started with Serverless FaaS and AWS Lambda shows how to use a simple Java function to store a JSON document to Couchbase using AWS Lambda. This blog builds upon that and shows how an AWS IoT Button can be used as a trigger for that Lambda function.

By end of this blog, you’ll learn:

How to configure AWS IoT Button
Use IoT Button as trigger for Lambda Function
Test IoT button

The overall flow will be:

serverless-iot-couchbase

Iot button click will invoke HelloCouchbaseLambda Lambda function. This function is uses Couchbase Java SDK to create a JSON document in Couchbase.

This blog is also playing catch up with Collecting iBeacon Data with Couchbase and Raspberry Pi IoT Devices by Nic and The CouchCase by Matthew on their summer projects. One last blog will be published in this series. That will show how multiple AWS IoT buttons can be used for some fun.

Let’s get started!

Configure IoT Button

The fastest way to configure IoT button is using the mobile app for iOS or Android.

More details about configuring IoT Button using mobile app.

Here are some snapshots from configuring button using the mobile app.

Bring up the app, click on + to start configuring a new button:

Enter button’s serial number:

Configure the button with wifi network:

Upload all the certificates etc:

After this, the button is configured and ready to use. This blog skipped the part where a template Lambda Function is associated with the button click.

If mobile app cannot be used then the button can be configured manually.

Use IoT Button as Trigger for Lambda Function

The aws lambda create-event-source-mapping CLI allows to create an event source for Lambda function. As of AWS CLI version 1.11.21, only Amazon Kinesis stream or an Amazon DynamoDB stream can be used. But for this blog, we’ll use IoT button as a trigger. And this has to be configured using AWS Lambda Console.

IoT Button is only supported in a limited number of regions. For example, it is not supported in the us-west-1 region but us-west-2 region works.

The list of regions not supported are greyed out in the following list:

Lambda Function can be triggered by several events. Lambda Function is invoked when any of these events occur. By default, no triggers are associated with a Lambda Function. For our HelloCouchbaseLambda function, these can be seen at us-west-2.console.aws.amazon.com/lambda/home?region=us-west-2#/functions/HelloCouchbaseLambda?tab=triggers.

Click on Add trigger to add a new trigger:

Select on the empty square to create a new trigger, and select AWS IoT:

For the button previously registered, get the serial number from us-west-2.console.aws.amazon.com/iotv2/home?region=us-west-2#/thinghub:

Specify the serial number of the button in the AWS IoT trigger:

Click on Submit to create the trigger:

And this confirms that the trigger has been added.

Test IoT Button

Before testing the button, let’s login to the Couchbase instance and verify the number of JSON documents in the bucket:

This can be verified at http://<EC2-IP-Address>:8091/index.html#sec=buckets. As expected, no documents exists in the bucket.

Press the button once, and refresh the page. It shows that one document is now stored in the bucket. This is verified in the Couchbase Web Console:

Click on Documents to see the complete list of documents:

Click on the document ID to see more details about the document:

Only timestamp is stored in this JSON document.

Now, let’s update HelloCouchbaseLambda code to include request id in the document as well. This can be achieved by adding the following line of code in the Java class:

buttonDocument.setRequestId(context.getAwsRequestId());

buttonDocument.setRequestId(context.getAwsRequestId());

A new deployment package can be built and uploaded using the following command:

mvn clean package; 
aws lambda update-function-code \
--function-name HelloCouchbaseLambda \
--zip-file fileb:///Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT.jar \
--region us-west-2 \
--publish

mvn clean package;

aws lambda update-function-code \

--function-name HelloCouchbaseLambda \

--zip-file fileb:///Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT.jar \

--region us-west-2 \

--publish

Now clicking the button will update the number of documents. But the updated document will have an additional attribute populated as shown:

How are you going to take AWS IoT button and use it with Lambda and Couchbase? Let us know at Couchbase Forums.

References

AWS IoT Button
AWS IoT Button Developer Guide
Couchbase Server Docs
Couchbase Forums
Follow us at @couchbasedev

Source: https://blog.couchbase.com/2016/december/aws-iot-button-lambda-couchbase

Serverless FaaS with AWS Lambda and Java

December 24, 2016couchbase, techtipamazon, lambda, serverlessarungupta

What is Serverless Architecture?

Serverless architecture runs custom code in ephemeral containers that are fully managed by a 3rd party. The custom code is typically a small part of a complete application. It is also called as function. This gives another name for serverless architecture as Function as a Service (FaaS). The container is ephemeral because it may only last for one invocation. The container may be reused but that’s not something you can rely upon. As a developer, you upload the code to FaaS platform, the service then handles all the capacity, scaling, patching and administration of the infrastructure to run your code.

An application built using Serverless Architecture follows the event-driven approach. For example, an activity happened in the application such as a click. This is

This is very different from a classical architecture where the application code is typically deployed in an application server such as Tomcat or WildFly. Scaling your application means starting additional instances of the application server or spinning up additional containers with the packaged application server. The Load Balancer need to be updated with the new IP addresses. Operating system need to be patched, upgraded and maintained.

Serverless Architectures explain the difference between the classical programming model and this new serverless architecture.

FaaS platform takes your application is divided into multiple functions. Each function is deployed in FaaS. The service spins up additional compute instances to meet the scalability demands of your application. FaaS platform provides the execution environment and takes care of starting and tearing down the containers to run your function.

Read Serverless Architectures for more details about these images.

One of the big advantages of FaaS is that you are only charged for the compute time, i.e. the time your code is running. There is no charge when your code is not running.

Another way to look at how Functions are different from VMs and Containers:

Note that Linux containers instead of Docker containers are used as an implementation for AWS Lambda.

How is FaaS different from PaaS?

As quoted at Serverless Architectures, a quick answer is provided by the following tweet:

If your PaaS can efficiently start instances in 20ms that run for half a second, then call it serverless. https://t.co/S3YzvqFYLR

— adrian cockcroft (@adrianco) May 28, 2016

In other words most PaaS applications are not geared towards bringing entire applications up and down for every request, whereas FaaS platforms do exactly this.

Abstracting the Back-end with FaaS explain the difference with different *aaS offerings. The image from the blog is captured below:

faas

Serverless Architectures also provide great details about what FaaS is and is not.

AWS Lambda, Google Cloud Functions and Azure Functions are some of the options for running serverless applications.

This blog will show how to write your first AWS Lambda function.

What is AWS Lambda?

AWS Lambda is FaaS service from Amazon Web Services. It runs your code on a high-availability compute infrastructure and performs all of the administration of the compute resources, including server and operating system maintenance, capacity provisioning and automatic scaling, code monitoring and logging.

AWS Lambda charges you for the duration your code runs in increments of 100ms. There is no cost associated with storing the Lambda function in AWS. First million requests per month are free and the pricing after that is nominal. Read more details on Lambda pricing. It also provides visibility into performance by providing real time metrics and logs to AWS CloudWatch. All you need to do is write the code!

Here is a quick introduction:

Also check out What’s New in AWS Lambda from AWS ReInvent 2016:

Also checkout Serverless Architectural Patterns and Best Practices from AWS ReInvent 2016:

The code you run on AWS Lambda is called a Lambda Function. You upload your code as a zip file or design it using the AWS Lambda Management Console. There is a built-in support for AWS SDK and this simplifies the ability to call other AWS services.

In short, Lambda is scalable, serverless, compute in the cloud.

AWS Lambda provides several execution environments:

Node.js – v0.10.36, v4.3.2 (recommended)
Java – Java 8
Python – Python 2.7
.NET Core – .NET Core 1.0.1 (C#)

This blog will show:

Build a Java application that stores a JSON document to Couchbase
Use Maven to create a deployment package for Java application
Create a Lambda Function
Update the Lambda Function

The complete code in this blog is available at github.com/arun-gupta/serverless/tree/master/aws/hellocouchbase.

Java Application for AWS Lambda

First, lets look at a Java application that will be used for this Lambda function. Programming Model for Lambda Functions in Java provide more details about how to write your Lambda function code in Java.

Our Lambda function will implemented the pre-defined interface com.amazonaws.services.lambda.runtime.RequestHandler. The code looks like:

public class HelloCouchbase implements RequestHandler<Request, String> {

    CouchbaseCluster cluster;
    Bucket bucket;
    LambdaLogger logger;

    @Override
    public String handleRequest(Request request, Context context) {
        SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");
        String timestamp = dateFormat.format(Calendar.getInstance().getTime());

        logger = context.getLogger();
        logger.log("Request received: %s" + timestamp);
        ButtonDocument buttonDocument = new ButtonDocument();
        buttonDocument.setId(context.getAwsRequestId());
        buttonDocument.setRequestId(context.getAwsRequestId());
        buttonDocument.setTimestamp(String.valueOf(timestamp));
        
        getBucket().upsert(buttonDocument.toJson());

        return buttonDocument.toString();
    }

public class HelloCouchbase implements RequestHandler<Request, String> {

CouchbaseCluster cluster;

Bucket bucket;

LambdaLogger logger;

@Override

public String handleRequest(Request request, Context context) {

SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");

String timestamp = dateFormat.format(Calendar.getInstance().getTime());

logger = context.getLogger();

logger.log("Request received: %s" + timestamp);

ButtonDocument buttonDocument = new ButtonDocument();

buttonDocument.setId(context.getAwsRequestId());

buttonDocument.setRequestId(context.getAwsRequestId());

buttonDocument.setTimestamp(String.valueOf(timestamp));

getBucket().upsert(buttonDocument.toJson());

return buttonDocument.toString();

}

handleRequest method is where the function code is implemented. Context provides useful information about Lambda execution environment. Some of the information from the context is stored a JSON document. Finally, Couchbase Java SDK API upsert is used to write a JSON document to the identified Couchbase instance. Couchbase on Amazon EC2 provide complete instructions to install Couchbase on AWS EC2.

Information about the Couchbase server is obtained as:

public CouchbaseCluster getCluster() {
    if (null == cluster) {
        logger.log("env: " + System.getenv("COUCHBASE_HOST"));
        cluster = CouchbaseCluster.create(System.getenv("COUCHBASE_HOST"));
    }
    return cluster;
}

public CouchbaseCluster getCluster() {

if (null == cluster) {

logger.log("env: " + System.getenv("COUCHBASE_HOST"));

cluster = CouchbaseCluster.create(System.getenv("COUCHBASE_HOST"));

}

return cluster;

}

This is once again using Couchbase Java API CouchbaseCluster as a main entry point to the Couchbase cluster. The COUCHBASE_HOST environment variable is passed when the Lambda function is created. In our case, this would point to a single node Couchbase cluster running on AWS EC2. Environment variables were recently introduced in AWS Lambda.

Finally, you need to access bucket in the server:

public Bucket getBucket() {
    while (null == bucket) {
        logger.log("Trying to connect to the database");
        bucket = getCluster().openBucket("serverless", 2L, TimeUnit.MINUTES);

        try {
            Thread.sleep(3000);
        } catch (Exception e) {
            logger.log("Thread sleep Exception: " + e.toString());
            throw new RuntimeException(e);
        }
    }

    return bucket;
}

public Bucket getBucket() {

while (null == bucket) {

logger.log("Trying to connect to the database");

bucket = getCluster().openBucket("serverless", 2L, TimeUnit.MINUTES);

try {

Thread.sleep(3000);

} catch (Exception e) {

logger.log("Thread sleep Exception: " + e.toString());

throw new RuntimeException(e);

}

return bucket;

}

The bucket name is serverless and all JSON documents are stored in this.

A simple Hello World application may be used for creating this function as well.

Create AWS Lambda Deployment Package

AWS Lambda function needs a deployment package. This package is either a .zip or .jar file that contains all the dependencies of the function. Our application is packaged using Maven, and so we’ll use a Maven plugin to create a deployment package.

The application has pom.xml with the following plugin fragment:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>2.3</version>
    <configuration>
        <createDependencyReducedPom>false</createDependencyReducedPom>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>shade</goal>
            </goals>
        </execution>
    </executions>
</plugin>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-shade-plugin</artifactId>

<createDependencyReducedPom>false</createDependencyReducedPom>

</configuration>

<phase>package</phase>

<goals>

<goal>shade</goal>

</goals>

</execution>

</executions>

</plugin>

More details about Maven configuration are available in Creating a .jar Deployment Package Using Maven without any IDE. The maven-shade-plugin allows to create an uber-jar including all the dependencies. The shade goal is tied to the package phase. So the mvn package command will generate a single deployment jar.

Package the application using mvn package command. This will show the output:

[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ hellocouchbase ---
[INFO] Building jar: /Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-shade-plugin:2.3:shade (default) @ hellocouchbase ---
[INFO] Including com.amazonaws:aws-lambda-java-core:jar:1.1.0 in the shaded jar.
[INFO] Including com.couchbase.client:java-client:jar:2.3.6 in the shaded jar.
[INFO] Including com.couchbase.client:core-io:jar:1.3.6 in the shaded jar.
[INFO] Including io.reactivex:rxjava:jar:1.1.8 in the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT.jar with /Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT-shaded.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ hellocouchbase ---

[INFO] Building jar: /Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT.jar

[INFO]

[INFO] --- maven-shade-plugin:2.3:shade (default) @ hellocouchbase ---

[INFO] Including com.amazonaws:aws-lambda-java-core:jar:1.1.0 in the shaded jar.

[INFO] Including com.couchbase.client:java-client:jar:2.3.6 in the shaded jar.

[INFO] Including com.couchbase.client:core-io:jar:1.3.6 in the shaded jar.

[INFO] Including io.reactivex:rxjava:jar:1.1.8 in the shaded jar.

[INFO] Replacing original artifact with shaded artifact.

[INFO] Replacing /Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT.jar with /Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT-shaded.jar

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

The target/hello-couchbase-1.0-SNAPSHOT.jar is the shaded jar that will be deployed to AWS Lambda.

More details about creating a deployment package are at Creating a Deployment Package.

Create AWS Lambda Function

Create AWS Lambda Function using AWS CLI. The CLI command in this case looks like:

aws lambda create-function \
--function-name HelloWorld \
--role arn:aws:iam::<account-id>:role/service-role/myLambdaRole \
--zip-file fileb:///Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT.jar \
--handler org.sample.serverless.aws.couchbase.HelloCouchbaseLambda \
--description "Hello Couchbase Lambda" \
--runtime java8  \
--region us-west-2 \
--timeout 30 \
--memory-size 1024 \
--publish

aws lambda create-function \

--function-name HelloWorld \

--role arn:aws:iam::<account-id>:role/service-role/myLambdaRole \

--zip-file fileb:///Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT.jar \

--handler org.sample.serverless.aws.couchbase.HelloCouchbaseLambda \

--description "Hello Couchbase Lambda" \

--runtime java8 \

--region us-west-2 \

--timeout 30 \

--memory-size 1024 \

--publish

In this CLI:

create-function creates a Lambda function
--function-name provides the function name. The function name is case sensitive.
--role specifies Amazon Resource Name (ARN) of an IAM role that Lambda assume when it executes your function to access any other AWS resources. If you’ve executed a Lambda function using AWS Console then this role is created for you.
--zip-file points to the deployment package that was created in previous step. fileb is an AWS CLI specific protocol to indicate that the content uploaded is binary.
--handler is the Java class that is called to begin execution of the function
--publish request AWS Lambda to create the Lambda function and publish a version as an atomic operation. Otherwise multiple versions may be created and may be published at a later point.

Lambda Console shows:

Test AWS Lambda Function

Test the AWS Lambda Function using AWS CLI.

aws lambda invoke \
--function-name HelloCouchbaseLambda \
--region us-west-2 \
--payload '' \
hellocouchbase.out

aws lambda invoke \

--function-name HelloCouchbaseLambda \

--region us-west-2 \

--payload '' \

hellocouchbase.out

It shows the output as:

{
    "StatusCode": 200
}

{

"StatusCode": 200

}

The output from the command is stored in hellocouchbase.out and looks like:

"{\"id\":\"e6bbe71a-ca4f-11e6-95a7-95f2ed410493\",\"installationId\":null,\"requestId\":\"e6bbe71a-ca4f-11e6-95a7-95f2ed410493\",\"identityId\":null,\"timestamp\":\"2016-12-25 03:12:01.157\"}"

"{\"id\":\"e6bbe71a-ca4f-11e6-95a7-95f2ed410493\",\"installationId\":null,\"requestId\":\"e6bbe71a-ca4f-11e6-95a7-95f2ed410493\",\"identityId\":null,\"timestamp\":\"2016-12-25 03:12:01.157\"}"

Invoking this function stores a JSON document in Couchbase. Documents stored in Couchbase can be seen using Couchbase Web Console. The password is Administrator and password is the EC2 instance id.

All data buckets in this Couchbase instance are shown below:

Note that the serverless bucket is manually created.

Clicking on Documents shows details of different documents stored in the bucket:

Clicking on each document shows more details about the JSON document:

Lambda function can also be tested using the Console:

Update AWS Lambda Function

If the application logic changes then a new deployment package needs to be uploaded for the Lambda function. In this case, mvn package will create a deployment package and aws lambda CLI command is used to update the function code:

aws lambda update-function-code \
--function-name HelloCouchbaseLambda \
--zip-file fileb:///Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT.jar \
--region us-west-2 \
--publish

aws lambda update-function-code \

--function-name HelloCouchbaseLambda \

--zip-file fileb:///Users/arungupta/workspaces/serverless/aws/hellocouchbase/hellocouchbase/target/hellocouchbase-1.0-SNAPSHOT.jar \

--region us-west-2 \

--publish

Shows the result:

{
    "CodeSha256": "w510ejw/OoVsQt2JiLG2bPZPAaFvQCRrYYYlQWctCQE=", 
    "FunctionName": "HelloCouchbaseLambda", 
    "CodeSize": 6978108, 
    "MemorySize": 1024, 
    "FunctionArn": "arn:aws:lambda:us-west-2:<account-id>:function:HelloCouchbaseLambda:8", 
    "Environment": {
        "Variables": {
            "COUCHBASE_HOST": "ec2-35-165-249-235.us-west-2.compute.amazonaws.com"
        }
    }, 
    "Version": "8", 
    "Role": "arn:aws:iam::<account-id>:role/service-role/myLambdaRole", 
    "Timeout": 30, 
    "LastModified": "2016-12-25T04:17:38.717+0000", 
    "Handler": "org.sample.serverless.aws.couchbase.HelloCouchbaseLambda", 
    "Runtime": "java8", 
    "Description": "Java Hello Couchbase"
}

{

"CodeSha256": "w510ejw/OoVsQt2JiLG2bPZPAaFvQCRrYYYlQWctCQE=",

"FunctionName": "HelloCouchbaseLambda",

"CodeSize": 6978108,

"MemorySize": 1024,

"FunctionArn": "arn:aws:lambda:us-west-2:<account-id>:function:HelloCouchbaseLambda:8",

"Environment": {

"Variables": {

"COUCHBASE_HOST": "ec2-35-165-249-235.us-west-2.compute.amazonaws.com"

}

"Version": "8",

"Role": "arn:aws:iam::<account-id>:role/service-role/myLambdaRole",

"Timeout": 30,

"LastModified": "2016-12-25T04:17:38.717+0000",

"Handler": "org.sample.serverless.aws.couchbase.HelloCouchbaseLambda",

"Runtime": "java8",

"Description": "Java Hello Couchbase"

}

The function can then be invoked again.

During writing of this blog, this was often used to debug the function as well. This is because Lambda functions do not have any state or box associated with them. And so you cannot log in to a box to check out if the function did not deploy correctly. You can certainly use CloudWatch log statements once the function is working.

AWS Lambda References

Serverless Architectures
AWS Lambda: How it works
Couchbase Server Docs
Couchbase Forums
Follow us at @couchbasedev

Source: https://blog.couchbase.com/2016/december/serverless-faas-aws-lambda-java

Kubernetes Monitoring with Heapster, InfluxDB and Grafana

December 2, 2016containers, couchbasecouchbase, kubernetes, loggingarungupta

Kubernetes provides detailed insights about resource usage in the cluster. This is enabled by using Heapster, cAdvisor, InfluxDB and Grafana.

Heapster is installed as a cluster-wide pod. It gathers monitoring and events data for all pods on each node by talking to the Kubelet. Kubelet itself fetches this data from cAdvisor. This data is persisted in InfluxDB and then visualized using Grafana.

Resource Usage Monitoring provide more details about monitoring resources in Kubernetes.

Heapster, InfluxDB and Grafana are Kubernetes addons. They are enabled by default if you are running the cluster on Amazon Web Services or Google Cloud. But need to be explicitly enabled if the cluster is started using minikube or kops addons.

Start a Kubernetes cluster on Amazon Web Services as:

KUBERNETES_PROVIDER=aws; kube-up.sh

More details about starting a Kubernetes cluster are available at Getting Started with Kubernetes 1.4.

By default, it creates a 4-node Kubernetes cluster in us-west-2a region. More details about the cluster can be seen using the command kubectl cluster-info and it shows the results as:

Kubernetes master is running at https://35.165.6.91
Elasticsearch is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging
Heapster is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/heapster
Kibana is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/kibana-logging
KubeDNS is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
Grafana is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana
InfluxDB is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Kubernetes master is running at https://35.165.6.91

Elasticsearch is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging

Heapster is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/heapster

Kibana is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/kibana-logging

KubeDNS is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/kube-dns

kubernetes-dashboard is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

Grafana is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana

InfluxDB is running at https://35.165.6.91/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Note the URL for the Grafana service. Open this URL in a browser window. You’ll be prompted for an invalid certificate warning but this can be safely ignored at this time. In production system, appropriate certificates should be installed.

Then you’ll be prompted for credentials. These can be obtained using kubectl config view command. It will show the output as:

- name: aws_kubernetes-basic-auth
  user:
    password: ZeH4JpQzAtGDEBdb
    username: admin

- name: aws_kubernetes-basic-auth

user:

password: ZeH4JpQzAtGDEBdb

username: admin

Use the value from username and password fields.

This shows the default dashboard:

kubernetes-grafana-empty-dashboard

It consists of two dashboards – one for cluster and another for pods.

For this blog, a 4-node Couchbase cluster was created following the steps outlined in Create a Couchbase Cluster using Kubernetes.

A cluster-wide dashboard shows CPU, Memory, Filesystem and Network usage across all the hosts and looks like:

CPU, memory, filesystem and network usage for all nodes may be seen:

kubernetes-grafana-cluster-per-node

Details for each node may be seen by selecting the node:

kubernetes-grafana-cluster-nodelist

CPU, memory, filesystem and network usage for each node is displayed:

kubernetes-grafana-cluster-one-node

Pods dashboard shows CPU, memory, filesystem and network usage for each pod:

A different pod may be chosen:

A complete list of all services running in the Kubernetes can be seen using kubectl get services --all-namespaces command. It shows the output as:

kubectl.sh get svc --all-namespaces
NAMESPACE     NAME                       CLUSTER-IP     EXTERNAL-IP        PORT(S)             AGE
default       couchbase-master-service   10.0.70.206    aef06961eb8f3...   8091/TCP            1h
default       kubernetes                 10.0.0.1       <none>             443/TCP             1h
kube-system   elasticsearch-logging      10.0.54.112    <none>             9200/TCP            1h
kube-system   heapster                   10.0.146.18    <none>             80/TCP              1h
kube-system   kibana-logging             10.0.123.37    <none>             5601/TCP            1h
kube-system   kube-dns                   10.0.0.10      <none>             53/UDP,53/TCP       1h
kube-system   kubernetes-dashboard       10.0.146.179   <none>             80/TCP              1h
kube-system   monitoring-grafana         10.0.33.81     <none>             80/TCP              1h
kube-system   monitoring-influxdb        10.0.26.251    <none>             8083/TCP,8086/TCP   1h

kubectl.sh get svc --all-namespaces

NAMESPACE NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE

default couchbase-master-service 10.0.70.206 aef06961eb8f3... 8091/TCP 1h

default kubernetes 10.0.0.1 <none> 443/TCP 1h

kube-system elasticsearch-logging 10.0.54.112 <none> 9200/TCP 1h

kube-system heapster 10.0.146.18 <none> 80/TCP 1h

kube-system kibana-logging 10.0.123.37 <none> 5601/TCP 1h

kube-system kube-dns 10.0.0.10 <none> 53/UDP,53/TCP 1h

kube-system kubernetes-dashboard 10.0.146.179 <none> 80/TCP 1h

kube-system monitoring-grafana 10.0.33.81 <none> 80/TCP 1h

kube-system monitoring-influxdb 10.0.26.251 <none> 8083/TCP,8086/TCP 1h

A complete list of all the pods running in the Kubernetes cluster can be seen using kubectl get pods --all-namespaces. It shows the output as:

kubectl.sh get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE default couchbase-master-rc-q9awd 1/1 Running 17 56m default couchbase-worker-rc-b1qkc 1/1 Running 15 54m default couchbase-worker-rc-j1c5z 1/1 Running 17 52m default couchbase-worker-rc-ju7z3 1/1 Running 15 52m kube-system elasticsearch-logging-v1-18ylh 1/1 Running 0 1h kube-system elasticsearch-logging-v1-fupap 1/1 Running 0 1h kube-system fluentd-elasticsearch-ip-172-20-0-94.us-west-2.compute.internal 1/1 Running 0 1h kube-system fluentd-elasticsearch-ip-172-20-0-95.us-west-2.compute.internal 1/1 Running 0 1h kube-system fluentd-elasticsearch-ip-172-20-0-96.us-west-2.compute.internal 1/1 Running 15 1h kube-system fluentd-elasticsearch-ip-172-20-0-97.us-west-2.compute.internal 1/1 Running 17 1h kube-system heapster-v1.2.0-1374379659-jms8e 4/4 Running 0 1h kube-system kibana-logging-v1-fcg4b 1/1 Running 3 1h kube-system kube-dns-v20-wpip4 3/3 Running 0 1h kube-system kube-proxy-ip-172-20-0-94.us-west-2.compute.internal 1/1 Running 0 1h kube-system kube-proxy-ip-172-20-0-95.us-west-2.compute.internal 1/1 Running 0 1h kube-system kube-proxy-ip-172-20-0-96.us-west-2.compute.internal 1/1 Running 15 1h kube-system kube-proxy-ip-172-20-0-97.us-west-2.compute.internal 1/1 Running 17 1h kube-system kubernetes-dashboard-v1.4.0-yxxgx 1/1 Running 0 1h kube-system monitoring-influxdb-grafana-v4-7asy4 2/2 Running 0 1h

Some references:

Kubernetes Resource Monitoring
Couchbase Cluster using Kubernetes, Docker Swarm, DC/OS and Amazon ECS
Follow us @couchbasedev

Source: blog.couchbase.com/2016/december/kubernetes-monitoring-heapster-influxdb-grafana

Couchbase Cluster on Mesos with DC/OS

November 23, 2016containers, couchbasearungupta

Apache Mesos is an open source cluster manager developed at UC Berkeley. It provides resource isolation and sharing across distributed applications. Mesos consists of a master daemon that manages slave daemons running on each cluster node.Mesos frameworks are applications that runs on Mesos and run tasks on these slaves. Marathon is a container orchestration platform running on Mesos.Multiple container formats are supported and Docker is certainly the most common one!

Docker Container using Apache Mesos and Marathon explains how to setup Mesos and Marathon. The setup is quite involving and a bit flaky. It required to download and Install Mesos Master and Slave, ZooKeeper, Docker Engine, and Marathon.

DC/OS is a distributed operating system using Mesos as its kernel. Couchbase on Mesos using DC/OS and Amazon explained how to run a single Couchbase container on DC/OS using CloudFormation templates.

Running a single node Couchbase may work during initial development phase. The need to start a multi-node Couchbase cluster becomes eminent as you move along further in development, and certainly needed during the production phase.

So, you’d like to run a Couchbase cluster on DC/OS?

Couchbase Cluster on DC/OS is complete walk through of how to setup a Couchbase cluster on DC/OS. It walks through the following steps:

What is Couchbase?
Couchbase Cluster
Setup DC/OS on Amazon Web Services
Configure CLI and Install Marathon Load Balancer
Create Couchbase “startup” service
Create Couchbase “node” service
Scale Couchbase Cluster
Rebalance Couchbase Cluster
Conclusion

DC/OS dashboard with a Couchbase cluster looks like:

For further information check out:

Couchbase on Containers
Couchbase Developer Portal
Ask questions on Couchbase Forums or Stack Overflow
Download Couchbase

You can also follow us at @couchbasedev and @couchbase.

Source: http://blog.couchbase.com/2016/november/couchbase-cluster-mesos-dcos

Health Check of Docker Containers

November 11, 2016containers, couchbasedockerarungupta

One of the new features in Docker 1.12 is how health check for a container can be baked into the image definition. And this can be overridden at the command line.

Just like the CMD instruction, there can be multiple HEALTHCHECK instructions in Dockerfile but only the last one is effective.

This is a great addition because a container reporting status as Up 1 hour may return errors. The container may be up but there is no way for the application inside the container to provide a status. This instruction fixes that.

The Dockerfile that builds arungupta/couchbase image is:

FROM couchbase:latest

COPY configure-node.sh /opt/couchbase

HEALTHCHECK --interval=5s --timeout=3s CMD curl --fail http://localhost:8091/pools || exit 1

CMD ["/opt/couchbase/configure-node.sh"]

FROM couchbase:latest

COPY configure-node.sh /opt/couchbase

HEALTHCHECK --interval=5s --timeout=3s CMD curl --fail http://localhost:8091/pools || exit 1

CMD ["/opt/couchbase/configure-node.sh"]

It uses configure-node.sh script to configure the server using Couchbase REST API. The new instruction to notice here is HEALTHCHECK.

This instruction can be specified as:

HEALTHCHECK <options> CMD <command>

HEALTHCHECK <options> CMD <command>

The <options> can be:

--interval=DURATION (default 30s)
--timeout=DURATION (default 30s)
--retries=N (default 3)

The <command> is the command that runs inside the container to check the health.

If health check is enabled, then the container can have three states:

starting – Initial status when the container is still starting
healthy – If the command succeeds then the container is healthy
unhealthy – If a single run of the <command> takes longer than the specified timeout then it is considered unhealthy. If a health check fails then the <command> will run retries number of times and will be declared unhealthy if the <command> still fails.

The commands exit status indicates the health status of the container. The following values are allowed:

0 – container is healthy
1 – container is not healthy

In our instruction, /pools REST API is invoked using curl. If the command fails then an exit status of 1 is returned, and this marks the container unhealthy for that attempt. This command is invoked every 5 seconds. The container is marked unhealthy if the command does not return successfully within 3 seconds.

Run the container as:

docker run -d --name db arungupta/couchbase:latest

docker run -d --name db arungupta/couchbase:latest

Check the status:

docker ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS                            PORTS                                                        NAMES
55b14302671e        arungupta/couchbase:latest   "/entrypoint.sh /opt/"   2 seconds ago       Up 1 seconds (health: starting)   8091-8094/tcp, 11207/tcp, 11210-11211/tcp, 18091-18093/tcp   db

docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

55b14302671e arungupta/couchbase:latest "/entrypoint.sh /opt/" 2 seconds ago Up 1 seconds (health: starting) 8091-8094/tcp, 11207/tcp, 11210-11211/tcp, 18091-18093/tcp db

Notice how health: starting status is reported in the STATUS column. Checking after a few seconds shows the status:

docker ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED              STATUS                        PORTS                                                        NAMES
55b14302671e        arungupta/couchbase:latest   "/entrypoint.sh /opt/"   About a minute ago   Up About a minute (healthy)   8091-8094/tcp, 11207/tcp, 11210-11211/tcp, 18091-18093/tcp   db

docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

55b14302671e arungupta/couchbase:latest "/entrypoint.sh /opt/" About a minute ago Up About a minute (healthy) 8091-8094/tcp, 11207/tcp, 11210-11211/tcp, 18091-18093/tcp db

And now its reported healthy.

More details about this HEALTHCHECK instruction can be found on docs.docker.com.

Now, if you are running an image that does not have HEALTHCHECK instruction then the docker run command can be used to specify similar values. An equivalent runtime command would be:

docker run -d --name db --health-cmd "curl --fail http://localhost:8091/pools || exit 1" --health-interval=5s --timeout=3s arungupta/couchbase

docker run -d --name db --health-cmd "curl --fail http://localhost:8091/pools || exit 1" --health-interval=5s --timeout=3s arungupta/couchbase

Last 5 health checks for a container can be obtained using the docker inspect command:

docker inspect --format='{{json .State.Health}}' db

docker inspect --format='{{json .State.Health}}' db

The output is shown as:

{
  "Status": "healthy",
  "FailingStreak": 0,
  "Log": [
    {
      "Start": "2016-11-12T03:23:03.351561Z",
      "End": "2016-11-12T03:23:03.422176171Z",
      "ExitCode": 0,
      "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100   768  100   768    0     0   595k      0 --:--:-- --:--:-- --:--:--  750k\n{\"isAdminCreds\":true,\"isROAdminCreds\":false,\"isEnterprise\":true,\"pools\":[{\"name\":\"default\",\"uri\":\"/pools/default?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"streamingUri\":\"/poolsStreaming/default?uuid=1b84cdbd136e4e8466049dd062dd6969\"}],\"settings\":{\"maxParallelIndexers\":\"/settings/maxParallelIndexers?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"viewUpdateDaemon\":\"/settings/viewUpdateDaemon?uuid=1b84cdbd136e4e8466049dd062dd6969\"},\"uuid\":\"1b84cdbd136e4e8466049dd062dd6969\",\"implementationVersion\":\"4.5.1-2844-enterprise\",\"componentsVersion\":{\"lhttpc\":\"1.3.0\",\"os_mon\":\"2.2.14\",\"public_key\":\"0.21\",\"asn1\":\"2.0.4\",\"kernel\":\"2.16.4\",\"ale\":\"4.5.1-2844-enterprise\",\"inets\":\"5.9.8\",\"ns_server\":\"4.5.1-2844-enterprise\",\"crypto\":\"3.2\",\"ssl\":\"5.3.3\",\"sasl\":\"2.3.4\",\"stdlib\":\"1.19.4\"}}"
    },
    {
      "Start": "2016-11-12T03:23:08.423558928Z",
      "End": "2016-11-12T03:23:08.510122392Z",
      "ExitCode": 0,
      "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100   768  100   768    0     0   309k      0 --:--:-- --:--:-- --:--:--  375k\n{\"isAdminCreds\":true,\"isROAdminCreds\":false,\"isEnterprise\":true,\"pools\":[{\"name\":\"default\",\"uri\":\"/pools/default?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"streamingUri\":\"/poolsStreaming/default?uuid=1b84cdbd136e4e8466049dd062dd6969\"}],\"settings\":{\"maxParallelIndexers\":\"/settings/maxParallelIndexers?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"viewUpdateDaemon\":\"/settings/viewUpdateDaemon?uuid=1b84cdbd136e4e8466049dd062dd6969\"},\"uuid\":\"1b84cdbd136e4e8466049dd062dd6969\",\"implementationVersion\":\"4.5.1-2844-enterprise\",\"componentsVersion\":{\"lhttpc\":\"1.3.0\",\"os_mon\":\"2.2.14\",\"public_key\":\"0.21\",\"asn1\":\"2.0.4\",\"kernel\":\"2.16.4\",\"ale\":\"4.5.1-2844-enterprise\",\"inets\":\"5.9.8\",\"ns_server\":\"4.5.1-2844-enterprise\",\"crypto\":\"3.2\",\"ssl\":\"5.3.3\",\"sasl\":\"2.3.4\",\"stdlib\":\"1.19.4\"}}"
    },
    {
      "Start": "2016-11-12T03:23:13.511446818Z",
      "End": "2016-11-12T03:23:13.58141325Z",
      "ExitCode": 0,
      "Output": " {\"isAdminCreds\":true,\"isROAdminCreds\":false,\"isEnterprise\":true,\"pools\":[{\"name\":\"default\",\"uri\":\"/pools/default?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"streamingUri\":\"/poolsStreaming/default?uuid=1b84cdbd136e4e8466049dd062dd6969\"}],\"settings\":{\"maxParallelIndexers\":\"/settings/maxParallelIndexers?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"viewUpdateDaemon\":\"/settings/viewUpdateDaemon?uuid=1b84cdbd136e4e8466049dd062dd6969\"},\"uuid\":\"1b84cdbd136e4e8466049dd062dd6969\",\"implementationVersion\":\"4.5.1-2844-enterprise\",\"componentsVersion\":{\"lhttpc\":\"1.3.0\",\"os_mon\":\"2.2.14\",\"public_key\":\"0.21\",\"asn1\":\"2.0.4\",\"kernel\":\"2.16.4\",\"ale\":\"4.5.1-2844-enterprise\",\"inets\":\"5.9.8\",\"ns_server\":\"4.5.1-2844-enterprise\",\"crypto\":\"3.2\",\"ssl\":\"5.3.3\",\"sasl\":\"2.3.4\",\"stdlib\":\"1.19.4\"}} % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100   768  100   768    0     0   248k      0 --:--:-- --:--:-- --:--:--  375k\n"
    },
    {
      "Start": "2016-11-12T03:23:18.583512367Z",
      "End": "2016-11-12T03:23:18.677727356Z",
      "ExitCode": 0,
      "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dlo{\"isAdminCreds\":true,\"isROAdminCreds\":false,\"isEnterprise\":true,\"pools\":[{\"name\":\"default\",\"uri\":\"/pools/default?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"streamingUri\":\"/poolsStreaming/default?uuid=1b84cdbd136e4e8466049dd062dd6969\"}],\"settings\":{\"maxParallelIndexers\":\"/settings/maxParallelIndexers?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"viewUpdateDaemon\":\"/settings/viewUpdateDaemon?uuid=1b84cdbd136e4e8466049dd062dd6969\"},\"uuid\":\"1b84cdbd136e4e8466049dd062dd6969\",\"implementationVersion\":\"4.5.1-2844-enterprise\",\"componentsVersion\":{\"lhttpc\":\"1.3.0\",\"os_mon\":\"2.2.14\",\"public_key\":\"0.21\",\"asn1\":\"2.0.4\",\"kernel\":\"2.16.4\",\"ale\":\"4.5.1-2844-enterprise\",\"inets\":\"5.9.8\",\"ns_server\":\"4.5.1-2844-enterprise\",\"crypto\":\"3.2\",\"ssl\":\"5.3.3\",\"sasl\":\"2.3.4\",\"stdlib\":\"1.19.4\"}}ad  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100   768  100   768    0     0   307k      0 --:--:-- --:--:-- --:--:--  375k\n"
    },
    {
      "Start": "2016-11-12T03:23:23.679661467Z",
      "End": "2016-11-12T03:23:23.782372291Z",
      "ExitCode": 0,
      "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left{\"isAdminCreds\":true,\"isROAdminCreds\":false,\"isEnterprise\":true,\"pools\":[{\"name\":\"default\",\"uri\":\"/pools/default?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"streamingUri\":\"/poolsStreaming/default?uuid=1b84cdbd136e4e8466049dd062dd6969\"}],\"settings\":{\"maxParallelIndexers\":\"/settings/maxParallelIndexers?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"viewUpdateDaemon\":\"/settings/viewUpdateDaemon?uuid=1b84cdbd136e4e8466049dd062dd6969\"},\"uuid\":\"1b84cdbd136e4e8466049dd062dd6969\",\"implementationVersion\":\"4.5.1-2844-enterprise\",\"componentsVersion\":{\"lhttpc\":\"1.3.0\",\"os_mon\":\"2.2.14\",\"public_key\":\"0.21\",\"asn1\":\"2.0.4\",\"kernel\":\"2.16.4\",\"ale\":\"4.5.1-2844-enterprise\",\"inets\":\"5.9.8\",\"ns_server\":\"4.5.1-2844-enterprise\",\"crypto\":\"3.2\",\"ssl\":\"5.3.3\",\"sasl\":\"2.3.4\",\"stdlib\":\"1.19.4\"}}  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100   768  100   768    0     0   439k      0 --:--:-- --:--:-- --:--:--  750k\n"
    }
  ]
}

{

"Status": "healthy",

"FailingStreak": 0,

"Log": [

{

"Start": "2016-11-12T03:23:03.351561Z",

"End": "2016-11-12T03:23:03.422176171Z",

"ExitCode": 0,

"Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 768 100 768 0 0 595k 0 --:--:-- --:--:-- --:--:-- 750k\n{\"isAdminCreds\":true,\"isROAdminCreds\":false,\"isEnterprise\":true,\"pools\":[{\"name\":\"default\",\"uri\":\"/pools/default?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"streamingUri\":\"/poolsStreaming/default?uuid=1b84cdbd136e4e8466049dd062dd6969\"}],\"settings\":{\"maxParallelIndexers\":\"/settings/maxParallelIndexers?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"viewUpdateDaemon\":\"/settings/viewUpdateDaemon?uuid=1b84cdbd136e4e8466049dd062dd6969\"},\"uuid\":\"1b84cdbd136e4e8466049dd062dd6969\",\"implementationVersion\":\"4.5.1-2844-enterprise\",\"componentsVersion\":{\"lhttpc\":\"1.3.0\",\"os_mon\":\"2.2.14\",\"public_key\":\"0.21\",\"asn1\":\"2.0.4\",\"kernel\":\"2.16.4\",\"ale\":\"4.5.1-2844-enterprise\",\"inets\":\"5.9.8\",\"ns_server\":\"4.5.1-2844-enterprise\",\"crypto\":\"3.2\",\"ssl\":\"5.3.3\",\"sasl\":\"2.3.4\",\"stdlib\":\"1.19.4\"}}"

{

"Start": "2016-11-12T03:23:08.423558928Z",

"End": "2016-11-12T03:23:08.510122392Z",

"ExitCode": 0,

"Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 768 100 768 0 0 309k 0 --:--:-- --:--:-- --:--:-- 375k\n{\"isAdminCreds\":true,\"isROAdminCreds\":false,\"isEnterprise\":true,\"pools\":[{\"name\":\"default\",\"uri\":\"/pools/default?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"streamingUri\":\"/poolsStreaming/default?uuid=1b84cdbd136e4e8466049dd062dd6969\"}],\"settings\":{\"maxParallelIndexers\":\"/settings/maxParallelIndexers?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"viewUpdateDaemon\":\"/settings/viewUpdateDaemon?uuid=1b84cdbd136e4e8466049dd062dd6969\"},\"uuid\":\"1b84cdbd136e4e8466049dd062dd6969\",\"implementationVersion\":\"4.5.1-2844-enterprise\",\"componentsVersion\":{\"lhttpc\":\"1.3.0\",\"os_mon\":\"2.2.14\",\"public_key\":\"0.21\",\"asn1\":\"2.0.4\",\"kernel\":\"2.16.4\",\"ale\":\"4.5.1-2844-enterprise\",\"inets\":\"5.9.8\",\"ns_server\":\"4.5.1-2844-enterprise\",\"crypto\":\"3.2\",\"ssl\":\"5.3.3\",\"sasl\":\"2.3.4\",\"stdlib\":\"1.19.4\"}}"

{

"Start": "2016-11-12T03:23:13.511446818Z",

"End": "2016-11-12T03:23:13.58141325Z",

"ExitCode": 0,

"Output": " {\"isAdminCreds\":true,\"isROAdminCreds\":false,\"isEnterprise\":true,\"pools\":[{\"name\":\"default\",\"uri\":\"/pools/default?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"streamingUri\":\"/poolsStreaming/default?uuid=1b84cdbd136e4e8466049dd062dd6969\"}],\"settings\":{\"maxParallelIndexers\":\"/settings/maxParallelIndexers?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"viewUpdateDaemon\":\"/settings/viewUpdateDaemon?uuid=1b84cdbd136e4e8466049dd062dd6969\"},\"uuid\":\"1b84cdbd136e4e8466049dd062dd6969\",\"implementationVersion\":\"4.5.1-2844-enterprise\",\"componentsVersion\":{\"lhttpc\":\"1.3.0\",\"os_mon\":\"2.2.14\",\"public_key\":\"0.21\",\"asn1\":\"2.0.4\",\"kernel\":\"2.16.4\",\"ale\":\"4.5.1-2844-enterprise\",\"inets\":\"5.9.8\",\"ns_server\":\"4.5.1-2844-enterprise\",\"crypto\":\"3.2\",\"ssl\":\"5.3.3\",\"sasl\":\"2.3.4\",\"stdlib\":\"1.19.4\"}} % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 768 100 768 0 0 248k 0 --:--:-- --:--:-- --:--:-- 375k\n"

{

"Start": "2016-11-12T03:23:18.583512367Z",

"End": "2016-11-12T03:23:18.677727356Z",

"ExitCode": 0,

"Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dlo{\"isAdminCreds\":true,\"isROAdminCreds\":false,\"isEnterprise\":true,\"pools\":[{\"name\":\"default\",\"uri\":\"/pools/default?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"streamingUri\":\"/poolsStreaming/default?uuid=1b84cdbd136e4e8466049dd062dd6969\"}],\"settings\":{\"maxParallelIndexers\":\"/settings/maxParallelIndexers?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"viewUpdateDaemon\":\"/settings/viewUpdateDaemon?uuid=1b84cdbd136e4e8466049dd062dd6969\"},\"uuid\":\"1b84cdbd136e4e8466049dd062dd6969\",\"implementationVersion\":\"4.5.1-2844-enterprise\",\"componentsVersion\":{\"lhttpc\":\"1.3.0\",\"os_mon\":\"2.2.14\",\"public_key\":\"0.21\",\"asn1\":\"2.0.4\",\"kernel\":\"2.16.4\",\"ale\":\"4.5.1-2844-enterprise\",\"inets\":\"5.9.8\",\"ns_server\":\"4.5.1-2844-enterprise\",\"crypto\":\"3.2\",\"ssl\":\"5.3.3\",\"sasl\":\"2.3.4\",\"stdlib\":\"1.19.4\"}}ad Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 768 100 768 0 0 307k 0 --:--:-- --:--:-- --:--:-- 375k\n"

{

"Start": "2016-11-12T03:23:23.679661467Z",

"End": "2016-11-12T03:23:23.782372291Z",

"ExitCode": 0,

"Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left{\"isAdminCreds\":true,\"isROAdminCreds\":false,\"isEnterprise\":true,\"pools\":[{\"name\":\"default\",\"uri\":\"/pools/default?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"streamingUri\":\"/poolsStreaming/default?uuid=1b84cdbd136e4e8466049dd062dd6969\"}],\"settings\":{\"maxParallelIndexers\":\"/settings/maxParallelIndexers?uuid=1b84cdbd136e4e8466049dd062dd6969\",\"viewUpdateDaemon\":\"/settings/viewUpdateDaemon?uuid=1b84cdbd136e4e8466049dd062dd6969\"},\"uuid\":\"1b84cdbd136e4e8466049dd062dd6969\",\"implementationVersion\":\"4.5.1-2844-enterprise\",\"componentsVersion\":{\"lhttpc\":\"1.3.0\",\"os_mon\":\"2.2.14\",\"public_key\":\"0.21\",\"asn1\":\"2.0.4\",\"kernel\":\"2.16.4\",\"ale\":\"4.5.1-2844-enterprise\",\"inets\":\"5.9.8\",\"ns_server\":\"4.5.1-2844-enterprise\",\"crypto\":\"3.2\",\"ssl\":\"5.3.3\",\"sasl\":\"2.3.4\",\"stdlib\":\"1.19.4\"}} Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 768 100 768 0 0 439k 0 --:--:-- --:--:-- --:--:-- 750k\n"

}

]

}

Source: http://blog.couchbase.com/2016/november/docker-health-check-keeping-containers-healthy

Docker for AWS – Getting Started Video

November 3, 2016containers, couchbaseaws, couchbase, dockerarungupta

Want to create a highly-available Docker cluster on Amazon Web Services? Run multi-container applications on it using Docker Services?

Docker for AWS allows you to exactly do that! This video shows:

Create a highly-available Docker cluster on Amazon Web Services (0:00)
Check configuration (5:43)
Use Docker services to create a Couchbase cluster (8:23)

Enjoy!

couchbase.com/containers provide more details about how to run Couchbase in different container frameworks. More information about Couchbase:

Couchbase Developer Portal
Couchbase Forums
@couchbasedev or @couchbase

Source: blog.couchbase.com/2016/november/docker-for-aws-getting-started-video

Multimaster Kubernetes Cluster on Amazon Using Kops

November 2, 2016containers, couchbasearungupta

Getting Started with Kubernetes 1.4 using Spring Boot and Couchbase explains how to get started with Kubernetes 1.4 on Amazon Web Services. A Couchbase service is created in the cluster and a Spring Boot application stores a JSON document in the database. It uses kube-up.sh script from the Kubernetes binary download at github.com/kubernetes/kubernetes/releases/download/v1.4.0/kubernetes.tar.gz to start the cluster. This script is capable of creating a Kubernetes cluster with single master only. This is a fundamental flaw of distributed applications where the master becomes a Single Point of Failure.

Meet kops – short for Kubernetes Operations.

This is the easiest way to get a highly-available Kubernetes cluster up and running. The kubectl script is the CLI for running commands against running clusters. Think of kops as kubectl for cluster.

This blog will show how to create a highly-available Kubernetes cluster on Amazon using kops. And once the cluster is created, then it’ll create a Couchbase service on it and run a Spring Boot application to store JSON document in the database.

Many thanks to justinsb, sarahz, razic, jaygorrell, shrugs, bkpandey and others at Kubernetes slack channel for helping me through the details!

Download kops and kubectl

Download Kops latest release. This blog was tested with 1.4.1 on OSX.Complete set of commands for kops can be seen:

kops-darwin-amd64 --help
kops is kubernetes ops.
It allows you to create, destroy, upgrade and maintain clusters.

Usage:
  kops [command]

Available Commands:
  create         create resources
  delete         delete clusters
  describe       describe objects
  edit           edit items
  export         export clusters/kubecfg
  get            list or get objects
  import         import clusters
  rolling-update rolling update clusters
  secrets        Manage secrets & keys
  toolbox        Misc infrequently used commands
  update         update clusters
  upgrade        upgrade clusters
  version        Print the client version information

Flags:
      --alsologtostderr                  log to standard error as well as files
      --config string                    config file (default is $HOME/.kops.yaml)
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files (default false)
      --name string                      Name of cluster
      --state string                     Location of state storage
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          log level for V logs
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

Use "kops [command] --help" for more information about a command.

kops-darwin-amd64 --help

kops is kubernetes ops.

It allows you to create, destroy, upgrade and maintain clusters.

Usage:

kops [command]

Available Commands:

create create resources

delete delete clusters

describe describe objects

edit edit items

export export clusters/kubecfg

get list or get objects

import import clusters

rolling-update rolling update clusters

secrets Manage secrets & keys

toolbox Misc infrequently used commands

update update clusters

upgrade upgrade clusters

version Print the client version information

Flags:

--alsologtostderr log to standard error as well as files

--config string config file (default is $HOME/.kops.yaml)

--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)

--log_dir string If non-empty, write log files in this directory

--logtostderr log to standard error instead of files (default false)

--name string Name of cluster

--state string Location of state storage

--stderrthreshold severity logs at or above this threshold go to stderr (default 2)

-v, --v Level log level for V logs

--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging

Use "kops [command] --help" for more information about a command.

Download kubectl:

curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/v1.4.1/bin/darwin/amd64/kubectl && chmod +x kubectl

curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/v1.4.1/bin/darwin/amd64/kubectl && chmod +x kubectl

Include kubectl in your PATH.

Create Bucket and NS Records on Amazon

There is a bit of setup involved at this time, and hopefully this will get cleaned up over next releases. Bringing up a cluster on AWS provide detailed steps and more background. Here is what the blog followed:

Pick a domain where Kubernetes cluster will be hosted. This blog uses kubernetes.arungupta.me domain. You can pick a top level domain or a sub-domain.
Amazon Route 53 is a highly available and scalable DNS service. Login to Amazon Console and created a hosted zone for this domain using Route 53 service.Created zone looks like:The values shown in the Value column are important as they’ll be used later for creating NS records.
Create a S3 bucket using Amazon Console to store cluster configuration – this is called state store.
The domain kubernetes.arungupta.me is hosted on GoDaddy. For each value shown in the Value column of Route53 hosted zone, create a NS record using GoDaddy Domain Control Center for this domain.Select the type of record:
For each value, add the record as shown:Completed set of records look like:

Start Kubernetes Multimaster Cluster

Let’s understand a bit about Amazon regions and zones:

Amazon EC2 is hosted in multiple locations world-wide. These locations are composed of regions and Availability Zones. Each region is a separate geographic area. Each region has multiple, isolated locations known as Availability Zones.

Amazon Docs

A highly-available Kubernetes cluster can be created across zones but not across regions.

Find out availability zones within a region:

aws ec2 describe-availability-zones --region us-west-2
{
    "AvailabilityZones": [
        {
            "State": "available", 
            "RegionName": "us-west-2", 
            "Messages": [], 
            "ZoneName": "us-west-2a"
        }, 
        {
            "State": "available", 
            "RegionName": "us-west-2", 
            "Messages": [], 
            "ZoneName": "us-west-2b"
        }, 
        {
            "State": "available", 
            "RegionName": "us-west-2", 
            "Messages": [], 
            "ZoneName": "us-west-2c"
        }
    ]
}

aws ec2 describe-availability-zones --region us-west-2

{

"AvailabilityZones": [

{

"State": "available",

"RegionName": "us-west-2",

"Messages": [],

"ZoneName": "us-west-2a"

{

"State": "available",

"RegionName": "us-west-2",

"Messages": [],

"ZoneName": "us-west-2b"

{

"State": "available",

"RegionName": "us-west-2",

"Messages": [],

"ZoneName": "us-west-2c"

}

]

}

Create a multi-master cluster:

kops-darwin-amd64 create cluster --name=kubernetes.arungupta.me --cloud=aws --zones=us-west-2a,us-west-2b,us-west-2c --master-size=m4.large --node-count=3 --node-size=m4.2xlarge --master-zones=us-west-2a,us-west-2b,us-west-2c --state=s3://kops-couchbase --yes

kops-darwin-amd64 create cluster --name=kubernetes.arungupta.me --cloud=aws --zones=us-west-2a,us-west-2b,us-west-2c --master-size=m4.large --node-count=3 --node-size=m4.2xlarge --master-zones=us-west-2a,us-west-2b,us-west-2c --state=s3://kops-couchbase --yes

Most of the switches are self-explanatory. Some switches need a bit of explanation:

Specifying multiple zones using --master-zones (must be odd number) create multiple masters across AZ
--cloud=aws is optional if cloud can be inferred from zones
--yes is used to specify the immediate creation of cluster. Otherwise only the state is stored in the bucket, and the cluster needs to be created separately.

Complete set of CLI switches can be seen:

./kops-darwin-amd64 create cluster --help
Creates a k8s cluster.

Usage:
  kops create cluster [flags]

Flags:
      --admin-access string         Restrict access to admin endpoints (SSH, HTTPS) to this CIDR.  If not set, access will not be restricted by IP.
      --associate-public-ip         Specify --associate-public-ip=[true|false] to enable/disable association of public IP for master ASG and nodes. Default is 'true'. (default true)
      --channel string              Channel for default versions and configuration to use (default "stable")
      --cloud string                Cloud provider to use - gce, aws
      --dns-zone string             DNS hosted zone to use (defaults to last two components of cluster name)
      --image string                Image to use
      --kubernetes-version string   Version of kubernetes to run (defaults to version in channel)
      --master-size string          Set instance size for masters
      --master-zones string         Zones in which to run masters (must be an odd number)
      --model string                Models to apply (separate multiple models with commas) (default "config,proto,cloudup")
      --network-cidr string         Set to override the default network CIDR
      --networking string           Networking mode to use.  kubenet (default), classic, external. (default "kubenet")
      --node-count int              Set the number of nodes
      --node-size string            Set instance size for nodes
      --out string                  Path to write any local output
      --project string              Project to use (must be set on GCE)
      --ssh-public-key string       SSH public key to use (default "~/.ssh/id_rsa.pub")
      --target string               Target - direct, terraform (default "direct")
      --vpc string                  Set to use a shared VPC
      --yes                         Specify --yes to immediately create the cluster
      --zones string                Zones in which to run the cluster

Global Flags:
      --alsologtostderr                  log to standard error as well as files
      --config string                    config file (default is $HOME/.kops.yaml)
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files (default false)
      --name string                      Name of cluster
      --state string                     Location of state storage
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          log level for V logs
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

./kops-darwin-amd64 create cluster --help

Creates a k8s cluster.

Usage:

kops create cluster [flags]

Flags:

--admin-access string Restrict access to admin endpoints (SSH, HTTPS) to this CIDR. If not set, access will not be restricted by IP.

--associate-public-ip Specify --associate-public-ip=[true|false] to enable/disable association of public IP for master ASG and nodes. Default is 'true'. (default true)

--channel string Channel for default versions and configuration to use (default "stable")

--cloud string Cloud provider to use - gce, aws

--dns-zone string DNS hosted zone to use (defaults to last two components of cluster name)

--image string Image to use

--kubernetes-version string Version of kubernetes to run (defaults to version in channel)

--master-size string Set instance size for masters

--master-zones string Zones in which to run masters (must be an odd number)

--model string Models to apply (separate multiple models with commas) (default "config,proto,cloudup")

--network-cidr string Set to override the default network CIDR

--networking string Networking mode to use. kubenet (default), classic, external. (default "kubenet")

--node-count int Set the number of nodes

--node-size string Set instance size for nodes

--out string Path to write any local output

--project string Project to use (must be set on GCE)

--ssh-public-key string SSH public key to use (default "~/.ssh/id_rsa.pub")

--target string Target - direct, terraform (default "direct")

--vpc string Set to use a shared VPC

--yes Specify --yes to immediately create the cluster

--zones string Zones in which to run the cluster

Global Flags:

--alsologtostderr log to standard error as well as files

--config string config file (default is $HOME/.kops.yaml)

--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)

--log_dir string If non-empty, write log files in this directory

--logtostderr log to standard error instead of files (default false)

--name string Name of cluster

--state string Location of state storage

--stderrthreshold severity logs at or above this threshold go to stderr (default 2)

-v, --v Level log level for V logs

--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging

Once the cluster is created, get more details about the cluster:

kubectl cluster-info
Kubernetes master is running at https://api.kubernetes.arungupta.me
KubeDNS is running at https://api.kubernetes.arungupta.me/api/v1/proxy/namespaces/kube-system/services/kube-dns

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

kubectl cluster-info

Kubernetes master is running at https://api.kubernetes.arungupta.me

KubeDNS is running at https://api.kubernetes.arungupta.me/api/v1/proxy/namespaces/kube-system/services/kube-dns

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Check cluster client and server version:

kubectl version
Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.1", GitCommit:"33cf7b9acbb2cb7c9c72a10d6636321fb180b159", GitTreeState:"clean", BuildDate:"2016-10-10T18:19:49Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.3", GitCommit:"4957b090e9a4f6a68b4a40375408fdc74a212260", GitTreeState:"clean", BuildDate:"2016-10-16T06:20:04Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}

kubectl version

Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.1", GitCommit:"33cf7b9acbb2cb7c9c72a10d6636321fb180b159", GitTreeState:"clean", BuildDate:"2016-10-10T18:19:49Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"darwin/amd64"}

Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.3", GitCommit:"4957b090e9a4f6a68b4a40375408fdc74a212260", GitTreeState:"clean", BuildDate:"2016-10-16T06:20:04Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}

Check all nodes in the cluster:

kubectl get nodes 
NAME                                           STATUS    AGE
ip-172-20-111-151.us-west-2.compute.internal   Ready     1h
ip-172-20-116-40.us-west-2.compute.internal    Ready     1h
ip-172-20-48-41.us-west-2.compute.internal     Ready     1h
ip-172-20-49-105.us-west-2.compute.internal    Ready     1h
ip-172-20-80-233.us-west-2.compute.internal    Ready     1h
ip-172-20-82-93.us-west-2.compute.internal     Ready     1h

kubectl get nodes

NAME STATUS AGE

ip-172-20-111-151.us-west-2.compute.internal Ready 1h

ip-172-20-116-40.us-west-2.compute.internal Ready 1h

ip-172-20-48-41.us-west-2.compute.internal Ready 1h

ip-172-20-49-105.us-west-2.compute.internal Ready 1h

ip-172-20-80-233.us-west-2.compute.internal Ready 1h

ip-172-20-82-93.us-west-2.compute.internal Ready 1h

Or find out only the master nodes:

kubectl get nodes -l kubernetes.io/role=master
NAME                                           STATUS    AGE
ip-172-20-111-151.us-west-2.compute.internal   Ready     1h
ip-172-20-48-41.us-west-2.compute.internal     Ready     1h
ip-172-20-82-93.us-west-2.compute.internal     Ready     1h

kubectl get nodes -l kubernetes.io/role=master

NAME STATUS AGE

ip-172-20-111-151.us-west-2.compute.internal Ready 1h

ip-172-20-48-41.us-west-2.compute.internal Ready 1h

ip-172-20-82-93.us-west-2.compute.internal Ready 1h

Check all the clusters:

kops-darwin-amd64 get clusters --state=s3://kops-couchbase
NAME                    CLOUD   ZONES
kubernetes.arungupta.me aws     us-west-2a,us-west-2b,us-west-2c

kops-darwin-amd64 get clusters --state=s3://kops-couchbase

NAME CLOUD ZONES

kubernetes.arungupta.me aws us-west-2a,us-west-2b,us-west-2c

Kubernetes Dashboard Addon

By default, a cluster created using kops does not have the UI dashboard. But this can be added as an add on:

kubectl create -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/kubernetes-dashboard/v1.4.0.yaml
deployment "kubernetes-dashboard-v1.4.0" created
service "kubernetes-dashboard" created

kubectl create -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/kubernetes-dashboard/v1.4.0.yaml

deployment "kubernetes-dashboard-v1.4.0" created

service "kubernetes-dashboard" created

Now complete details about the cluster can be seen:

kubectl cluster-info
Kubernetes master is running at https://api.kubernetes.arungupta.me
KubeDNS is running at https://api.kubernetes.arungupta.me/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at https://api.kubernetes.arungupta.me/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

kubectl cluster-info

Kubernetes master is running at https://api.kubernetes.arungupta.me

KubeDNS is running at https://api.kubernetes.arungupta.me/api/v1/proxy/namespaces/kube-system/services/kube-dns

kubernetes-dashboard is running at https://api.kubernetes.arungupta.me/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

And the Kubernetes UI dashboard is at the shown URL. In our case, this is https://api.kubernetes.arungupta.me/ui and looks like:

Credentials for accessing this dashboard can be obtained using the kubectl config view command. The values are shown like:

- name: kubernetes.arungupta.me-basic-auth
  user:
    password: PASSWORD
    username: admin

- name: kubernetes.arungupta.me-basic-auth

user:

password: PASSWORD

username: admin

Deploy Couchbase Service

As explained in Getting Started with Kubernetes 1.4 using Spring Boot and Couchbase, let’s run a Couchbase service:

kubectl create -f ~/workspaces/kubernetes-java-sample/maven/couchbase-service.yml 
service "couchbase-service" created
replicationcontroller "couchbase-rc" created

kubectl create -f ~/workspaces/kubernetes-java-sample/maven/couchbase-service.yml

service "couchbase-service" created

replicationcontroller "couchbase-rc" created

This configuration file is at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/couchbase-service.yml.

Get the list of services:

kubectl get svc
NAME                CLUSTER-IP     EXTERNAL-IP   PORT(S)                                AGE
couchbase-service   100.65.4.139   <none>        8091/TCP,8092/TCP,8093/TCP,11210/TCP   27s
kubernetes          100.64.0.1     <none>        443/TCP                                2h

kubectl get svc

NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE

couchbase-service 100.65.4.139 <none> 8091/TCP,8092/TCP,8093/TCP,11210/TCP 27s

kubernetes 100.64.0.1 <none> 443/TCP 2h

Describe the service:

kubectl describe svc/couchbase-service
Name:                   couchbase-service
Namespace:              default
Labels:                 <none>
Selector:               app=couchbase-rc-pod
Type:                   ClusterIP
IP:                     100.65.4.139
Port:                   admin   8091/TCP
Endpoints:              100.96.5.2:8091
Port:                   views   8092/TCP
Endpoints:              100.96.5.2:8092
Port:                   query   8093/TCP
Endpoints:              100.96.5.2:8093
Port:                   memcached       11210/TCP
Endpoints:              100.96.5.2:11210
Session Affinity:       None

kubectl describe svc/couchbase-service

Name: couchbase-service

Namespace: default

Labels: <none>

Selector: app=couchbase-rc-pod

Type: ClusterIP

IP: 100.65.4.139

Port: admin 8091/TCP

Endpoints: 100.96.5.2:8091

Port: views 8092/TCP

Endpoints: 100.96.5.2:8092

Port: query 8093/TCP

Endpoints: 100.96.5.2:8093

Port: memcached 11210/TCP

Endpoints: 100.96.5.2:11210

Session Affinity: None

Get the pods:

kubectl get pods
NAME                 READY     STATUS    RESTARTS   AGE
couchbase-rc-e35v5   1/1       Running   0          1m

kubectl get pods

NAME READY STATUS RESTARTS AGE

couchbase-rc-e35v5 1/1 Running 0 1m

Run Spring Boot Application

The Spring Boot application runs against the Couchbase service and stores a JSON document in it.

Start the Spring Boot application:

kubectl create -f ~/workspaces/kubernetes-java-sample/maven/bootiful-couchbase.yml 
job "bootiful-couchbase" created

kubectl create -f ~/workspaces/kubernetes-java-sample/maven/bootiful-couchbase.yml

job "bootiful-couchbase" created

This configuration file is at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/bootiful-couchbase.yml.

See list of all the pods:

kubectl get pods --show-all
NAME                       READY     STATUS      RESTARTS   AGE
bootiful-couchbase-ainv8   0/1       Completed   0          1m
couchbase-rc-e35v5         1/1       Running     0          3m

kubectl get pods --show-all

NAME READY STATUS RESTARTS AGE

bootiful-couchbase-ainv8 0/1 Completed 0 1m

couchbase-rc-e35v5 1/1 Running 0 3m

Check logs of the complete pod:

kubectl logs bootiful-couchbase-ainv8

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.4.0.RELEASE)

2016-11-02 18:48:56.035  INFO 7 --- [           main] org.example.webapp.Application           : Starting Application v1.0-SNAPSHOT on bootiful-couchbase-ainv8 with PID 7 (/maven/bootiful-couchbase.jar started by root in /)
2016-11-02 18:48:56.040  INFO 7 --- [           main] org.example.webapp.Application           : No active profile set, falling back to default profiles: default
2016-11-02 18:48:56.115  INFO 7 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@108c4c35: startup date [Wed Nov 02 18:48:56 UTC 2016]; root of context hierarchy
2016-11-02 18:48:57.021  INFO 7 --- [           main] com.couchbase.client.core.CouchbaseCore  : CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile='null', sslKeystorePassword='null', queryEnabled=false, queryPort=8093, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=8, computationPoolSize=8, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, searchServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.2.8 (git: 2.2.8, core: 1.2.9), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=1000, dcpConnectionBufferSize=20971520, dcpConnectionBufferAckThreshold=0.2, dcpConnectionName=dcp/core-io, callbacksOnIoPool=false, queryTimeout=7500, viewTimeout=7500, kvTimeout=2500, connectTimeout=5000, disconnectTimeout=25000, dnsSrvEnabled=false}
2016-11-02 18:48:57.245  INFO 7 --- [      cb-io-1-1] com.couchbase.client.core.node.Node      : Connected to Node couchbase-service
2016-11-02 18:48:57.291  INFO 7 --- [      cb-io-1-1] com.couchbase.client.core.node.Node      : Disconnected from Node couchbase-service
2016-11-02 18:48:57.533  INFO 7 --- [      cb-io-1-2] com.couchbase.client.core.node.Node      : Connected to Node couchbase-service
2016-11-02 18:48:57.638  INFO 7 --- [-computations-4] c.c.c.core.config.ConfigurationProvider  : Opened bucket books
2016-11-02 18:48:58.152  INFO 7 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
Book{isbn=978-1-4919-1889-0, name=Minecraft Modding with Forge, cost=29.99}
2016-11-02 18:48:58.402  INFO 7 --- [           main] org.example.webapp.Application           : Started Application in 2.799 seconds (JVM running for 3.141)
2016-11-02 18:48:58.403  INFO 7 --- [       Thread-5] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@108c4c35: startup date [Wed Nov 02 18:48:56 UTC 2016]; root of context hierarchy
2016-11-02 18:48:58.404  INFO 7 --- [       Thread-5] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown
2016-11-02 18:48:58.410  INFO 7 --- [      cb-io-1-2] com.couchbase.client.core.node.Node      : Disconnected from Node couchbase-service
2016-11-02 18:48:58.410  INFO 7 --- [       Thread-5] c.c.c.core.config.ConfigurationProvider  : Closed bucket books

kubectl logs bootiful-couchbase-ainv8

. ____ _ __ _ _

/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \

( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \

\\/ ___)| |_)| | | | | || (_| | ) ) ) )

' |____| .__|_| |_|_| |_\__, | / / / /

=========|_|==============|___/=/_/_/_/

:: Spring Boot :: (v1.4.0.RELEASE)

2016-11-02 18:48:56.035 INFO 7 --- [ main] org.example.webapp.Application : Starting Application v1.0-SNAPSHOT on bootiful-couchbase-ainv8 with PID 7 (/maven/bootiful-couchbase.jar started by root in /)

2016-11-02 18:48:56.040 INFO 7 --- [ main] org.example.webapp.Application : No active profile set, falling back to default profiles: default

2016-11-02 18:48:56.115 INFO 7 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@108c4c35: startup date [Wed Nov 02 18:48:56 UTC 2016]; root of context hierarchy

2016-11-02 18:48:57.021 INFO 7 --- [ main] com.couchbase.client.core.CouchbaseCore : CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile='null', sslKeystorePassword='null', queryEnabled=false, queryPort=8093, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=8, computationPoolSize=8, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, searchServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.2.8 (git: 2.2.8, core: 1.2.9), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=1000, dcpConnectionBufferSize=20971520, dcpConnectionBufferAckThreshold=0.2, dcpConnectionName=dcp/core-io, callbacksOnIoPool=false, queryTimeout=7500, viewTimeout=7500, kvTimeout=2500, connectTimeout=5000, disconnectTimeout=25000, dnsSrvEnabled=false}

2016-11-02 18:48:57.245 INFO 7 --- [ cb-io-1-1] com.couchbase.client.core.node.Node : Connected to Node couchbase-service

2016-11-02 18:48:57.291 INFO 7 --- [ cb-io-1-1] com.couchbase.client.core.node.Node : Disconnected from Node couchbase-service

2016-11-02 18:48:57.533 INFO 7 --- [ cb-io-1-2] com.couchbase.client.core.node.Node : Connected to Node couchbase-service

2016-11-02 18:48:57.638 INFO 7 --- [-computations-4] c.c.c.core.config.ConfigurationProvider : Opened bucket books

2016-11-02 18:48:58.152 INFO 7 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup

Book{isbn=978-1-4919-1889-0, name=Minecraft Modding with Forge, cost=29.99}

2016-11-02 18:48:58.402 INFO 7 --- [ main] org.example.webapp.Application : Started Application in 2.799 seconds (JVM running for 3.141)

2016-11-02 18:48:58.403 INFO 7 --- [ Thread-5] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@108c4c35: startup date [Wed Nov 02 18:48:56 UTC 2016]; root of context hierarchy

2016-11-02 18:48:58.404 INFO 7 --- [ Thread-5] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown

2016-11-02 18:48:58.410 INFO 7 --- [ cb-io-1-2] com.couchbase.client.core.node.Node : Disconnected from Node couchbase-service

2016-11-02 18:48:58.410 INFO 7 --- [ Thread-5] c.c.c.core.config.ConfigurationProvider : Closed bucket books

The updated dashboard now looks like:

Delete the Kubernetes Cluster

Kubernetes cluster can be deleted as:

kops-darwin-amd64 delete cluster --name=kubernetes.arungupta.me --state=s3://kops-couchbase --yes
TYPE                    NAME                                                                                    ID
autoscaling-config      master-us-west-2a.masters.kubernetes.arungupta.me-20161101235639                        master-us-west-2a.masters.kubernetes.arungupta.me-20161101235639
autoscaling-config      master-us-west-2b.masters.kubernetes.arungupta.me-20161101235639                        master-us-west-2b.masters.kubernetes.arungupta.me-20161101235639
autoscaling-config      master-us-west-2c.masters.kubernetes.arungupta.me-20161101235639                        master-us-west-2c.masters.kubernetes.arungupta.me-20161101235639
autoscaling-config      nodes.kubernetes.arungupta.me-20161101235639                                            nodes.kubernetes.arungupta.me-20161101235639
autoscaling-group       master-us-west-2a.masters.kubernetes.arungupta.me                                       master-us-west-2a.masters.kubernetes.arungupta.me
autoscaling-group       master-us-west-2b.masters.kubernetes.arungupta.me                                       master-us-west-2b.masters.kubernetes.arungupta.me
autoscaling-group       master-us-west-2c.masters.kubernetes.arungupta.me                                       master-us-west-2c.masters.kubernetes.arungupta.me
autoscaling-group       nodes.kubernetes.arungupta.me                                                           nodes.kubernetes.arungupta.me
dhcp-options            kubernetes.arungupta.me                                                                 dopt-9b7b08ff
iam-instance-profile    masters.kubernetes.arungupta.me                                                         masters.kubernetes.arungupta.me
iam-instance-profile    nodes.kubernetes.arungupta.me                                                           nodes.kubernetes.arungupta.me
iam-role                masters.kubernetes.arungupta.me                                                         masters.kubernetes.arungupta.me
iam-role                nodes.kubernetes.arungupta.me                                                           nodes.kubernetes.arungupta.me
instance                master-us-west-2a.masters.kubernetes.arungupta.me                                       i-8798eb9f
instance                master-us-west-2b.masters.kubernetes.arungupta.me                                       i-eca96ab3
instance                master-us-west-2c.masters.kubernetes.arungupta.me                                       i-63fd3dbf
instance                nodes.kubernetes.arungupta.me                                                           i-21a96a7e
instance                nodes.kubernetes.arungupta.me                                                           i-57fb3b8b
instance                nodes.kubernetes.arungupta.me                                                           i-5c99ea44
internet-gateway        kubernetes.arungupta.me                                                                 igw-b624abd2
keypair                 kubernetes.kubernetes.arungupta.me-18:90:41:6f:5f:79:6a:a8:d5:b6:b8:3f:10:d5:d3:f3      kubernetes.kubernetes.arungupta.me-18:90:41:6f:5f:79:6a:a8:d5:b6:b8:3f:10:d5:d3:f3
route-table             kubernetes.arungupta.me                                                                 rtb-e44df183
route53-record          api.internal.kubernetes.arungupta.me.                                                   Z6I41VJM5VCZV/api.internal.kubernetes.arungupta.me.
route53-record          api.kubernetes.arungupta.me.                                                            Z6I41VJM5VCZV/api.kubernetes.arungupta.me.
route53-record          etcd-events-us-west-2a.internal.kubernetes.arungupta.me.                                Z6I41VJM5VCZV/etcd-events-us-west-2a.internal.kubernetes.arungupta.me.
route53-record          etcd-events-us-west-2b.internal.kubernetes.arungupta.me.                                Z6I41VJM5VCZV/etcd-events-us-west-2b.internal.kubernetes.arungupta.me.
route53-record          etcd-events-us-west-2c.internal.kubernetes.arungupta.me.                                Z6I41VJM5VCZV/etcd-events-us-west-2c.internal.kubernetes.arungupta.me.
route53-record          etcd-us-west-2a.internal.kubernetes.arungupta.me.                                       Z6I41VJM5VCZV/etcd-us-west-2a.internal.kubernetes.arungupta.me.
route53-record          etcd-us-west-2b.internal.kubernetes.arungupta.me.                                       Z6I41VJM5VCZV/etcd-us-west-2b.internal.kubernetes.arungupta.me.
route53-record          etcd-us-west-2c.internal.kubernetes.arungupta.me.                                       Z6I41VJM5VCZV/etcd-us-west-2c.internal.kubernetes.arungupta.me.
security-group          masters.kubernetes.arungupta.me                                                         sg-3e790f47
security-group          nodes.kubernetes.arungupta.me                                                           sg-3f790f46
subnet                  us-west-2a.kubernetes.arungupta.me                                                      subnet-3cdbc958
subnet                  us-west-2b.kubernetes.arungupta.me                                                      subnet-18c3f76e
subnet                  us-west-2c.kubernetes.arungupta.me                                                      subnet-b30f6deb
volume                  us-west-2a.etcd-events.kubernetes.arungupta.me                                          vol-202350a8
volume                  us-west-2a.etcd-main.kubernetes.arungupta.me                                            vol-0a235082
volume                  us-west-2b.etcd-events.kubernetes.arungupta.me                                          vol-401f5bf4
volume                  us-west-2b.etcd-main.kubernetes.arungupta.me                                            vol-691f5bdd
volume                  us-west-2c.etcd-events.kubernetes.arungupta.me                                          vol-aefe163b
volume                  us-west-2c.etcd-main.kubernetes.arungupta.me                                            vol-e9fd157c
vpc                     kubernetes.arungupta.me                                                                 vpc-e5f50382

internet-gateway:igw-b624abd2   still has dependencies, will retry
keypair:kubernetes.kubernetes.arungupta.me-18:90:41:6f:5f:79:6a:a8:d5:b6:b8:3f:10:d5:d3:f3      ok
instance:i-5c99ea44     ok
instance:i-63fd3dbf     ok
instance:i-eca96ab3     ok
instance:i-21a96a7e     ok
autoscaling-group:master-us-west-2a.masters.kubernetes.arungupta.me     ok
autoscaling-group:master-us-west-2b.masters.kubernetes.arungupta.me     ok
autoscaling-group:master-us-west-2c.masters.kubernetes.arungupta.me     ok
autoscaling-group:nodes.kubernetes.arungupta.me ok
iam-instance-profile:nodes.kubernetes.arungupta.me      ok
iam-instance-profile:masters.kubernetes.arungupta.me    ok
instance:i-57fb3b8b     ok
instance:i-8798eb9f     ok
route53-record:Z6I41VJM5VCZV/etcd-events-us-west-2a.internal.kubernetes.arungupta.me.   ok
iam-role:nodes.kubernetes.arungupta.me  ok
iam-role:masters.kubernetes.arungupta.me        ok
autoscaling-config:nodes.kubernetes.arungupta.me-20161101235639 ok
autoscaling-config:master-us-west-2b.masters.kubernetes.arungupta.me-20161101235639     ok
subnet:subnet-b30f6deb  still has dependencies, will retry
subnet:subnet-3cdbc958  still has dependencies, will retry
subnet:subnet-18c3f76e  still has dependencies, will retry
autoscaling-config:master-us-west-2a.masters.kubernetes.arungupta.me-20161101235639     ok
autoscaling-config:master-us-west-2c.masters.kubernetes.arungupta.me-20161101235639     ok
volume:vol-0a235082     still has dependencies, will retry
volume:vol-202350a8     still has dependencies, will retry
volume:vol-401f5bf4     still has dependencies, will retry
volume:vol-e9fd157c     still has dependencies, will retry
volume:vol-aefe163b     still has dependencies, will retry
volume:vol-691f5bdd     still has dependencies, will retry
security-group:sg-3f790f46      still has dependencies, will retry
security-group:sg-3e790f47      still has dependencies, will retry
Not all resources deleted; waiting before reattempting deletion
        internet-gateway:igw-b624abd2
        security-group:sg-3f790f46
        volume:vol-aefe163b
        route-table:rtb-e44df183
        volume:vol-401f5bf4
        subnet:subnet-18c3f76e
        security-group:sg-3e790f47
        volume:vol-691f5bdd
        subnet:subnet-3cdbc958
        volume:vol-202350a8
        volume:vol-0a235082
        dhcp-options:dopt-9b7b08ff
        subnet:subnet-b30f6deb
        volume:vol-e9fd157c
        vpc:vpc-e5f50382
internet-gateway:igw-b624abd2   still has dependencies, will retry
volume:vol-e9fd157c     still has dependencies, will retry
subnet:subnet-3cdbc958  still has dependencies, will retry
subnet:subnet-18c3f76e  still has dependencies, will retry
subnet:subnet-b30f6deb  still has dependencies, will retry
volume:vol-0a235082     still has dependencies, will retry
volume:vol-aefe163b     still has dependencies, will retry
volume:vol-691f5bdd     still has dependencies, will retry
volume:vol-202350a8     still has dependencies, will retry
volume:vol-401f5bf4     still has dependencies, will retry
security-group:sg-3f790f46      still has dependencies, will retry
security-group:sg-3e790f47      still has dependencies, will retry
Not all resources deleted; waiting before reattempting deletion
        subnet:subnet-b30f6deb
        volume:vol-e9fd157c
        vpc:vpc-e5f50382
        internet-gateway:igw-b624abd2
        security-group:sg-3f790f46
        volume:vol-aefe163b
        route-table:rtb-e44df183
        volume:vol-401f5bf4
        subnet:subnet-18c3f76e
        security-group:sg-3e790f47
        volume:vol-691f5bdd
        subnet:subnet-3cdbc958
        volume:vol-202350a8
        volume:vol-0a235082
        dhcp-options:dopt-9b7b08ff
subnet:subnet-18c3f76e  still has dependencies, will retry
subnet:subnet-b30f6deb  still has dependencies, will retry
internet-gateway:igw-b624abd2   still has dependencies, will retry
subnet:subnet-3cdbc958  still has dependencies, will retry
volume:vol-691f5bdd     still has dependencies, will retry
volume:vol-0a235082     still has dependencies, will retry
volume:vol-202350a8     still has dependencies, will retry
volume:vol-401f5bf4     still has dependencies, will retry
volume:vol-aefe163b     still has dependencies, will retry
volume:vol-e9fd157c     still has dependencies, will retry
security-group:sg-3e790f47      still has dependencies, will retry
security-group:sg-3f790f46      still has dependencies, will retry
Not all resources deleted; waiting before reattempting deletion
        internet-gateway:igw-b624abd2
        security-group:sg-3f790f46
        volume:vol-aefe163b
        route-table:rtb-e44df183
        volume:vol-401f5bf4
        subnet:subnet-18c3f76e
        security-group:sg-3e790f47
        volume:vol-691f5bdd
        subnet:subnet-3cdbc958
        volume:vol-202350a8
        volume:vol-0a235082
        dhcp-options:dopt-9b7b08ff
        subnet:subnet-b30f6deb
        volume:vol-e9fd157c
        vpc:vpc-e5f50382
subnet:subnet-b30f6deb  still has dependencies, will retry
volume:vol-202350a8     still has dependencies, will retry
internet-gateway:igw-b624abd2   still has dependencies, will retry
subnet:subnet-18c3f76e  still has dependencies, will retry
volume:vol-e9fd157c     still has dependencies, will retry
volume:vol-aefe163b     still has dependencies, will retry
volume:vol-401f5bf4     still has dependencies, will retry
volume:vol-691f5bdd     still has dependencies, will retry
security-group:sg-3e790f47      still has dependencies, will retry
security-group:sg-3f790f46      still has dependencies, will retry
subnet:subnet-3cdbc958  still has dependencies, will retry
volume:vol-0a235082     still has dependencies, will retry
Not all resources deleted; waiting before reattempting deletion
        internet-gateway:igw-b624abd2
        security-group:sg-3f790f46
        volume:vol-aefe163b
        route-table:rtb-e44df183
        subnet:subnet-18c3f76e
        security-group:sg-3e790f47
        volume:vol-691f5bdd
        volume:vol-401f5bf4
        volume:vol-202350a8
        subnet:subnet-3cdbc958
        volume:vol-0a235082
        dhcp-options:dopt-9b7b08ff
        subnet:subnet-b30f6deb
        volume:vol-e9fd157c
        vpc:vpc-e5f50382
subnet:subnet-18c3f76e  ok
volume:vol-e9fd157c     ok
volume:vol-401f5bf4     ok
volume:vol-0a235082     ok
volume:vol-691f5bdd     ok
subnet:subnet-3cdbc958  ok
volume:vol-aefe163b     ok
subnet:subnet-b30f6deb  ok
internet-gateway:igw-b624abd2   ok
volume:vol-202350a8     ok
security-group:sg-3f790f46      ok
security-group:sg-3e790f47      ok
route-table:rtb-e44df183        ok
vpc:vpc-e5f50382        ok
dhcp-options:dopt-9b7b08ff      ok

Cluster deleted

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

kops-darwin-amd64 delete cluster --name=kubernetes.arungupta.me --state=s3://kops-couchbase --yes

TYPE NAME ID

autoscaling-config master-us-west-2a.masters.kubernetes.arungupta.me-20161101235639 master-us-west-2a.masters.kubernetes.arungupta.me-20161101235639

autoscaling-config master-us-west-2b.masters.kubernetes.arungupta.me-20161101235639 master-us-west-2b.masters.kubernetes.arungupta.me-20161101235639

autoscaling-config master-us-west-2c.masters.kubernetes.arungupta.me-20161101235639 master-us-west-2c.masters.kubernetes.arungupta.me-20161101235639

autoscaling-config nodes.kubernetes.arungupta.me-20161101235639 nodes.kubernetes.arungupta.me-20161101235639

autoscaling-group master-us-west-2a.masters.kubernetes.arungupta.me master-us-west-2a.masters.kubernetes.arungupta.me

autoscaling-group master-us-west-2b.masters.kubernetes.arungupta.me master-us-west-2b.masters.kubernetes.arungupta.me

autoscaling-group master-us-west-2c.masters.kubernetes.arungupta.me master-us-west-2c.masters.kubernetes.arungupta.me

autoscaling-group nodes.kubernetes.arungupta.me nodes.kubernetes.arungupta.me

dhcp-options kubernetes.arungupta.me dopt-9b7b08ff

iam-instance-profile masters.kubernetes.arungupta.me masters.kubernetes.arungupta.me

iam-instance-profile nodes.kubernetes.arungupta.me nodes.kubernetes.arungupta.me

iam-role masters.kubernetes.arungupta.me masters.kubernetes.arungupta.me

iam-role nodes.kubernetes.arungupta.me nodes.kubernetes.arungupta.me

instance master-us-west-2a.masters.kubernetes.arungupta.me i-8798eb9f

instance master-us-west-2b.masters.kubernetes.arungupta.me i-eca96ab3

instance master-us-west-2c.masters.kubernetes.arungupta.me i-63fd3dbf

instance nodes.kubernetes.arungupta.me i-21a96a7e

instance nodes.kubernetes.arungupta.me i-57fb3b8b

instance nodes.kubernetes.arungupta.me i-5c99ea44

internet-gateway kubernetes.arungupta.me igw-b624abd2

keypair kubernetes.kubernetes.arungupta.me-18:90:41:6f:5f:79:6a:a8:d5:b6:b8:3f:10:d5:d3:f3 kubernetes.kubernetes.arungupta.me-18:90:41:6f:5f:79:6a:a8:d5:b6:b8:3f:10:d5:d3:f3

route-table kubernetes.arungupta.me rtb-e44df183

route53-record api.internal.kubernetes.arungupta.me. Z6I41VJM5VCZV/api.internal.kubernetes.arungupta.me.

route53-record api.kubernetes.arungupta.me. Z6I41VJM5VCZV/api.kubernetes.arungupta.me.

route53-record etcd-events-us-west-2a.internal.kubernetes.arungupta.me. Z6I41VJM5VCZV/etcd-events-us-west-2a.internal.kubernetes.arungupta.me.

route53-record etcd-events-us-west-2b.internal.kubernetes.arungupta.me. Z6I41VJM5VCZV/etcd-events-us-west-2b.internal.kubernetes.arungupta.me.

route53-record etcd-events-us-west-2c.internal.kubernetes.arungupta.me. Z6I41VJM5VCZV/etcd-events-us-west-2c.internal.kubernetes.arungupta.me.

route53-record etcd-us-west-2a.internal.kubernetes.arungupta.me. Z6I41VJM5VCZV/etcd-us-west-2a.internal.kubernetes.arungupta.me.

route53-record etcd-us-west-2b.internal.kubernetes.arungupta.me. Z6I41VJM5VCZV/etcd-us-west-2b.internal.kubernetes.arungupta.me.

route53-record etcd-us-west-2c.internal.kubernetes.arungupta.me. Z6I41VJM5VCZV/etcd-us-west-2c.internal.kubernetes.arungupta.me.

security-group masters.kubernetes.arungupta.me sg-3e790f47

security-group nodes.kubernetes.arungupta.me sg-3f790f46

subnet us-west-2a.kubernetes.arungupta.me subnet-3cdbc958

subnet us-west-2b.kubernetes.arungupta.me subnet-18c3f76e

subnet us-west-2c.kubernetes.arungupta.me subnet-b30f6deb

volume us-west-2a.etcd-events.kubernetes.arungupta.me vol-202350a8

volume us-west-2a.etcd-main.kubernetes.arungupta.me vol-0a235082

volume us-west-2b.etcd-events.kubernetes.arungupta.me vol-401f5bf4

volume us-west-2b.etcd-main.kubernetes.arungupta.me vol-691f5bdd

volume us-west-2c.etcd-events.kubernetes.arungupta.me vol-aefe163b

volume us-west-2c.etcd-main.kubernetes.arungupta.me vol-e9fd157c

vpc kubernetes.arungupta.me vpc-e5f50382

internet-gateway:igw-b624abd2 still has dependencies, will retry

keypair:kubernetes.kubernetes.arungupta.me-18:90:41:6f:5f:79:6a:a8:d5:b6:b8:3f:10:d5:d3:f3 ok

instance:i-5c99ea44 ok

instance:i-63fd3dbf ok

instance:i-eca96ab3 ok

instance:i-21a96a7e ok

autoscaling-group:master-us-west-2a.masters.kubernetes.arungupta.me ok

autoscaling-group:master-us-west-2b.masters.kubernetes.arungupta.me ok

autoscaling-group:master-us-west-2c.masters.kubernetes.arungupta.me ok

autoscaling-group:nodes.kubernetes.arungupta.me ok

iam-instance-profile:nodes.kubernetes.arungupta.me ok

iam-instance-profile:masters.kubernetes.arungupta.me ok

instance:i-57fb3b8b ok

instance:i-8798eb9f ok

route53-record:Z6I41VJM5VCZV/etcd-events-us-west-2a.internal.kubernetes.arungupta.me. ok

iam-role:nodes.kubernetes.arungupta.me ok

iam-role:masters.kubernetes.arungupta.me ok

autoscaling-config:nodes.kubernetes.arungupta.me-20161101235639 ok

autoscaling-config:master-us-west-2b.masters.kubernetes.arungupta.me-20161101235639 ok

subnet:subnet-b30f6deb still has dependencies, will retry

subnet:subnet-3cdbc958 still has dependencies, will retry

subnet:subnet-18c3f76e still has dependencies, will retry

autoscaling-config:master-us-west-2a.masters.kubernetes.arungupta.me-20161101235639 ok

autoscaling-config:master-us-west-2c.masters.kubernetes.arungupta.me-20161101235639 ok

volume:vol-0a235082 still has dependencies, will retry

volume:vol-202350a8 still has dependencies, will retry

volume:vol-401f5bf4 still has dependencies, will retry

volume:vol-e9fd157c still has dependencies, will retry

volume:vol-aefe163b still has dependencies, will retry

volume:vol-691f5bdd still has dependencies, will retry

security-group:sg-3f790f46 still has dependencies, will retry

security-group:sg-3e790f47 still has dependencies, will retry

Not all resources deleted; waiting before reattempting deletion

internet-gateway:igw-b624abd2

security-group:sg-3f790f46

volume:vol-aefe163b

route-table:rtb-e44df183

volume:vol-401f5bf4

subnet:subnet-18c3f76e

security-group:sg-3e790f47

volume:vol-691f5bdd

subnet:subnet-3cdbc958

volume:vol-202350a8

volume:vol-0a235082

dhcp-options:dopt-9b7b08ff

subnet:subnet-b30f6deb

volume:vol-e9fd157c

vpc:vpc-e5f50382

internet-gateway:igw-b624abd2 still has dependencies, will retry

volume:vol-e9fd157c still has dependencies, will retry

subnet:subnet-3cdbc958 still has dependencies, will retry

subnet:subnet-18c3f76e still has dependencies, will retry

subnet:subnet-b30f6deb still has dependencies, will retry

volume:vol-0a235082 still has dependencies, will retry

volume:vol-aefe163b still has dependencies, will retry

volume:vol-691f5bdd still has dependencies, will retry

volume:vol-202350a8 still has dependencies, will retry

volume:vol-401f5bf4 still has dependencies, will retry

security-group:sg-3f790f46 still has dependencies, will retry

security-group:sg-3e790f47 still has dependencies, will retry

Not all resources deleted; waiting before reattempting deletion

subnet:subnet-b30f6deb

volume:vol-e9fd157c

vpc:vpc-e5f50382

internet-gateway:igw-b624abd2

security-group:sg-3f790f46

volume:vol-aefe163b

route-table:rtb-e44df183

volume:vol-401f5bf4

subnet:subnet-18c3f76e

security-group:sg-3e790f47

volume:vol-691f5bdd

subnet:subnet-3cdbc958

volume:vol-202350a8

volume:vol-0a235082

dhcp-options:dopt-9b7b08ff

subnet:subnet-18c3f76e still has dependencies, will retry

subnet:subnet-b30f6deb still has dependencies, will retry

internet-gateway:igw-b624abd2 still has dependencies, will retry

subnet:subnet-3cdbc958 still has dependencies, will retry

volume:vol-691f5bdd still has dependencies, will retry

volume:vol-0a235082 still has dependencies, will retry

volume:vol-202350a8 still has dependencies, will retry

volume:vol-401f5bf4 still has dependencies, will retry

volume:vol-aefe163b still has dependencies, will retry

volume:vol-e9fd157c still has dependencies, will retry

security-group:sg-3e790f47 still has dependencies, will retry

security-group:sg-3f790f46 still has dependencies, will retry

Not all resources deleted; waiting before reattempting deletion

internet-gateway:igw-b624abd2

security-group:sg-3f790f46

volume:vol-aefe163b

route-table:rtb-e44df183

volume:vol-401f5bf4

subnet:subnet-18c3f76e

security-group:sg-3e790f47

volume:vol-691f5bdd

subnet:subnet-3cdbc958

volume:vol-202350a8

volume:vol-0a235082

dhcp-options:dopt-9b7b08ff

subnet:subnet-b30f6deb

volume:vol-e9fd157c

vpc:vpc-e5f50382

subnet:subnet-b30f6deb still has dependencies, will retry

volume:vol-202350a8 still has dependencies, will retry

internet-gateway:igw-b624abd2 still has dependencies, will retry

subnet:subnet-18c3f76e still has dependencies, will retry

volume:vol-e9fd157c still has dependencies, will retry

volume:vol-aefe163b still has dependencies, will retry

volume:vol-401f5bf4 still has dependencies, will retry

volume:vol-691f5bdd still has dependencies, will retry

security-group:sg-3e790f47 still has dependencies, will retry

security-group:sg-3f790f46 still has dependencies, will retry

subnet:subnet-3cdbc958 still has dependencies, will retry

volume:vol-0a235082 still has dependencies, will retry

Not all resources deleted; waiting before reattempting deletion

internet-gateway:igw-b624abd2

security-group:sg-3f790f46

volume:vol-aefe163b

route-table:rtb-e44df183

subnet:subnet-18c3f76e

security-group:sg-3e790f47

volume:vol-691f5bdd

volume:vol-401f5bf4

volume:vol-202350a8

subnet:subnet-3cdbc958

volume:vol-0a235082

dhcp-options:dopt-9b7b08ff

subnet:subnet-b30f6deb

volume:vol-e9fd157c

vpc:vpc-e5f50382

subnet:subnet-18c3f76e ok

volume:vol-e9fd157c ok

volume:vol-401f5bf4 ok

volume:vol-0a235082 ok

volume:vol-691f5bdd ok

subnet:subnet-3cdbc958 ok

volume:vol-aefe163b ok

subnet:subnet-b30f6deb ok

internet-gateway:igw-b624abd2 ok

volume:vol-202350a8 ok

security-group:sg-3f790f46 ok

security-group:sg-3e790f47 ok

route-table:rtb-e44df183 ok

vpc:vpc-e5f50382 ok

dhcp-options:dopt-9b7b08ff ok

Cluster deleted

couchbase.com/containers provide more details about how to run Couchbase in different container frameworks.

More information about Couchbase:

Couchbase Developer Portal
Couchbase Forums
@couchbasedev or @couchbase

Source: blog.couchbase.com/2016/november/multimaster-kubernetes-cluster-amazon-kops

Docker Container Anti Patterns

October 30, 2016containers, couchbasecontainers, dockerarungupta

This blog will explain 10 containers anti-patterns that I’ve seen over the past few months:

Data or logs in containers – Containers are ideal for stateless applications and are meant to be ephemeral. This means no data or logs should be stored in the container otherwise they’ll be lost when the container terminates. Instead use volume mapping to persist them outside the containers. ELK stack could be used to store and process logs. If managed volumes are used during early testing process, then remove them using -v switch with the docker rm command.
IP addresses of container – Each container is assigned an IP address. Multiple containers communicate with each other to create an application, for example an application deployed on an application server will need to talk with a database. Existing containers are terminated and new containers are started all the time. Relying upon the IP address of the container will require constantly updating the application configuration. This will make the application fragile. Instead create services. This will provide a logical name that can be referred independent of the growing and shrinking number of containers. And it also provides a basic load balancing as well.
Run a single process in a container – A Dockerfile has use one CMD and ENTRYPOINT. Often, CMD will use a script that will perform some configuration of the image and then start the container. Don’t try to start multiple processes using that script. Its important to follow separation of concerns pattern when creating Docker images. This will make managing your containers, collecting logs, updating each individual process that much harder. You may consider breaking up application into multiple containers and manage them independently.
Don’t use docker exec – The docker exec command starts a new command in a running container. This is useful for attaching a shell using the docker exec -it {cid} bash. But other than that the container is already running the process that its supposed to be running.
Keep your image lean – Create a new directory and include Dockerfile and other relevant files in that directory. Also consider using .dockerignore to remove any logs, source code, logs etc before creating the image. Make sure to remove any downloaded artifacts after they are unzipped.
Create image from a running container – A new image can be created using the docker commit command. This is useful when any changes in the container have been made. But images created using this are non-reproducible. Instead make changes in the Dockerfile, terminate existing containers and start a new container with the updated image.
Security credentials in Docker image – Do not store security credentials in the Dockerfile. They are in clear text and checked into a repository. This makes them completely vulnerable. Use -e to specify passwords as runtime environment variable. Alternatively --env-file can be used to read environment variables from a file. Another approach is to used CMD or ENTRYPOINT to specify a script. This script will pull the credentials from a third party and then configure your application.
latest tag: Starting with an image like couchbase is tempting. If no tags are specified then a container is started using the image couchbase:latest. This image may not actually be latest and instead refer to an older version. Taking an application into production requires a fully controller environment with exact version of the image. Read this Docker: The latest confusion post by fellow Docker Captain @adrianmouat. Make sure to always use tag when running a container. For example, use couchbase:enterprise-4.5.1 instead of just couchbase.
Impedance mismatch – Don’t use different images, or even different tags in dev, test, staging and production environment. The image that is the “source of truth” should be created once and pushed to a repo. That image should be used for different environments going forward. In some cases, you may consider running your unit tests on the WAR file as part of maven build and then create the image. But any system integration testing should be done on the image that will be pushed in production.
Publishing ports – Don’t use -P to publish all the exposed ports. This will allow you to run multiple containers and publish their exposed ports. But this also means that all the ports will be published. Instead use -p to publish specific ports.

Adding more based upon discussion on twitter …

Root user – Don’t run containers as root user. The host and the container share the same kernel. If the container is compromised, a root user can do more damage to the underlying hosts. Use RUN groupadd -r couchbase && useradd -r -g couchbase couchbase to create a group and a user in it. Use the USER instruction to switch to that user. Each USER creates a new layer in the image. Avoid switching the user back and forth to reduce the number of layers. Thanks to @Aleksandar_78 for this tip!
Dependency between containers – Often applications rely upon containers to be started in a certain order. For example, a database container must be up before an application can connect to it. The application should be resilient to such changes as the containers may be terminated or started at any time. In this case, have the application container wait for the database connection to succeed before proceeding further. Do not use wait-for scripts in Dockerfile for the containers to startup in a specific order. Particularly waiting for a certain number of seconds for a particular container to start is very fragile. Thanks to @ratnopam for this tip!

What other anti-patterns do you follow?

Docker for Java developers is a self-paced hands-on workshop that explains how to get started with Docker for Java developers.

Interested in a more deep dive tutorial? Watch this 2-hours tutorial from JavaOne!

couchbase.com/containers shows how to run Couchbase in a variety of frameworks.

Source: blog.couchbase.com/2016/october/docker-container-anti-patterns

Docker on Windows 2016 Server

October 28, 2016containerscontainers, docker, windowsarungupta

This blog is the first part of a multi-part series. The first part showed how to set up Windows Server 2016 as a VirtualBox VM. This second part will show how to configure Docker on Windows 2016 VM.

Start an elevated PowerShell session:

Run the script to install Docker:

Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
Install-Package -Name docker -ProviderName DockerMsftProvider
Restart-Computer -Force

Install-Module -Name DockerMsftProvider -Repository PSGallery -Force

Install-Package -Name docker -ProviderName DockerMsftProvider

Restart-Computer -Force

This will install the PowerShell module, enable containers feature and install Docker.

The VM needs to be restarted in order for the containers to be enabled. Refer to Container Host Deployment – Windows Server for more detailed instructions.

The VM reboots. Start a PowerShell and check Docker version using docker version command:More details about Docker can be found using the docker info command:
Run your first Docker container using the docker run -it -p 80:80 microsoft/iis command:This will download the Microsoft IIS server Docker image. This is going to take a while so please be patient!

Once the 8.9 GB image is downloaded (after a while), the IIS server is started for you. Check the list of images using the docker images command and the list of running containers using the docker ps command:

More details about the container can be found using the docker inspect command:

[
  {
    "Id": "c15555e4b60ec24bbad2b8834c82e14313d193e07da2ca7d3948802e36f4d828",
    "Created": "2016-10-27T23:45:20.4069634Z",
    "Path": "C:\\ServiceMonitor.exe",
    "Args": [
      "w3svc"
    ],
    "State": {
      "Status": "running",
      "Running": true,
      "Paused": false,
      "Restarting": false,
      "OOMKilled": false,
      "Dead": false,
      "Pid": 4228,
      "ExitCode": 0,
      "Error": "",
      "StartedAt": "2016-10-27T23:45:34.5990616Z",
      "FinishedAt": "0001-01-01T00:00:00Z"
    },
    "Image": "sha256:b6a44de60ef92f9eceb68effcfc5fdbd5ecb53677e8c4f1e8dcd02b0c11cb760",
    "ResolvConfPath": "",
    "HostnamePath": "",
    "HostsPath": "",
    "LogPath": "C:\\ProgramData\\docker\\containers\\c15555e4b60ec24bbad2b8834c82e14313d193e07da2ca7d3948802e36f4d828\\c15555e4b60ec24bbad2b8834c82e14313d193e07da2ca7d3948802e36f4d828-json.log",
    "Name": "/cocky_mahavira",
    "RestartCount": 0,
    "Driver": "windowsfilter",
    "MountLabel": "",
    "ProcessLabel": "",
    "AppArmorProfile": "",
    "ExecIDs": null,
    "HostConfig": {
      "Binds": null,
      "ContainerIDFile": "",
      "LogConfig": {
        "Type": "json-file",
        "Config": {}
      },
      "NetworkMode": "default",
      "PortBindings": {
        "80/tcp": [
          {
            "HostIp": "",
            "HostPort": "80"
          }
        ]
      },
      "RestartPolicy": {
        "Name": "no",
        "MaximumRetryCount": 0
      },
      "AutoRemove": false,
      "VolumeDriver": "",
      "VolumesFrom": null,
      "CapAdd": null,
      "CapDrop": null,
      "Dns": [],
      "DnsOptions": [],
      "DnsSearch": [],
      "ExtraHosts": null,
      "GroupAdd": null,
      "IpcMode": "",
      "Cgroup": "",
      "Links": null,
      "OomScoreAdj": 0,
      "PidMode": "",
      "Privileged": false,
      "PublishAllPorts": false,
      "ReadonlyRootfs": false,
      "SecurityOpt": null,
      "UTSMode": "",
      "UsernsMode": "",
      "ShmSize": 0,
      "ConsoleSize": [
        50,
        120
      ],
      "Isolation": "process",
      "CpuShares": 0,
      "Memory": 0,
      "CgroupParent": "",
      "BlkioWeight": 0,
      "BlkioWeightDevice": null,
      "BlkioDeviceReadBps": null,
      "BlkioDeviceWriteBps": null,
      "BlkioDeviceReadIOps": null,
      "BlkioDeviceWriteIOps": null,
      "CpuPeriod": 0,
      "CpuQuota": 0,
      "CpusetCpus": "",
      "CpusetMems": "",
      "Devices": [],
      "DiskQuota": 0,
      "KernelMemory": 0,
      "MemoryReservation": 0,
      "MemorySwap": 0,
      "MemorySwappiness": -1,
      "OomKillDisable": false,
      "PidsLimit": 0,
      "Ulimits": null,
      "CpuCount": 0,
      "CpuPercent": 0,
      "IOMaximumIOps": 0,
      "IOMaximumBandwidth": 0
    },
    "GraphDriver": {
      "Name": "windowsfilter",
      "Data": {
        "dir": "C:\\ProgramData\\docker\\windowsfilter\\c15555e4b60ec24bbad2b8834c82e14313d193e07da2ca7d3948802e36f4d828"
      }
    },
    "Mounts": [],
    "Config": {
      "Hostname": "c15555e4b60e",
      "Domainname": "",
      "User": "",
      "AttachStdin": true,
      "AttachStdout": true,
      "AttachStderr": true,
      "ExposedPorts": {
        "80/tcp": {}
      },
      "Tty": true,
      "OpenStdin": true,
      "StdinOnce": true,
      "Env": null,
      "Cmd": null,
      "ArgsEscaped": true,
      "Image": "microsoft/iis",
      "Volumes": null,
      "WorkingDir": "",
      "Entrypoint": [
        "C:\\ServiceMonitor.exe",
        "w3svc"
      ],
      "OnBuild": null,
      "Labels": {}
    },
    "NetworkSettings": {
      "Bridge": "",
      "SandboxID": "b23c5766990bb9373e6cbb69cb54b011f57974ed381e60c1eacb7c3c47e303b2",
      "HairpinMode": false,
      "LinkLocalIPv6Address": "",
      "LinkLocalIPv6PrefixLen": 0,
      "Ports": {
        "80/tcp": [
          {
            "HostIp": "0.0.0.0",
            "HostPort": "80"
          }
        ]
      },
      "SandboxKey": "b23c5766990b",
      "SecondaryIPAddresses": null,
      "SecondaryIPv6Addresses": null,
      "EndpointID": "",
      "Gateway": "",
      "GlobalIPv6Address": "",
      "GlobalIPv6PrefixLen": 0,
      "IPAddress": "",
      "IPPrefixLen": 0,
      "IPv6Gateway": "",
      "MacAddress": "",
      "Networks": {
        "nat": {
          "IPAMConfig": null,
          "Links": null,
          "Aliases": null,
          "NetworkID": "9ae352610731342dde19246e759944e79af7a19077d879024a17cedca4b061c9",
          "EndpointID": "198d0a52838f1e5178b3fedc154e9f18409c13f0de489e99c06faf9ed2fbedd9",
          "Gateway": "",
          "IPAddress": "172.26.192.89",
          "IPPrefixLen": 16,
          "IPv6Gateway": "",
          "GlobalIPv6Address": "",
          "GlobalIPv6PrefixLen": 0,
          "MacAddress": "00:15:5d:7e:60:60"
        }
      }
    }
  }
]

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

[

{

"Id": "c15555e4b60ec24bbad2b8834c82e14313d193e07da2ca7d3948802e36f4d828",

"Created": "2016-10-27T23:45:20.4069634Z",

"Path": "C:\\ServiceMonitor.exe",

"Args": [

"w3svc"

"State": {

"Status": "running",

"Running": true,

"Paused": false,

"Restarting": false,

"OOMKilled": false,

"Dead": false,

"Pid": 4228,

"ExitCode": 0,

"Error": "",

"StartedAt": "2016-10-27T23:45:34.5990616Z",

"FinishedAt": "0001-01-01T00:00:00Z"

"Image": "sha256:b6a44de60ef92f9eceb68effcfc5fdbd5ecb53677e8c4f1e8dcd02b0c11cb760",

"ResolvConfPath": "",

"HostnamePath": "",

"HostsPath": "",

"LogPath": "C:\\ProgramData\\docker\\containers\\c15555e4b60ec24bbad2b8834c82e14313d193e07da2ca7d3948802e36f4d828\\c15555e4b60ec24bbad2b8834c82e14313d193e07da2ca7d3948802e36f4d828-json.log",

"Name": "/cocky_mahavira",

"RestartCount": 0,

"Driver": "windowsfilter",

"MountLabel": "",

"ProcessLabel": "",

"AppArmorProfile": "",

"ExecIDs": null,

"HostConfig": {

"Binds": null,

"ContainerIDFile": "",

"LogConfig": {

"Type": "json-file",

"Config": {}

"NetworkMode": "default",

"PortBindings": {

"80/tcp": [

{

"HostIp": "",

"HostPort": "80"

}

]

"RestartPolicy": {

"Name": "no",

"MaximumRetryCount": 0

"AutoRemove": false,

"VolumeDriver": "",

"VolumesFrom": null,

"CapAdd": null,

"CapDrop": null,

"Dns": [],

"DnsOptions": [],

"DnsSearch": [],

"ExtraHosts": null,

"GroupAdd": null,

"IpcMode": "",

"Cgroup": "",

"Links": null,

"OomScoreAdj": 0,

"PidMode": "",

"Privileged": false,

"PublishAllPorts": false,

"ReadonlyRootfs": false,

"SecurityOpt": null,

"UTSMode": "",

"UsernsMode": "",

"ShmSize": 0,

"ConsoleSize": [

50,

120

"Isolation": "process",

"CpuShares": 0,

"Memory": 0,

"CgroupParent": "",

"BlkioWeight": 0,

"BlkioWeightDevice": null,

"BlkioDeviceReadBps": null,

"BlkioDeviceWriteBps": null,

"BlkioDeviceReadIOps": null,

"BlkioDeviceWriteIOps": null,

"CpuPeriod": 0,

"CpuQuota": 0,

"CpusetCpus": "",

"CpusetMems": "",

"Devices": [],

"DiskQuota": 0,

"KernelMemory": 0,

"MemoryReservation": 0,

"MemorySwap": 0,

"MemorySwappiness": -1,

"OomKillDisable": false,

"PidsLimit": 0,

"Ulimits": null,

"CpuCount": 0,

"CpuPercent": 0,

"IOMaximumIOps": 0,

"IOMaximumBandwidth": 0

"GraphDriver": {

"Name": "windowsfilter",

"Data": {

"dir": "C:\\ProgramData\\docker\\windowsfilter\\c15555e4b60ec24bbad2b8834c82e14313d193e07da2ca7d3948802e36f4d828"

}

"Mounts": [],

"Config": {

"Hostname": "c15555e4b60e",

"Domainname": "",

"User": "",

"AttachStdin": true,

"AttachStdout": true,

"AttachStderr": true,

"ExposedPorts": {

"80/tcp": {}

"Tty": true,

"OpenStdin": true,

"StdinOnce": true,

"Env": null,

"Cmd": null,

"ArgsEscaped": true,

"Image": "microsoft/iis",

"Volumes": null,

"WorkingDir": "",

"Entrypoint": [

"C:\\ServiceMonitor.exe",

"w3svc"

"OnBuild": null,

"Labels": {}

"NetworkSettings": {

"Bridge": "",

"SandboxID": "b23c5766990bb9373e6cbb69cb54b011f57974ed381e60c1eacb7c3c47e303b2",

"HairpinMode": false,

"LinkLocalIPv6Address": "",

"LinkLocalIPv6PrefixLen": 0,

"Ports": {

"80/tcp": [

{

"HostIp": "0.0.0.0",

"HostPort": "80"

}

]

"SandboxKey": "b23c5766990b",

"SecondaryIPAddresses": null,

"SecondaryIPv6Addresses": null,

"EndpointID": "",

"Gateway": "",

"GlobalIPv6Address": "",

"GlobalIPv6PrefixLen": 0,

"IPAddress": "",

"IPPrefixLen": 0,

"IPv6Gateway": "",

"MacAddress": "",

"Networks": {

"nat": {

"IPAMConfig": null,

"Links": null,

"Aliases": null,

"NetworkID": "9ae352610731342dde19246e759944e79af7a19077d879024a17cedca4b061c9",

"EndpointID": "198d0a52838f1e5178b3fedc154e9f18409c13f0de489e99c06faf9ed2fbedd9",

"Gateway": "",

"IPAddress": "172.26.192.89",

"IPPrefixLen": 16,

"IPv6Gateway": "",

"GlobalIPv6Address": "",

"GlobalIPv6PrefixLen": 0,

"MacAddress": "00:15:5d:7e:60:60"

}

]

The exact IP address of the container can be found using the command:

docker inspect --format '{{ NetworkSettings.Networks.nat.IPAddress }}' cocky_mahavira

docker inspect --format '{{ NetworkSettings.Networks.nat.IPAddress }}' cocky_mahavira

IIS main page is accessible at http://<container-ip>, as shown below:

The next part will show how to create your own Docker image on Windows Server 2016.

Source: blog.couchbase.com/2016/october/docker-on-windows-2016-server

Persisting Couchbase Data Across Container Restarts

October 27, 2016containers, couchbasecouchbase, dockerarungupta

Best Practices for Virtualized Platforms provide best practices for running Couchbase on a virtualized platform like Amazon Web Services and Azure. In addition, it also provide some recommendations for running it as Docker container.

One of the recommendations is to map Couchbase node specific data to a local folder. Let’s understand that in more detail.

Implicit Per-Container Storage

If a Couchbase container is started as:

docker run -d -p 8091-8093:8091-8093 -p 11210:11210 --name db couchbase/server:sandbox

docker run -d -p 8091-8093:8091-8093 -p 11210:11210 --name db couchbase/server:sandbox

This container:

Starts in a detached mode using -d
Different query, caching and administration ports are mapped using -p
A name is provided using --name
Image is couchbase/server:sandbox

By default, the data for the container is stored in a managed volume. Checking volume mounts using the docker inspect command shows:

docker inspect --format '{{json .Mounts }}' db  | jq
[
  {
    "Name": "aa3c06f9c506d52bfb5d3d265f7b63045df0fea996998f12ce08b2543345e948",
    "Source": "/var/lib/docker/volumes/aa3c06f9c506d52bfb5d3d265f7b63045df0fea996998f12ce08b2543345e948/_data",
    "Destination": "/opt/couchbase/var",
    "Driver": "local",
    "Mode": "",
    "RW": true,
    "Propagation": ""
  }
]

docker inspect --format '{{json .Mounts }}' db | jq

[

{

"Name": "aa3c06f9c506d52bfb5d3d265f7b63045df0fea996998f12ce08b2543345e948",

"Source": "/var/lib/docker/volumes/aa3c06f9c506d52bfb5d3d265f7b63045df0fea996998f12ce08b2543345e948/_data",

"Destination": "/opt/couchbase/var",

"Driver": "local",

"Mode": "",

"RW": true,

"Propagation": ""

}

]

The data for Couchbase is stored in the container filesystem defined by the value of Source attribute. This can be verified by logging into the root filesystem:

docker run -it --pid=host --privileged debian:jessie nsenter -t 1 -m -p -n

docker run -it --pid=host --privileged debian:jessie nsenter -t 1 -m -p -n

Now you can see the data directory:

010e52853bc6:~# ls /var/lib/docker/volumes | grep aa3c
aa3c06f9c506d52bfb5d3d265f7b63045df0fea996998f12ce08b2543345e948

010e52853bc6:~# ls /var/lib/docker/volumes | grep aa3c

aa3c06f9c506d52bfb5d3d265f7b63045df0fea996998f12ce08b2543345e948

A new directory is created for a new run of the container. This directory is still around when the container is stopped and removed but no longer easily accessible. Thus no data is preserved across container restarts.

The volume can be explicitly removed, along with container, using the command:

docker rm -v db

docker rm -v db

If the container terminates then the entire state of the application is lost.

Explicit Host Directory Mapping

Now, let’s start a Couchbase container with explicit volume mapping:

docker run -d -p 8091-8093:8091-8093 -p 11210:11210 --name db -v ~/couchbase:/opt/couchbase/var couchbase/server:sandbox

docker run -d -p 8091-8093:8091-8093 -p 11210:11210 --name db -v ~/couchbase:/opt/couchbase/var couchbase/server:sandbox

This container is very similar to the container started earlier. The main difference is that a directory from host ~/couchbase is mapped to a directory in the container /opt/couchbase/var.

Couchbase container persists any data in /opt/couchbase/var directory in the container filesystem. Now that directory is mapped to a directory on the host filesystem. This allows to persist state of the container outside on the host filesystem. The bypasses the union filesystem used by Docker and exposes the host filesystem to the container. This allows the state to persist across container restarts. The new container only needs to start with the exact same volume mapping.

More details about the container can be seen as:

docker inspect --format '{{json .Mounts }}' db | jq

docker inspect --format '{{json .Mounts }}' db | jq

jq is a JSON processor that needs to be installed separately. And the output is shown as:

[
  {
    "Source": "/Users/arungupta/couchbase",
    "Destination": "/opt/couchbase/var",
    "Mode": "",
    "RW": true,
    "Propagation": "rprivate"
  }
]

[

{

"Source": "/Users/arungupta/couchbase",

"Destination": "/opt/couchbase/var",

"Mode": "",

"RW": true,

"Propagation": "rprivate"

}

]

This shows the source and destination directory. RW shows that the volume is read/write.

If the container is started using Docker for Mac, then Couchbase Web Console is accessible at http://localhost:8091. The Data Buckets tab shows the default travel-sample bucket:

Click on Create New Data Bucket to create a new data bucket. Give it the name sample:

The Data Buckets tab is updated with this newly created bucket:

Now stop and remove the container:

docker stop db
docker rm db

docker stop db

docker rm db

Start the container again using the same command:

docker run -d -p 8091-8093:8091-8093 -p 11210:11210 --name db -v ~/couchbase:/opt/couchbase/var couchbase/server:sandbox

docker run -d -p 8091-8093:8091-8093 -p 11210:11210 --name db -v ~/couchbase:/opt/couchbase/var couchbase/server:sandbox

Data Buckets tab will show the same two buckets in the Couchbase Web Console.

In this case, if the container is started on a different host then the state would not be available. Or if the host dies then the state is lost.

An alternative and a more robust and foolproof way to manage persistence in containers is using a shared network filesystem such as Ceph, GlusterFS or Network Filesystem. Some other common approaches are to use Docker Volume Plugins like Flocker from ClusterHQ or Software Defined Storage such as PortWorx. All of these storage technique simplify how state of a container can be saved in a multi-container multi-host environment. A future blog will cover these techniques in detail.

Read more details in Managing data in containers.

couchbase.com/containers provide more details about how to run Couchbase in different container frameworks.

More information about Couchbase:

Couchbase Developer Portal
Couchbase Forums
@couchbasedev or @couchbase

Source: blog.couchbase.com/2016/october/persisting-couchbase-data-across-container-restarts

Windows Server 2016 Using VirtualBox – Getting Ready for Docker

October 17, 2016containersdocker, windowsarungupta

Windows Server 2016 was announced a few weeks ago. Download Windows 2016 Server Evaluation version. This blog is the first part of a multi-part series. The first part will show how to set up Windows Server 2016 as a VirtualBox VM. This is an update to Windows Server 2016 using VirtualBox for Docker Containers.

Download Windows 2016 Server Evaluation version. Its ~5GB download so try on a reliable and fast Internet connection.
Create a new VM using Virtual Box:
Set up a dynamically allocated HDD, make sure to choose 30GB (as shown):

The installed operating system is ~17GB. The base windowsservercore image is 3.8GB. So if you chose the default 20GB HDD, then no Docker containers can run on it.
Point to the downloaded ISO:
Start the VM, take default settings:
Click on Next:

Click on Install now.
Select the Desktop Experience version otherwise the standard Windows desktop will not be available after Windows is booted:

Click on Next.
Accept the license terms:
Select Custom Install as we are installing as a VM:
Take the default for the allocated space:
Click on Next to start the installation:

Wait for a few minutes for the installation to complete:

It will take a few minutes for the install to complete.
Enter the administrator’s password:

Seems like it requires alphabets, numbers and special characters. The dialog expects to meet the password criteria without showing the criteria, this is weird!
Entering the credentials shows the Windows login screen:
In a classical Windows, you do a three-finger salute of Ctrl+Alt+Del to view Desktop. This key combination needs to be sent to Windows VM using Virtual Box menu:
Server Manager Dashboard shows up:
Latest updates need to be installed. Click on the Start icon, and search for update:
Select Check for Updates.
Update window looks like:
Select Check for updates again.
Latest updates are downloaded and installed:
Once the updates are downloaded and installed, then the VM needs to be restarted:
Click on Restart now to restart the VM.
Restarted VM looks like as shown:

This shows that Windows Server 2016 VM is now ready. Subsequent parts of this multi-part blog will show how to configure Docker containers and run a multi-container application using Docker Compose.

Source: blog.couchbase.com/2016/october/windows-server-2016-using-virtualbox-getting-ready-docker

Minikube – Rapid Dev & Testing for Kubernetes

September 30, 2016containers, couchbase, javacontainers, couchbase, kubernetes, springarungupta

One of the attendees from Kubernetes for Java Developers training suggested to try minikube for simplified Kubernetes dev and testing. This blog will show how to get started with minikube using a simple Java application.

Minikube starts a single node Kubernetes cluster on your local machine for rapid development and testing. Requirements lists the exact set of requirements for different operating systems.

This blog will show:

Start one node Kubernetes cluster
Run Couchbase service
Run Java application
View Kubernetes Dashboard

All Kubernetes resource description files used in this blog are at github.com/arun-gupta/kubernetes-java-sample/tree/master/maven.

Start Kubernetes Cluster using Minikube

Create a new directory with the name minikube.

In that directory, download kubectl CLI:

curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/v1.4.0/bin/darwin/amd64/kubectl && chmod +x kubectl

curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/v1.4.0/bin/darwin/amd64/kubectl && chmod +x kubectl

Download minikube CLI:

curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.10.0/minikube-darwin-amd64 && chmod +x minikube

curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.10.0/minikube-darwin-amd64 && chmod +x minikube

Start the cluster:

minikube start
Starting local Kubernetes cluster...
Kubectl is now configured to use the cluster.

minikube start

Starting local Kubernetes cluster...

Kubectl is now configured to use the cluster.

The list of nodes can be seen:

kubectl get nodes
NAME       STATUS    AGE
minikube   Ready     2h

kubectl get nodes

NAME STATUS AGE

minikube Ready 2h

More details about the cluster can be obtained using the kubectl cluster-info command:

kubectl cluster-info
Kubernetes master is running at https://192.168.99.100:8443
kubernetes-dashboard is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

kubectl cluster-info

Kubernetes master is running at https://192.168.99.100:8443

kubernetes-dashboard is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Behind the scenes, a Virtual Box VM is started.

Complete set of commands supported can be seen by using --help:

minikube --help
Minikube is a CLI tool that provisions and manages single-node Kubernetes clusters optimized for development workflows.

Usage:
  minikube [command]

Available Commands:
  dashboard        Opens/displays the kubernetes dashboard URL for your local cluster
  delete           Deletes a local kubernetes cluster.
  docker-env       sets up docker env variables; similar to '$(docker-machine env)'
  get-k8s-versions Gets the list of available kubernetes versions available for minikube.
  ip               Retrieve the IP address of the running cluster.
  logs             Gets the logs of the running localkube instance, used for debugging minikube, not user code.
  config           Modify minikube config
  service          Gets the kubernetes URL for the specified service in your local cluster
  ssh              Log into or run a command on a machine with SSH; similar to 'docker-machine ssh'
  start            Starts a local kubernetes cluster.
  status           Gets the status of a local kubernetes cluster.
  stop             Stops a running local kubernetes cluster.
  version          Print the version of minikube.

Flags:
      --alsologtostderr[=false]: log to standard error as well as files
      --log-flush-frequency=5s: Maximum number of seconds between log flushes
      --log_backtrace_at=:0: when logging hits line file:N, emit a stack trace
      --log_dir="": If non-empty, write log files in this directory
      --logtostderr[=false]: log to standard error instead of files
      --show-libmachine-logs[=false]: Whether or not to show logs from libmachine.
      --stderrthreshold=2: logs at or above this threshold go to stderr
      --v=0: log level for V logs
      --vmodule=: comma-separated list of pattern=N settings for file-filtered logging

Use "minikube [command] --help" for more information about a command.

minikube --help

Minikube is a CLI tool that provisions and manages single-node Kubernetes clusters optimized for development workflows.

Usage:

minikube [command]

Available Commands:

dashboard Opens/displays the kubernetes dashboard URL for your local cluster

delete Deletes a local kubernetes cluster.

docker-env sets up docker env variables; similar to '$(docker-machine env)'

get-k8s-versions Gets the list of available kubernetes versions available for minikube.

ip Retrieve the IP address of the running cluster.

logs Gets the logs of the running localkube instance, used for debugging minikube, not user code.

config Modify minikube config

service Gets the kubernetes URL for the specified service in your local cluster

ssh Log into or run a command on a machine with SSH; similar to 'docker-machine ssh'

start Starts a local kubernetes cluster.

status Gets the status of a local kubernetes cluster.

stop Stops a running local kubernetes cluster.

version Print the version of minikube.

Flags:

--alsologtostderr[=false]: log to standard error as well as files

--log-flush-frequency=5s: Maximum number of seconds between log flushes

--log_backtrace_at=:0: when logging hits line file:N, emit a stack trace

--log_dir="": If non-empty, write log files in this directory

--logtostderr[=false]: log to standard error instead of files

--show-libmachine-logs[=false]: Whether or not to show logs from libmachine.

--stderrthreshold=2: logs at or above this threshold go to stderr

--v=0: log level for V logs

--vmodule=: comma-separated list of pattern=N settings for file-filtered logging

Use "minikube [command] --help" for more information about a command.

Run Couchbase Service

Create a Couchbase service:

kubectl create -f couchbase-service.yml 
service "couchbase-service" created
replicationcontroller "couchbase-rc" created

kubectl create -f couchbase-service.yml

service "couchbase-service" created

replicationcontroller "couchbase-rc" created

This will start a Couchbase service. The service is using the pods created by the replication controller. The replication controller creates a single node Couchbase server.

The configuration file is at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/couchbase-service.yml and looks like:

apiVersion: v1
kind: Service
metadata: 
  name: couchbase-service
spec: 
  selector: 
    app: couchbase-rc-pod
  ports:
    - name: admin
      port: 8091
    - name: views
      port: 8092
    - name: query
      port: 8093
    - name: memcached
      port: 11210
---
apiVersion: v1
kind: ReplicationController
metadata:
  name: couchbase-rc
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: couchbase-rc-pod
    spec:
      containers:
      - name: couchbase
        image: arungupta/oreilly-couchbase
        ports:
        - containerPort: 8091
        - containerPort: 8092
        - containerPort: 8093
        - containerPort: 11210

apiVersion: v1

kind: Service

metadata:

spec:

selector:

app: couchbase-rc-pod

ports:

- name: admin

port: 8091

- name: views

port: 8092

- name: query

port: 8093

- name: memcached

port: 11210

---

apiVersion: v1

kind: ReplicationController

metadata:

spec:

replicas: 1

template:

metadata:

labels:

app: couchbase-rc-pod

spec:

containers:

- name: couchbase

image: arungupta/oreilly-couchbase

ports:

- containerPort: 8091

- containerPort: 8092

- containerPort: 8093

- containerPort: 11210

Run Java Application

Run the application:

kubectl create -f bootiful-couchbase.yml 
job "bootiful-couchbase" created

kubectl create -f bootiful-couchbase.yml

job "bootiful-couchbase" created

The configuration file is at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/bootiful-couchbase.yml and looks like:

apiVersion: batch/v1
kind: Job
metadata:
  name: bootiful-couchbase
  labels:
    name: bootiful-couchbase-pod
spec:
  template:
    metadata:
      name: bootiful-couchbase-pod
    spec:
      containers:
      - name: bootiful-couchbase
        image: arungupta/bootiful-couchbase
        env:
        - name: COUCHBASE_URI
          value: couchbase-service
      restartPolicy: Never

apiVersion: batch/v1

kind: Job

metadata:

labels:

spec:

template:

metadata:

spec:

containers:

- name: bootiful-couchbase

image: arungupta/bootiful-couchbase

env:

- name: COUCHBASE_URI

value: couchbase-service

restartPolicy: Never

This is run-once job which runs a Java (Spring Boot) application and upserts (insert or update) a JSON document in Couchbase.

In this job, COUCHBASE_URI environment variable value is set to couchbase-service. This is the service name created earlier. Docker image used for this service is arungupta/bootiful-couchbase and is created using fabric8-maven-plugin as shown at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/webapp/pom.xml#L57-L68. Specifically, the command for the Docker image is:

java -Dspring.couchbase.bootstrap-hosts=$COUCHBASE_URI -jar /maven/${project.artifactId}.jar

java -Dspring.couchbase.bootstrap-hosts=$COUCHBASE_URI -jar /maven/${project.artifactId}.jar

This ensures that COUCHBASE_URI environment variable is overriding spring.couchbase.bootstrap-hosts property as defined in application.properties of the Spring Boot application.

Kubernetes Dashboard

Kubernetes 1.4 included an updated dashboard. For minikube, this can be opened using the following command:

minikube dashboard
Waiting, endpoint for service is not ready yet...Opening kubernetes dashboard in default browser...

minikube dashboard

Waiting, endpoint for service is not ready yet...Opening kubernetes dashboard in default browser...

The default view is shown below:

But in our case, a few resources have already been created and so this will look like as shown:

Notice, our Jobs, Replication Controllers and Pods are shown here.

Shutdown Kubernetes Cluster

The cluster can be easily shutdown:

minikube stop
Stopping local Kubernetes cluster...
Machine stopped.

minikube stop

Stopping local Kubernetes cluster...

Machine stopped.

couchbase.com/containers provide more details about running Couchbase using different orchestration frameworks. Further references:

Couchbase Forums or StackOverflow
Follow us at @couchbasedev or @couchbase
Read more about Couchbase Server

Source: blog.couchbase.com/2016/september/minikube-rapid-dev–testing-kubernetes

Getting Started with Kubernetes 1.4 using Spring Boot and Couchbase

September 28, 2016containers, couchbase, javacontainers, couchbase, kubernetes, springarungupta

Kubernetes 1.4 was released earlier this week. Read the blog announcement and CHANGELOG. There are quite a few new features in this release but the key ones that I’m excited about are:

Install Kubernetes using kubeadm command. This is in addition to the usual mechanism of downloading from https://github.com/kubernetes/kubernetes/releases. The kubeadm init and kubeadm join commands looks very similar to docker swarm init and docker swarm join for Docker Swarm Mode.
Federated Replica Sets
ScheduledJob allows to run batch jobs at regular intervals.
Constraining pods to a node and affinity and anti-affinity of pods
Priority scheduling of pods
Nice looking Kubernetes Dashboard (more on this later)

This blog will show:

Create a Kubernetes cluster using Amazon Web Services
Create a Couchbase service
Run a Spring Boot application that stores a JSON document in Couchbase

All the resource description files in this blog are at github.com/arun-gupta/kubernetes-java-sample/tree/master/maven.

Start Kubernetes Cluster

Download binary github.com/kubernetes/kubernetes/releases/download/v1.4.0/kubernetes.tar.gz and extract

Include kubernetes/cluster in PATH

Start a 2-node Kubernetes cluster:

NUM_NODES=2 NODE_SIZE=m3.medium KUBERNETES_PROVIDER=aws kube-up.sh

NUM_NODES=2 NODE_SIZE=m3.medium KUBERNETES_PROVIDER=aws kube-up.sh

The log will be shown as:

... Starting cluster in us-west-2a using provider aws
... calling verify-prereqs
... calling kube-up
Starting cluster using os distro: jessie
Uploading to Amazon S3
+++ Staging server tars to S3 Storage: kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel
upload: ../../../../../var/folders/81/ttv4n16x7p390cttrm_675y00000gn/T/kubernetes.XXXXXX.bCmvLbtK/s3/bootstrap-script to s3://kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script
Uploaded server tars:
  SERVER_BINARY_TAR_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/kubernetes-server-linux-amd64.tar.gz
  SALT_TAR_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/kubernetes-salt.tar.gz
  BOOTSTRAP_SCRIPT_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script
INSTANCEPROFILE arn:aws:iam::598307997273:instance-profile/kubernetes-master    2016-07-29T15:13:35Z    AIPAJF3XKLNKOXOTQOCT4   kubernetes-master       /
ROLES   arn:aws:iam::598307997273:role/kubernetes-master        2016-07-29T15:13:33Z    /       AROAI3Q2KFBD5PCKRXCRM   kubernetes-master
ASSUMEROLEPOLICYDOCUMENT        2012-10-17
STATEMENT       sts:AssumeRole  Allow
PRINCIPAL       ec2.amazonaws.com
INSTANCEPROFILE arn:aws:iam::598307997273:instance-profile/kubernetes-minion    2016-07-29T15:13:39Z    AIPAIYSH5DJA4UPQIP4BE   kubernetes-minion       /
ROLES   arn:aws:iam::598307997273:role/kubernetes-minion        2016-07-29T15:13:37Z    /       AROAIQ57MPQYSHRPQCT2Q   kubernetes-minion
ASSUMEROLEPOLICYDOCUMENT        2012-10-17
STATEMENT       sts:AssumeRole  Allow
PRINCIPAL       ec2.amazonaws.com
Using SSH key with (AWS) fingerprint: SHA256:dX/5wpWuUxYar2NFuGwiZuRiydiZCyx4DGoZ5/jL/j8
Creating vpc.
Adding tag to vpc-6b5b4b0f: Name=kubernetes-vpc
Adding tag to vpc-6b5b4b0f: KubernetesCluster=kubernetes
Using VPC vpc-6b5b4b0f
Adding tag to dopt-8fe770eb: Name=kubernetes-dhcp-option-set
Adding tag to dopt-8fe770eb: KubernetesCluster=kubernetes
Using DHCP option set dopt-8fe770eb
Creating subnet.
Adding tag to subnet-623a0206: KubernetesCluster=kubernetes
Using subnet subnet-623a0206
Creating Internet Gateway.
Using Internet Gateway igw-251eab41
Associating route table.
Creating route table
Adding tag to rtb-d43cedb3: KubernetesCluster=kubernetes
Associating route table rtb-d43cedb3 to subnet subnet-623a0206
Adding route to route table rtb-d43cedb3
Using Route Table rtb-d43cedb3
Creating master security group.
Creating security group kubernetes-master-kubernetes.
Adding tag to sg-d20ca0ab: KubernetesCluster=kubernetes
Creating minion security group.
Creating security group kubernetes-minion-kubernetes.
Adding tag to sg-cd0ca0b4: KubernetesCluster=kubernetes
Using master security group: kubernetes-master-kubernetes sg-d20ca0ab
Using minion security group: kubernetes-minion-kubernetes sg-cd0ca0b4
Creating master disk: size 20GB, type gp2
Adding tag to vol-99a30b11: Name=kubernetes-master-pd
Adding tag to vol-99a30b11: KubernetesCluster=kubernetes
Allocated Elastic IP for master: 52.40.9.27
Adding tag to vol-99a30b11: kubernetes.io/master-ip=52.40.9.27
Generating certs for alternate-names: IP:52.40.9.27,IP:172.20.0.9,IP:10.0.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local,DNS:kubernetes-master
Starting Master
Adding tag to i-f95bdae1: Name=kubernetes-master
Adding tag to i-f95bdae1: Role=kubernetes-master
Adding tag to i-f95bdae1: KubernetesCluster=kubernetes
Waiting for master to be ready
Attempt 1 to check for master nodeWaiting for instance i-f95bdae1 to be running (currently pending)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be running (currently pending)
Sleeping for 3 seconds...
 [master running]
Attaching IP 52.40.9.27 to instance i-f95bdae1
Attaching persistent data volume (vol-99a30b11) to master
2016-09-29T05:14:28.098Z        /dev/sdb        i-f95bdae1      attaching       vol-99a30b11
cluster "aws_kubernetes" set.
user "aws_kubernetes" set.
context "aws_kubernetes" set.
switched to context "aws_kubernetes".
user "aws_kubernetes-basic-auth" set.
Wrote config for aws_kubernetes to /Users/arungupta/.kube/config
Creating minion configuration
Creating autoscaling group
 0 minions started; waiting
 0 minions started; waiting
 0 minions started; waiting
 0 minions started; waiting
 2 minions started; ready
Waiting for cluster initialization.

  This will continually check to see if the API for kubernetes is reachable.
  This might loop forever if there was some uncaught error during start
  up.

..............................................................................................................................................................................................................................Kubernetes cluster created.
Sanity checking cluster...
Attempt 1 to check Docker on node @ 54.70.225.33 ...working
Attempt 1 to check Docker on node @ 54.71.36.48 ...working

Kubernetes cluster is running.  The master is running at:

  https://52.40.9.27

The user name and password to use is located in /Users/arungupta/.kube/config.

... calling validate-cluster
Waiting for 2 ready nodes. 0 ready nodes, 0 registered. Retrying.
Waiting for 2 ready nodes. 0 ready nodes, 0 registered. Retrying.
Waiting for 2 ready nodes. 0 ready nodes, 0 registered. Retrying.
Waiting for 2 ready nodes. 0 ready nodes, 2 registered. Retrying.
Waiting for 2 ready nodes. 0 ready nodes, 2 registered. Retrying.
Found 2 node(s).
NAME                                         STATUS    AGE
ip-172-20-0-111.us-west-2.compute.internal   Ready     39s
ip-172-20-0-112.us-west-2.compute.internal   Ready     42s
Validate output:
NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok                   
controller-manager   Healthy   ok                   
etcd-0               Healthy   {"health": "true"}   
etcd-1               Healthy   {"health": "true"}   
Cluster validation succeeded
Done, listing cluster services:

Kubernetes master is running at https://52.40.9.27
Elasticsearch is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging
Heapster is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/heapster
Kibana is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kibana-logging
KubeDNS is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
Grafana is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana
InfluxDB is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

... Starting cluster in us-west-2a using provider aws

... calling verify-prereqs

... calling kube-up

Starting cluster using os distro: jessie

Uploading to Amazon S3

+++ Staging server tars to S3 Storage: kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel

upload: ../../../../../var/folders/81/ttv4n16x7p390cttrm_675y00000gn/T/kubernetes.XXXXXX.bCmvLbtK/s3/bootstrap-script to s3://kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script

Uploaded server tars:

SERVER_BINARY_TAR_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/kubernetes-server-linux-amd64.tar.gz

SALT_TAR_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/kubernetes-salt.tar.gz

BOOTSTRAP_SCRIPT_URL: https://s3.amazonaws.com/kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script

INSTANCEPROFILE arn:aws:iam::598307997273:instance-profile/kubernetes-master 2016-07-29T15:13:35Z AIPAJF3XKLNKOXOTQOCT4 kubernetes-master /

ROLES arn:aws:iam::598307997273:role/kubernetes-master 2016-07-29T15:13:33Z / AROAI3Q2KFBD5PCKRXCRM kubernetes-master

ASSUMEROLEPOLICYDOCUMENT 2012-10-17

STATEMENT sts:AssumeRole Allow

PRINCIPAL ec2.amazonaws.com

INSTANCEPROFILE arn:aws:iam::598307997273:instance-profile/kubernetes-minion 2016-07-29T15:13:39Z AIPAIYSH5DJA4UPQIP4BE kubernetes-minion /

ROLES arn:aws:iam::598307997273:role/kubernetes-minion 2016-07-29T15:13:37Z / AROAIQ57MPQYSHRPQCT2Q kubernetes-minion

ASSUMEROLEPOLICYDOCUMENT 2012-10-17

STATEMENT sts:AssumeRole Allow

PRINCIPAL ec2.amazonaws.com

Using SSH key with (AWS) fingerprint: SHA256:dX/5wpWuUxYar2NFuGwiZuRiydiZCyx4DGoZ5/jL/j8

Creating vpc.

Adding tag to vpc-6b5b4b0f: Name=kubernetes-vpc

Adding tag to vpc-6b5b4b0f: KubernetesCluster=kubernetes

Using VPC vpc-6b5b4b0f

Adding tag to dopt-8fe770eb: Name=kubernetes-dhcp-option-set

Adding tag to dopt-8fe770eb: KubernetesCluster=kubernetes

Using DHCP option set dopt-8fe770eb

Creating subnet.

Adding tag to subnet-623a0206: KubernetesCluster=kubernetes

Using subnet subnet-623a0206

Creating Internet Gateway.

Using Internet Gateway igw-251eab41

Associating route table.

Creating route table

Adding tag to rtb-d43cedb3: KubernetesCluster=kubernetes

Associating route table rtb-d43cedb3 to subnet subnet-623a0206

Adding route to route table rtb-d43cedb3

Using Route Table rtb-d43cedb3

Creating master security group.

Creating security group kubernetes-master-kubernetes.

Adding tag to sg-d20ca0ab: KubernetesCluster=kubernetes

Creating minion security group.

Creating security group kubernetes-minion-kubernetes.

Adding tag to sg-cd0ca0b4: KubernetesCluster=kubernetes

Using master security group: kubernetes-master-kubernetes sg-d20ca0ab

Using minion security group: kubernetes-minion-kubernetes sg-cd0ca0b4

Creating master disk: size 20GB, type gp2

Adding tag to vol-99a30b11: Name=kubernetes-master-pd

Adding tag to vol-99a30b11: KubernetesCluster=kubernetes

Allocated Elastic IP for master: 52.40.9.27

Adding tag to vol-99a30b11: kubernetes.io/master-ip=52.40.9.27

Generating certs for alternate-names: IP:52.40.9.27,IP:172.20.0.9,IP:10.0.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local,DNS:kubernetes-master

Starting Master

Adding tag to i-f95bdae1: Name=kubernetes-master

Adding tag to i-f95bdae1: Role=kubernetes-master

Adding tag to i-f95bdae1: KubernetesCluster=kubernetes

Waiting for master to be ready

Attempt 1 to check for master nodeWaiting for instance i-f95bdae1 to be running (currently pending)

Sleeping for 3 seconds...

Waiting for instance i-f95bdae1 to be running (currently pending)

Sleeping for 3 seconds...

[master running]

Attaching IP 52.40.9.27 to instance i-f95bdae1

Attaching persistent data volume (vol-99a30b11) to master

2016-09-29T05:14:28.098Z /dev/sdb i-f95bdae1 attaching vol-99a30b11

cluster "aws_kubernetes" set.

user "aws_kubernetes" set.

context "aws_kubernetes" set.

switched to context "aws_kubernetes".

user "aws_kubernetes-basic-auth" set.

Wrote config for aws_kubernetes to /Users/arungupta/.kube/config

Creating minion configuration

Creating autoscaling group

0 minions started; waiting

2 minions started; ready

Waiting for cluster initialization.

This will continually check to see if the API for kubernetes is reachable.

This might loop forever if there was some uncaught error during start

up.

..............................................................................................................................................................................................................................Kubernetes cluster created.

Sanity checking cluster...

Attempt 1 to check Docker on node @ 54.70.225.33 ...working

Attempt 1 to check Docker on node @ 54.71.36.48 ...working

Kubernetes cluster is running. The master is running at:

https://52.40.9.27

The user name and password to use is located in /Users/arungupta/.kube/config.

... calling validate-cluster

Waiting for 2 ready nodes. 0 ready nodes, 0 registered. Retrying.

Waiting for 2 ready nodes. 0 ready nodes, 2 registered. Retrying.

Found 2 node(s).

NAME STATUS AGE

ip-172-20-0-111.us-west-2.compute.internal Ready 39s

ip-172-20-0-112.us-west-2.compute.internal Ready 42s

Validate output:

NAME STATUS MESSAGE ERROR

scheduler Healthy ok

controller-manager Healthy ok

etcd-0 Healthy {"health": "true"}

etcd-1 Healthy {"health": "true"}

Cluster validation succeeded

Done, listing cluster services:

Kubernetes master is running at https://52.40.9.27

Elasticsearch is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging

Heapster is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/heapster

Kibana is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kibana-logging

KubeDNS is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kube-dns

kubernetes-dashboard is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

Grafana is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana

InfluxDB is running at https://52.40.9.27/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

This shows that the Kubernetes cluster has started successfully.

Deploy Couchbase Service

Create Couchbase service and replication controller:

kubectl.sh create -f couchbase-service.yml
service "couchbase-service" created
replicationcontroller "couchbase-rc" created

kubectl.sh create -f couchbase-service.yml

service "couchbase-service" created

replicationcontroller "couchbase-rc" created

The configuration file is at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/couchbase-service.yml.

This creates a Couchbase service and the backing replication controller. Name of the service is couchbase-service. This will be used later by the Spring Boot application to communicate with the database.

Check the status of pods:

kubectl.sh get -w pods
NAME                 READY     STATUS              RESTARTS   AGE
couchbase-rc-gu9gl   0/1       ContainerCreating   0          6s
NAME                 READY     STATUS    RESTARTS   AGE
couchbase-rc-gu9gl   1/1       Running   0          2m

kubectl.sh get -w pods

NAME READY STATUS RESTARTS AGE

couchbase-rc-gu9gl 0/1 ContainerCreating 0 6s

NAME READY STATUS RESTARTS AGE

couchbase-rc-gu9gl 1/1 Running 0 2m

Note, how the pod status changes from ContainerCreating to Running. The image is downloaded and started in the meanwhile.

Run Spring Boot Application

Run the application:

kubectl.sh create -f bootiful-couchbase.yml 
pod "bootiful-couchbase" created

kubectl.sh create -f bootiful-couchbase.yml

pod "bootiful-couchbase" created

The configuration file is at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/bootiful-couchbase.yml. In this service, COUCHBASE_URI environment variable value is set to couchbase-service. This is the service name created earlier.

Docker image used for this service is arungupta/bootiful-couchbase and is created using fabric8-maven-plugin as shown at github.com/arun-gupta/kubernetes-java-sample/blob/master/maven/webapp/pom.xml#L57-L68. Specifically, the command for the Docker image is:

java -Dspring.couchbase.bootstrap-hosts=$COUCHBASE_URI -jar /maven/${project.artifactId}.jar

java -Dspring.couchbase.bootstrap-hosts=$COUCHBASE_URI -jar /maven/${project.artifactId}.jar

This ensures that COUCHBASE_URI environment variable is overriding spring.couchbase.bootstrap-hosts property as defined in application.properties of the Spring Boot application.

Get the logs:

kubectl.sh logs -f bootiful-couchbase

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.4.0.RELEASE)

2016-09-29 05:37:29.227  INFO 5 --- [           main] org.example.webapp.Application           : Starting Application v1.0-SNAPSHOT on bootiful-couchbase with PID 5 (/maven/bootiful-couchbase.jar started by root in /)
2016-09-29 05:37:29.259  INFO 5 --- [           main] org.example.webapp.Application           : No active profile set, falling back to default profiles: default
2016-09-29 05:37:29.696  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@4ccabbaa: startup date [Thu Sep 29 05:37:29 UTC 2016]; root of context hierarchy
2016-09-29 05:37:34.375  INFO 5 --- [           main] c.c.client.core.env.CoreEnvironment      : ioPoolSize is less than 3 (1), setting to: 3
2016-09-29 05:37:34.376  INFO 5 --- [           main] c.c.client.core.env.CoreEnvironment      : computationPoolSize is less than 3 (1), setting to: 3
2016-09-29 05:37:35.026  INFO 5 --- [           main] com.couchbase.client.core.CouchbaseCore  : CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile='null', sslKeystorePassword='null', queryEnabled=false, queryPort=8093, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=3, computationPoolSize=3, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, searchServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.2.8 (git: 2.2.8, core: 1.2.9), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=1000, dcpConnectionBufferSize=20971520, dcpConnectionBufferAckThreshold=0.2, dcpConnectionName=dcp/core-io, callbacksOnIoPool=false, queryTimeout=7500, viewTimeout=7500, kvTimeout=2500, connectTimeout=5000, disconnectTimeout=25000, dnsSrvEnabled=false}
2016-09-29 05:37:36.063  INFO 5 --- [      cb-io-1-1] com.couchbase.client.core.node.Node      : Connected to Node couchbase-service
2016-09-29 05:37:36.256  INFO 5 --- [      cb-io-1-1] com.couchbase.client.core.node.Node      : Disconnected from Node couchbase-service
2016-09-29 05:37:37.727  INFO 5 --- [      cb-io-1-2] com.couchbase.client.core.node.Node      : Connected to Node couchbase-service
2016-09-29 05:37:38.316  INFO 5 --- [-computations-3] c.c.c.core.config.ConfigurationProvider  : Opened bucket books
2016-09-29 05:37:40.655  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
Book{isbn=978-1-4919-1889-0, name=Minecraft Modding with Forge, cost=29.99}
2016-09-29 05:37:41.497  INFO 5 --- [           main] org.example.webapp.Application           : Started Application in 14.64 seconds (JVM running for 16.631)
2016-09-29 05:37:41.514  INFO 5 --- [       Thread-5] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@4ccabbaa: startup date [Thu Sep 29 05:37:29 UTC 2016]; root of context hierarchy
2016-09-29 05:37:41.528  INFO 5 --- [       Thread-5] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown
2016-09-29 05:37:41.577  INFO 5 --- [      cb-io-1-2] com.couchbase.client.core.node.Node      : Disconnected from Node couchbase-service
2016-09-29 05:37:41.578  INFO 5 --- [       Thread-5] c.c.c.core.config.ConfigurationProvider  : Closed bucket books

kubectl.sh logs -f bootiful-couchbase

. ____ _ __ _ _

/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \

( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \

\\/ ___)| |_)| | | | | || (_| | ) ) ) )

' |____| .__|_| |_|_| |_\__, | / / / /

=========|_|==============|___/=/_/_/_/

:: Spring Boot :: (v1.4.0.RELEASE)

2016-09-29 05:37:29.227 INFO 5 --- [ main] org.example.webapp.Application : Starting Application v1.0-SNAPSHOT on bootiful-couchbase with PID 5 (/maven/bootiful-couchbase.jar started by root in /)

2016-09-29 05:37:29.259 INFO 5 --- [ main] org.example.webapp.Application : No active profile set, falling back to default profiles: default

2016-09-29 05:37:29.696 INFO 5 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@4ccabbaa: startup date [Thu Sep 29 05:37:29 UTC 2016]; root of context hierarchy

2016-09-29 05:37:34.375 INFO 5 --- [ main] c.c.client.core.env.CoreEnvironment : ioPoolSize is less than 3 (1), setting to: 3

2016-09-29 05:37:34.376 INFO 5 --- [ main] c.c.client.core.env.CoreEnvironment : computationPoolSize is less than 3 (1), setting to: 3

2016-09-29 05:37:35.026 INFO 5 --- [ main] com.couchbase.client.core.CouchbaseCore : CouchbaseEnvironment: {sslEnabled=false, sslKeystoreFile='null', sslKeystorePassword='null', queryEnabled=false, queryPort=8093, bootstrapHttpEnabled=true, bootstrapCarrierEnabled=true, bootstrapHttpDirectPort=8091, bootstrapHttpSslPort=18091, bootstrapCarrierDirectPort=11210, bootstrapCarrierSslPort=11207, ioPoolSize=3, computationPoolSize=3, responseBufferSize=16384, requestBufferSize=16384, kvServiceEndpoints=1, viewServiceEndpoints=1, queryServiceEndpoints=1, searchServiceEndpoints=1, ioPool=NioEventLoopGroup, coreScheduler=CoreScheduler, eventBus=DefaultEventBus, packageNameAndVersion=couchbase-java-client/2.2.8 (git: 2.2.8, core: 1.2.9), dcpEnabled=false, retryStrategy=BestEffort, maxRequestLifetime=75000, retryDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=100, upper=100000}, reconnectDelay=ExponentialDelay{growBy 1.0 MILLISECONDS, powers of 2; lower=32, upper=4096}, observeIntervalDelay=ExponentialDelay{growBy 1.0 MICROSECONDS, powers of 2; lower=10, upper=100000}, keepAliveInterval=30000, autoreleaseAfter=2000, bufferPoolingEnabled=true, tcpNodelayEnabled=true, mutationTokensEnabled=false, socketConnectTimeout=1000, dcpConnectionBufferSize=20971520, dcpConnectionBufferAckThreshold=0.2, dcpConnectionName=dcp/core-io, callbacksOnIoPool=false, queryTimeout=7500, viewTimeout=7500, kvTimeout=2500, connectTimeout=5000, disconnectTimeout=25000, dnsSrvEnabled=false}

2016-09-29 05:37:36.063 INFO 5 --- [ cb-io-1-1] com.couchbase.client.core.node.Node : Connected to Node couchbase-service

2016-09-29 05:37:36.256 INFO 5 --- [ cb-io-1-1] com.couchbase.client.core.node.Node : Disconnected from Node couchbase-service

2016-09-29 05:37:37.727 INFO 5 --- [ cb-io-1-2] com.couchbase.client.core.node.Node : Connected to Node couchbase-service

2016-09-29 05:37:38.316 INFO 5 --- [-computations-3] c.c.c.core.config.ConfigurationProvider : Opened bucket books

2016-09-29 05:37:40.655 INFO 5 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup

Book{isbn=978-1-4919-1889-0, name=Minecraft Modding with Forge, cost=29.99}

2016-09-29 05:37:41.497 INFO 5 --- [ main] org.example.webapp.Application : Started Application in 14.64 seconds (JVM running for 16.631)

2016-09-29 05:37:41.514 INFO 5 --- [ Thread-5] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@4ccabbaa: startup date [Thu Sep 29 05:37:29 UTC 2016]; root of context hierarchy

2016-09-29 05:37:41.528 INFO 5 --- [ Thread-5] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown

2016-09-29 05:37:41.577 INFO 5 --- [ cb-io-1-2] com.couchbase.client.core.node.Node : Disconnected from Node couchbase-service

2016-09-29 05:37:41.578 INFO 5 --- [ Thread-5] c.c.c.core.config.ConfigurationProvider : Closed bucket books

The main output statement to look in this is

Book{isbn=978-1-4919-1889-0, name=Minecraft Modding with Forge, cost=29.99}

Book{isbn=978-1-4919-1889-0, name=Minecraft Modding with Forge, cost=29.99}

This indicates that the JSON document is upserted (either inserted or updated) in the Couchbase database.

Kubernetes Dashboard

Kubernetes Dashboard is look more comprehensive and claimed to have 90% parity with the CLI. Use kubectl.sh config view command to view the configuration information about the cluster. It looks like:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: REDACTED
    server: https://52.40.9.27
  name: aws_kubernetes
contexts:
- context:
    cluster: aws_kubernetes
    user: aws_kubernetes
  name: aws_kubernetes
current-context: aws_kubernetes
kind: Config
preferences: {}
users:
- name: aws_kubernetes
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
    token: 3GuTCLvFnINHed9dWICICidlrSv8C0kg
- name: aws_kubernetes-basic-auth
  user:
    password: 8pxC121Oj7kN0nCa
    username: admin

apiVersion: v1

clusters:

- cluster:

certificate-authority-data: REDACTED

server: https://52.40.9.27

contexts:

- context:

cluster: aws_kubernetes

user: aws_kubernetes

current-context: aws_kubernetes

kind: Config

preferences: {}

users:

- name: aws_kubernetes

user:

client-certificate-data: REDACTED

client-key-data: REDACTED

token: 3GuTCLvFnINHed9dWICICidlrSv8C0kg

- name: aws_kubernetes-basic-auth

user:

password: 8pxC121Oj7kN0nCa

username: admin

The clusters.cluster.server property value shows the location of Kubernetes master. The users property show two users that can be used to access the dashboard. Second one uses basic authentication and so copy the username and password property value. In our case, Dashboard UI is accessible at https://52.40.9.27/ui.

All the Kubernetes resources can be easily seen in this fancy dashboard.

Shutdown Kubernetes Cluster

Finally, shutdown the Kubernetes cluster:

kube-down.sh
Bringing down cluster using provider: aws
Deleting instances in VPC: vpc-6b5b4b0f
Deleting auto-scaling group: kubernetes-minion-group-us-west-2a
Deleting auto-scaling launch configuration: kubernetes-minion-group-us-west-2a
Deleting auto-scaling group: kubernetes-minion-group-us-west-2a
Waiting for instances to be deleted
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-f95bdae1 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
All instances deleted
Releasing Elastic IP: 52.40.9.27
Deleting volume vol-99a30b11
Cleaning up resources in VPC: vpc-6b5b4b0f
Cleaning up security group: sg-cd0ca0b4
Cleaning up security group: sg-d20ca0ab
Deleting security group: sg-cd0ca0b4
Deleting security group: sg-d20ca0ab
Deleting VPC: vpc-6b5b4b0f
Done

kube-down.sh

Bringing down cluster using provider: aws

Deleting instances in VPC: vpc-6b5b4b0f

Deleting auto-scaling group: kubernetes-minion-group-us-west-2a

Deleting auto-scaling launch configuration: kubernetes-minion-group-us-west-2a

Deleting auto-scaling group: kubernetes-minion-group-us-west-2a

Waiting for instances to be deleted