Tag Archives: rest

Microservice using AWS Serverless Application Model and Couchbase

Amazon Web Services introduced Serverless Application Model, or SAM, a couple of months ago. It defines simplified syntax for expressing serverless resources. SAM extends AWS CloudFormation to add support for API Gateway, AWS Lambda and Amazon DynamoDB. This blog will show how to create a simple microservice using SAM. Of course, we’ll use Couchbase instead of DynamoDB!

This blog will also use the basic concepts explained in Microservice using AWS API Gateway, AWS Lambda and Couchbase. SAM will show the ease with which the entire stack for microservice can be deployed and managed.

As a refresher, here are key components in the architecture:

serverless-microservice

  • Client could be curl, AWS CLI/Console, Postman client or any other tool/API that can invoke a REST endpoint.
  • AWS API Gateway is used to provision APIs. The top level resource is available at path /books. HTTP GET and POST methods are published for the resource.
  • Each API triggers a Lambda function. Two Lambda functions are created, book-list function for listing all the books available and book-create function to create a new book.
  • Couchbase is used as a persistence store in EC2. All the JSON documents are stored and retrieved from this database.

Other blogs on serverless:

Let’s get started!

Serverless Application Model (SAM) Template

An AWS CloudFormation template with serverless resources conforming to the AWS SAM model is referred to as a SAM file or template. It is deployed as a CloudFormation stack.

Let’s take a look at our SAM template:

This template is available at github.com/arun-gupta/serverless/blob/master/aws/microservice/template.yml.

SAM template Specification provide complete details about contents in the template. The key parts of the template are:

  • Defines two resources, both of Lambda Function type identified by AWS::Serverless::Function attribute. Name of the Lambda function is defined by Resources.<resource>.
  • Class for each handler is defined by the value of Resources.<resource>.Properties.Handler attribute
  • Java 8 runtime is used to run the Function defined by Resources.<resource>.Properties.Runtime attribute
  • Code for the class is uploaded to an S3 bucket, in our case to s3://serverless-microservice/microservice-http-endpoint-1.0-SNAPSHOT.jar
  • Resources.<resource>.Properties.Environment.Variables.COUCHBASE_HOST attribute value defines the host where Couchbase is running. This can be easily deployed on EC2 as explained at Setup Couchbase.
  • Each Lambda function is triggered by an API. It is deployed using AWS API Gateway. The path is defined by Events.GetResource.Properties.Path. HTTP method is defined using Events.GetResource.Properties.Method attribute.

Java Application

The Java application that contains the Lambda functions is at github.com/arun-gupta/serverless/tree/master/aws/microservice/microservice-http-endpoint.

Lambda function that is triggered by HTTP GET method is shown:

A little bit of explanation:

  • Each Lambda function needs to implement the interface com.amazonaws.services.lambda.runtime.RequestHandler.
  • API Gateway and Lambda integration require a specific input format and output format. These formats are defined as GatewayRequest and GatewayResponse classes.
  • Function logic uses Couchbase Java SDK to query the Couchbase database. N1QL query is used to query the database. The results and exception are then wrapped in GatewayRequest and GatewayResponse.

Lambda function triggered by HTTP POST method is pretty straightforward as well:

A bit of explanation:

  • Incoming request payload is retrieved from GatewayRequest
  • Document inserted in Couchbase is returned as response.
  • Like the previous method, Function logic uses Couchbase Java SDK to query the Couchbase database. The results and exception are then wrapped in GatewayRequest and GatewayResponse.

Build the Java application as:

Upload Lambda Function to S3

SAM template reads the code from an S3 bucket. Let’s create a S3 bucket:

us-west-2 region is one of the supported regions for API Gateway. S3 bucket names are globally unique but their location is region specific.

Upload the code to S3 bucket:

The code is now uploaded to S3 bucket. SAM template is ready to be deployed!

Deploy SAM Template

Deploy the SAM template:

It shows the output:

This one command deploys Lambda functions and REST Resource/APIs that trigger these Lambda functions.

Invoke the Microservice

API Gateway publishes a REST API that can be invoked by curl, wget, AWS CLI/Console, Postman or any other app that can call a REST API. This blog will use AWS Console to show the interaction.

API Gateway home at us-west-2.console.aws.amazon.com/apigateway/home?region=us-west-2#/apis shows:

AWS SAM Microservice API

Click on the API to see all the APIs in this resource:

AWS SAM Microservice API Resources

Click on POST to see the default page for POST method execution:

AWS SAM Microservice API POST

Click on Test to test the API:

AWS SAM Microservice API POST Input

Add the payload in Request Body and click on Test to invoke the API. The results are shown as below:

AWS SAM Microservice API POST Output

Now click on GET to see the default execution page:

AWS SAM Microservice API GET

Click on Test to test the API:

AWS SAM Microservice API GET Input

No request body is needed, just click on Test the invoke the API. The results are as shown:

AWS SAM Microservice API GET Output

Output from the Couchbase database is shown in the Response Body.

References

Source: blog.couchbase.com/2017/january/microservice-aws-serverless-application-model-couchbase

Docker Networking with Couchbase and WildFly

Docker Multi-Host networking allows you to create virtual networks and attach containers to them so you can create the network topology that is right for your application. This blog will show how to use it with Docker Compose.

CRUD Java Application with Couchbase, Java EE, and WildFly explained how to use a Java EE application to provide a CRUD/REST interface on a data bucket in Couchbase. It required to manually download and run WildFly. The blog also used Couchbase server using Docker and required manual configuration to load travel-sample bucket.

Configure Couchbase Docker Container using REST API explained how to use Couchbase REST API to configure Couchbase Server.

Docker Multi-Host Networking

This blog will remove the explicit download of WildFly and manual configuration of Couchbase server:

  • Use Docker Compose to start WildFly and Couchbase (no download required)
  • Use a Maven profile to configure Couchbase server (no manual configuration required)
  • Uses Docker multi-host networking so that WildFly and Couchbase server can talk to each other

Lets get started!

Start Couchbase and WildFly using Docker Multi-Host Networking and Compose

  1. Start WildFly and Couchbase server using docker-compose.yml file from github.com/arun-gupta/docker-images/blob/master/wildfly-couchbase-javaee7/docker-compose.yml:
    arungupta/wildfly-admin image is used as it binds WildFly’s management to all network interfaces, and in addition also exposes port 9990. This enables WildFly Maven Plugin to be used to deploy the application.

    container_name is specified for Couchbase service and referred in WildFly service using COUCHBASE_URI. This is then used to connect to Couchbase from the Java EE application.

    The application environment is started as:

    --x-networking is an experimental switch added to Docker Compose 1.9 that allows to create a bridge or an overlay network. By default, it creates a bridge network that works on a single host. The network created can be seen as:

    Issue 2221 provide more explanation about the default networks created. wildflycouchbasejavaee7 is the new bridge network created for our application.  Issue #2345 provide some details about incorrect driver name in the output message.

Configure Couchbase Server

  1. Clone couchbase-javaee repo:

  2. Configure Couchbase server:

    exec-maven-plugin is used to invoke REST API and configure Couchbase server and is configured in a Maven profile. Make sure to setup docker.host property in pom.xml.

  3. Deploy the application to WildFly:

    Make sure to specify the correct host on CLI. In this case, this is the IP address obtained using docker-machine ip default.

Invoke the Application

  1. Invoke the REST endpoint using cURL:

    Complete set of REST endpoints are documented at CRUD Java Application with Couchbase, Java EE and WildFly. They are listed here for convenience:

    1. GET a single airline:
    2. Create a new airline using POST:

    3. Update an existing airline using PUT:
    4. Delete an existing airline using DELETE:

Enjoy!

REST vs WebSocket Comparison and Benchmarks

One of the common questions asked during my #JavaEE7 presentations around the world is how do WebSockets compare with REST ?

First of all, REST is a style of architecture so what really people mean is RESTful HTTP. As an architecture cannot be compared with a technology. But the term is so loosely used that they are used in place of each other commonly.

Lets start with a one line definition for WebSocket …

Bi-directional and full-duplex communication channel over a single TCP connection.

WebSocket solves a few issues with REST, or HTTP in general:

  • Bi-directional: HTTP is a uni-directional protocol where a request is always initiated by client, server processes and returns a response, and then the client consumes it. WebSocket is a bi-directional protocol where there are no pre-defined message patterns such as request/response. Either client or server can send a message to the other party.
  • Full-duplex: HTTP allows the request message to go from client to server and then server sends a response message to the client. At a given time, either client is talking to server or server is talking to client. WebSocket allows client and server to talk independent of each other.
  • Single TCP Connection: Typically a new TCP connection is initiated for a HTTP request and terminated after the response is received. A new TCP connection need to be established for another HTTP request/response. For WebSocket, the HTTP connection is upgraded using standard HTTP Upgrade mechanism and client and server communicate over that same TCP connection for the lifecycle of WebSocket connection.
  • Lean protocol: HTTP is a chatty protocol. Here is the set of HTTP headers sent in request message by Advanced REST Client Chrome extension.
    And the response headers received from WildFly 8:
    These are 663 characters exchanged for a trivial “Hello World” echo. The source code for this simple application is here.

    For WebSocket, after the initial HTTP handshake, the data is minimally framed with 2 bytes.

Lets take a look at a micro benchmark that shows the overhead caused by REST over a WebSocket echo endpoint. The payload is just a simple text array populated with ‘x’. The source code for the benchmark is available here.

The first graph shows the time (in milliseconds) taken to process N messages for a constant payload size.

websocket-rest-messages

Here is the raw data that feeds this graph:

websocket-rest-constant-payload

This graph and the table shows that the REST overhead increases with the number of messages. This is true because that many TCP connections need to be initiated and terminated and that many HTTP headers need to be sent and received. The last column particularly shows the multiplication factor for the amount of time to fulfill a REST request.

The second graph shows the time taken to process a fixed number of messages by varying the payload size.

websocket-rest-payload

Here is the raw data that feeds this graph:

websocket-rest-constant-messages

This graph shows that the incremental cost of processing the request/response for a REST endpoint is minimal and most of the time is spent in connection initiation/termination and honoring HTTP semantics.

These benchmarks were generated on WildFly 8 and the source code for the benchmark is available here.

Together the graph also shows that WebSocket is a more efficient protocol than RESTful HTTP. But does that mean it will replace RESTful HTTP ?

The answer to that, at least in the short term is, NO!

  • WebSocket is a low-level protocol, think of it as a socket on the web. Every thing, including a simple request/response design pattern, how to create/update/delete resources need, status codes etc to be build on top of it. All of these are well defined for HTTP.
  • WebSocket is a stateful protocol where as HTTP is a stateless protocol. WebSocket connections are know to scale vertically on a single server where as HTTP can scale horizontally. There are some proprietary solutions for WebSocket horizontal scaling, but they are not standards-based.
  • HTTP comes with a lot of other goodies such as caching, routing, multiplexing, gzipping and lot more. All of these need to be defined on top of WebSocket.
  • How will Search Engine Optimization (SEO) work with WebSocket ? Works very well for HTTP URLs.
  • All proxy, DNS, firewalls are not yet fully aware of WebSocket traffic. They allow port 80 but might restrict traffic by snooping on it first.
  • Security with WebSocket is all-or-nothing approach.

This blog does not provide any conclusion because its meant to trigger thoughts!

And if you want a complete introduction to JSR 356 WebSocket API in Java EE 7, then watch a recently concluded webinar at vJUG:

So, what do you think ?