Miles to go …

nginx (pronounced as “engine-ex”) is an open-source and high-performance HTTP server. It provides the common features such as reverse proxying with caching, load balancing, modular architecture using filters (gzipping, chunked responses, etc), virtual servers, flexible configuration and much more.

nginx is known for it’s high performance and low resource consumption. It’s a fairly popular front-end HTTP server in the Rails community along with Apache, Lighttpd, and others. This TOTD (Tip Of The Day) will show how to install/configure nginx for load-balancing/front-ending a cluster of Rails application running on GlassFish Gem.

1. Download, build, and install nginx using the simple script (borrowed from dzone):

   ~/tools > curl -L -O http://sysoev.ru/nginx/nginx-0.6.36.tar.gz ~/tools > tar -xzf nginx-0.6.36.tar.gz ~/tools > curl -L -O http://downloads.sourceforge.net/pcre/pcre-7.7.tar.gz ~/tools > tar -xzf pcre-7.7.tar.gz ~/tools/nginx-0.6.36 > ./configure –prefix=/usr/local/nginx –sbin-path=/usr/sbin –with-debug –with-http_ssl_module –with-pcre=../pcre-7.7 ~/tools/nginx-0.6.36 > make ~/tools/nginx-0.6.36 > sudo make install ~/tools/nginx-0.6.36 > which nginx /usr/sbin/nginx

   OK, nginx is now roaring and can be verified by visiting “http://localhost” as shown below:

   

2. Create a simple Rails scaffold as:
   

   ~/samples/jruby >~/tools/jruby/bin/jruby -S rails runner ~/samples/jruby/runner >~/tools/jruby/bin/jruby script/generate scaffold runlog miles:float minutes:integer ~/samples/jruby/runner >sed s/’adapter: sqlite3′/’adapter: jdbcsqlite3′/ <config/database.yml >config/database.yml.new ~/samples/jruby/runner >mv config/database.yml.new config/database.yml ~/samples/jruby/runner >~/tools/jruby/bin/jruby -S rake db:migrate

3. Run this application using GlassFish Gem on 3 separate ports as:
   

   ~/samples/jruby/runner >~/tools/jruby/bin/jruby -S glassfish Starting GlassFish server at: 192.168.1.145:3000 in development environment… Writing log messages to: /Users/arungupta/samples/jruby/runner/log/development.log. Press Ctrl+C to stop.

   The default port is 3000. Start the seond one by explicitly specifying the port using “-p” option ..

   ~/samples/jruby/runner >~/tools/jruby/bin/jruby -S glassfish -p 3001 Starting GlassFish server at: 192.168.1.145:3001 in development environment… Writing log messages to: /Users/arungupta/samples/jruby/runner/log/development.log. Press Ctrl+C to stop.

   and the last one on 3002 port …

   ~/samples/jruby/runner >~/tools/jruby/bin/jruby -S glassfish -p 3002 Starting GlassFish server at: 192.168.1.145:3002 in development environment… Writing log messages to: /Users/arungupta/samples/jruby/runner/log/development.log. Press Ctrl+C to stop.

   On Solaris and Linux, you can run GlassFish as a daemon as well.

4. Nginx currently uses a simple round-robin algorithm. Other load balancers such as nginx-upstream-fair (fair proxy) and nginx-ey-balancer (maximum connections) are also available. The built-in algorithm will be used for this blog. Edit “/usr/local/nginx/conf/nginx.conf” to specify an upstream module which provides load balancing:
   1. Create a cluster definition by adding an upstream module (configuration details) right before the “server” module:

      upstream glassfish {         server 127.0.0.1:3000;         server 127.0.0.1:3001;         server 127.0.0.1:3002;     }

      The cluster specifies a bunch of GlassFish Gem instances running at the backend. Each server can be weighted differently as explained here. The port numbers must exactly match as those specified at the start up. The modified “nginx.conf” looks like:

      

      The changes are highlighted on lines #35 through #39.

   2. Configure load balancing by specifying this cluster using “proxy_pass” directive as shown below:
      

      proxy_pass http://glassfish;

      in the “location” module. The updated “nginx.conf” looks like:

      

      The change is highlighted on line #52.

5. Restart nginx by using the following commands:
   

   su do kill -15 `cat /usr/local/nginx/logs/nginx.pid` sudo nginx

Now “http://localhost” shows the default Rails page as shown below:

“http://localhost/runlogs” now serves the page from the deployed Rails application.

Now lets configure logging so that the upstream server IP address and port are printed in the log files. In “nginx.conf”, uncomment “log_format” directive and add “$upstream_addr” variable as shown:

log_format  main  ‘$remote_addr – [$upstream_addr] $remote_user [$time_local] $request ‘                       ‘”$status” $body_bytes_sent “$http_referer” ‘                       ‘”$http_user_agent” “$http_x_forwarded_for”‘;     access_log  logs/access.log  main;

Also change the log format to “main” by uncommenting “access_log logs/access.log main;” line as shown above (default format is “combined”). Accessing “http://localhost/runlogs” shows the following lines in “logs/access.log”:

127.0.0.1 – [127.0.0.1:3000] – [29/Apr/2009:15:27:57 -0700] GET /runlogs/ HTTP/1.1 “200″ 3689 “-” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-” 127.0.0.1 – [127.0.0.1:3001] – [29/Apr/2009:15:27:57 -0700] GET /favicon.ico HTTP/1.1 “200″ 0 “http://localhost/runlogs/” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-” 127.0.0.1 – [127.0.0.1:3002] – [29/Apr/2009:15:27:57 -0700] GET /stylesheets/scaffold.css?1240977992 HTTP/1.1 “200″ 889 “http://localhost/runlogs/” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-”

The browser makes multiple requests (3 in this case) to load resources on a page and they are nicely load-balanced on the cluster. If an instance running on port 3002 is killed, then the access log show the entries like:

127.0.0.1 – [127.0.0.1:3000] – [29/Apr/2009:15:28:53 -0700] GET /runlogs/ HTTP/1.1 “200″ 3689 “-” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-” 127.0.0.1 – [127.0.0.1:3002, 127.0.0.1:3000] – [29/Apr/2009:15:28:53 -0700] GET /favicon.ico HTTP/1.1 “200″ 0 “http://localhost/runlogs/” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-” 127.0.0.1 – [127.0.0.1:3001] – [29/Apr/2009:15:28:53 -0700] GET /stylesheets/scaffold.css?1240977992 HTTP/1.1 “200″ 889 “http://localhost/runlogs/” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-”

The second log line shows that server running on port 3002 did not respond and so it automatically fall back to 3000, this is nice!

But this is inefficient because a back-end trip is made even for serving a static file (“/favicon.ico” and “/stylesheets/scaffold.css?1240977992″). This can be easily solved by enabling Rails page caching as described here and here.

More options about logging are described in NginxHttpLogModule and upstream module variables are defined in NginxHttpUpstreamModule.

Here are some nginx resources:

• nginx Website
• nginx Forum (very useful)
• nginx Wiki
• IRC #nginx

Are you using nginx to front-end your GlassFish cluster ?

Apache + JRuby + Rails + GlassFish = Easy Deployment! shows similar steps if you want to front-end your Rails application running using JRuby/GlassFish with Apache.

Hear all about it in Develop with Pleasure, Deploy with Fun: GlassFish and NetBeans for a Better Rails Experience session at Rails Conf next week.

Please leave suggestions on other TOTD (Tip Of The Day) that you’d like to see. A complete archive of all tips is available here.

Technorati: rubyonrails glassfish v3 gem jruby nginx loadbalancing clustering