Miles to go …

April 30, 2009

TOTD #81: How to use nginx to load balance a cluster of GlassFish Gem ?

Filed under: web2.0 — arungupta @ 4:00 am
nginx (pronounced as “engine-ex”) is an open-source and high-performance HTTP server. It provides the common features such as reverse proxying with caching, load balancing, modular architecture using filters (gzipping, chunked responses, etc), virtual servers, flexible configuration and much more.

nginx is known for it’s high performance and low resource consumption. It’s a fairly popular front-end HTTP server in the Rails community along with Apache, Lighttpd, and others. This TOTD (Tip Of The Day) will show how to install/configure nginx for load-balancing/front-ending a cluster of Rails application running on GlassFish Gem.

  1. Download, build, and install nginx using the simple script (borrowed from dzone):

    ~/tools > curl -L -O http://sysoev.ru/nginx/nginx-0.6.36.tar.gz
    ~/tools > tar -xzf nginx-0.6.36.tar.gz
    ~/tools > curl -L -O http://downloads.sourceforge.net/pcre/pcre-7.7.tar.gz
    ~/tools > tar -xzf pcre-7.7.tar.gz
    ~/tools/nginx-0.6.36 > ./configure –prefix=/usr/local/nginx –sbin-path=/usr/sbin –with-debug –with-http_ssl_module –with-pcre=../pcre-7.7
    ~/tools/nginx-0.6.36 > make
    ~/tools/nginx-0.6.36 > sudo make install
    ~/tools/nginx-0.6.36 > which nginx
    /usr/sbin/nginx

    OK, nginx is now roaring and can be verified by visiting “http://localhost” as shown below:

  2. Create a simple Rails scaffold as:
    ~/samples/jruby >~/tools/jruby/bin/jruby -S rails runner
    ~/samples/jruby/runner >~/tools/jruby/bin/jruby script/generate scaffold runlog miles:float minutes:integer
    ~/samples/jruby/runner >sed s/’adapter: sqlite3′/’adapter: jdbcsqlite3′/ <config/database.yml >config/database.yml.new
    ~/samples/jruby/runner >mv config/database.yml.new config/database.yml
    ~/samples/jruby/runner >~/tools/jruby/bin/jruby -S rake db:migrate
  3. Run this application using GlassFish Gem on 3 separate ports as:
    ~/samples/jruby/runner >~/tools/jruby/bin/jruby -S glassfish
    Starting GlassFish server at: 192.168.1.145:3000 in development environment…
    Writing log messages to: /Users/arungupta/samples/jruby/runner/log/development.log.
    Press Ctrl+C to stop.

    The default port is 3000. Start the seond one by explicitly specifying the port using “-p” option ..

    ~/samples/jruby/runner >~/tools/jruby/bin/jruby -S glassfish -p 3001
    Starting GlassFish server at: 192.168.1.145:3001 in development environment…
    Writing log messages to: /Users/arungupta/samples/jruby/runner/log/development.log.
    Press Ctrl+C to stop.

    and the last one on 3002 port …

    ~/samples/jruby/runner >~/tools/jruby/bin/jruby -S glassfish -p 3002
    Starting GlassFish server at: 192.168.1.145:3002 in development environment…
    Writing log messages to: /Users/arungupta/samples/jruby/runner/log/development.log.
    Press Ctrl+C to stop.

    On Solaris and Linux, you can run GlassFish as a daemon as well.

  4. Nginx currently uses a simple round-robin algorithm. Other load balancers such as nginx-upstream-fair (fair proxy) and nginx-ey-balancer (maximum connections) are also available. The built-in algorithm will be used for this blog. Edit “/usr/local/nginx/conf/nginx.conf” to specify an upstream module which provides load balancing:
    1. Create a cluster definition by adding an upstream module (configuration details) right before the “server” module:

      upstream glassfish {
              server 127.0.0.1:3000;
              server 127.0.0.1:3001;
              server 127.0.0.1:3002;
          }

      The cluster specifies a bunch of GlassFish Gem instances running at the backend. Each server can be weighted differently as explained here. The port numbers must exactly match as those specified at the start up. The modified “nginx.conf” looks like:

      The changes are highlighted on lines #35 through #39.

    2. Configure load balancing by specifying this cluster using “proxy_pass” directive as shown below:
      proxy_pass http://glassfish;

      in the “location” module. The updated “nginx.conf” looks like:

      The change is highlighted on line #52.

  5. Restart nginx by using the following commands:
    sudo kill -15 `cat /usr/local/ngin
    x/logs/nginx.pid`
    sudo nginx

    Now “http://localhost” shows the default Rails page as shown below:

    “http://localhost/runlogs” now serves the page from the deployed Rails application.

    Now lets configure logging so that the upstream server IP address and port are printed in the log files. In “nginx.conf”, uncomment “log_format” directive and add “$upstream_addr” variable as shown:

        log_format  main  ‘$remote_addr – [$upstream_addr] $remote_user [$time_local] $request ‘
                          ‘”$status” $body_bytes_sent “$http_referer” ‘
                          ‘”$http_user_agent” “$http_x_forwarded_for”‘;

        access_log  logs/access.log  main;

    Also change the log format to “main” by uncommenting “access_log logs/access.log main;” line as shown above (default format is “combined”). Accessing “http://localhost/runlogs” shows the following lines in “logs/access.log”:

    127.0.0.1 – [127.0.0.1:3000] – [29/Apr/2009:15:27:57 -0700] GET /runlogs/ HTTP/1.1 “200″ 3689 “-” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-”
    127.0.0.1 – [127.0.0.1:3001] – [29/Apr/2009:15:27:57 -0700] GET /favicon.ico HTTP/1.1 “200″ 0 “http://localhost/runlogs/” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-”
    127.0.0.1 – [127.0.0.1:3002] – [29/Apr/2009:15:27:57 -0700] GET /stylesheets/scaffold.css?1240977992 HTTP/1.1 “200″ 889 “http://localhost/runlogs/” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-”

    The browser makes multiple requests (3 in this case) to load resources on a page and they are nicely load-balanced on the cluster. If an instance running on port 3002 is killed, then the access log show the entries like:

    127.0.0.1 – [127.0.0.1:3000] – [29/Apr/2009:15:28:53 -0700] GET /runlogs/ HTTP/1.1 “200″ 3689 “-” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-”
    127.0.0.1 – [127.0.0.1:3002, 127.0.0.1:3000] – [29/Apr/2009:15:28:53 -0700] GET /favicon.ico HTTP/1.1 “200″ 0 “http://localhost/runlogs/” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-”
    127.0.0.1 – [127.0.0.1:3001] – [29/Apr/2009:15:28:53 -0700] GET /stylesheets/scaffold.css?1240977992 HTTP/1.1 “200″ 889 “http://localhost/runlogs/” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″ “-”

    The second log line shows that server running on port 3002 did not respond and so it automatically fall back to 3000, this is nice!

    But this is inefficient because a back-end trip is made even for serving a static file (“/favicon.ico” and “/stylesheets/scaffold.css?1240977992″). This can be easily solved by enabling Rails page caching as described here and here.

    More options about logging are described in NginxHttpLogModule and upstream module variables are defined in NginxHttpUpstreamModule.

    Here are some nginx resources:

    • nginx Website
    • nginx Forum (very useful)
    • nginx Wiki
    • IRC #nginx

    Are you using nginx to front-end your GlassFish cluster ?

    Apache + JRuby + Rails + GlassFish = Easy Deployment! shows similar steps if you want to front-end your Rails application running using JRuby/GlassFish with Apache.

    Hear all about it in Develop with Pleasure, Deploy with Fun: GlassFish and NetBeans for a Better Rails Experience session at Rails Conf next week.

    Please leave suggestions on other TOTD (Tip Of The Day) that you’d like to see. A complete archive of all tips is available here.

    Technorati: rubyonrails glassfish v3 gem jruby nginx loadbalancing clustering

    Share and Enjoy:
    • Print
    • Digg
    • Sphinn
    • del.icio.us
    • Facebook
    • Google Bookmarks
    • DZone
    • StumbleUpon
    • Technorati
    • Twitter
    • Slashdot

    April 29, 2009

    TOTD #80: Sinatra CRUD application using Haml templates with JRuby and GlassFish Gem

    Filed under: web2.0 — arungupta @ 2:00 am

    TOTD #79 showed how to run a trivial Sinatra application using GlassFish Gem. Sinatra provides support for Haml, Erb, Builder, Sass, and Inline templates as described here. This TOTD will show how to get started with creating a Sinatra CRUD application using Haml templates.

    Haml is based on one primary principle – Markup should be beautiful because beauty makes you faster.

    Get started by installing the Haml gem as:

    /tools/jruby-1.2.0 >./bin/jruby -S gem install haml –no-ri –no-rdoc
    JRuby limited openssl loaded. gem install jruby-openssl for full support.
    http://wiki.jruby.org/wiki/JRuby_Builtin_OpenSSL
    Successfully installed haml-2.0.9
    1 gem installed

    And follow the tutorial, documentation, and reference page for more details.

    Sinatra is ORM-agnostic and so any Ruby ORM framework such as ActiveRecord, DataMapper, Sequel, and others. DataMapper with JRuby requires work so this TOTD will show how to use ActiveRecord instead. There is sinatras-hat which allows to create RESTy CRUD apps with Sinatra. There probably are mutiple other ways to create this application but I prefer to understanding the wiring so this blog will use a bare minimal structure.

    Anyway, lets get started!

    1. Create the database as:

      ~/tools/jruby/samples/sinatra-sample >mysql –user root
      Welcome to the MySQL monitor.  Commands end with ; or \g.
      Your MySQL connection id is 664
      Server version: 5.1.30 MySQL Community Server (GPL)

      Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the buffer.

      mysql> create database hello_development;
      Query OK, 1 row affected (0.00 sec)

      mysql> use hello_development;
      Database changed
      mysql> CREATE TABLE `runners` (`id` int(11) DEFAULT NULL auto_increment PRIMARY KEY, `distance` float, `minutes` int(11), `created_at` datetime, `updated_at` datetime);             
      Query OK, 0 rows affected (0.06 sec)

    2. Update “hello.rb” from TOTD #79 such that it looks like:
      require ‘rubygems’
      require ‘sinatra’
      require ‘activerecord’

      ## Setup

      ActiveRecord::Base.establish_connection(
        :adapter  => “jdbcmysql”,
        :host     => “localhost”,
        :username => “root”,
        :password => “”,
        :database => “hello_development”
      )

      ## Models

      class Runner < ActiveRecord::Base
      end

      ## Controller Actions

      get ‘/hi’ do
        “Hello World!”
      end

      get ‘/’ do
        @runner = Runner.find(:all)
        haml :index
      end

      get ‘/new’ do
        haml :new
      end

      get ‘/:id’ do
        @runner = Runner.find(params[:id])
        if (@runner)
          haml :show
        else
          redirect ‘/’
        end
      end

      post ‘/’ do
        @runner = Runner.new(:distance => params[:distance], :minutes => params[:minutes])
        if @runner.save
          redirect “/#{@runner.id}”
        else
          redirect ‘/’
        end
      end

      Firstly, it pulls in the ActiveRecord dependency. Then “ActiveRecord::Base.establish_connection” is used to establish a connection with the previously created database. “Runner” is tagged as a model class by inheriting from “ActiveRecord::Base” and uses the default table name (“runners” in this case). Add four new methods:

      1. Three GET methods to show all the runners, a form to enter new data, and show a particular log entry. Each method requires a HAML template (will be created in next step) to render the information.
      2. One POST method to save the newly created log entry in the database.
    3. Create a new directory “views” and create the following files in that directory. Each file serves as a view and rendered from an action in “hello.rb”.
      1. “index.haml”: Show all the runners

        %h1 Listing all runners …
        %table
          %tr
            %th Distance
            %th Minutes
          – @runner.each do |r|
            %tr
              %td= r.distance
              %td= r.minutes
        %br
        %a{:href=>”/new”}
          New Runner

      2. “new.haml”: Enter a new entry
        %h1 Adding a new runner log …
        %form{:method=>”post”, :action=>”/”}
          Distance:
          %input{:type=>”text”, :name=>”distance”}
          %br
          Minutes:
          %input{:type=>”text”, :name=>”minutes”}
          %br
          %input{:type=>”submit”, :value=>”Submit”}
          %br

      3. “show.haml”: Show a particular log entry
        %h1 Showing a runner log …
        Dis
        tance:
        = @runner.distance
        %br
        Minutes:
        = @runner.minutes
        %br
        %br
        %a{:href=>”/”}= “Show All!”

        The complete directory structure looks like:

        .
        ./hello.rb
        ./views
        ./views/index.haml
        ./views/new.haml
        ./views/show.haml

    That’s it, now run the application as:

    ~/tools/jruby/samples/sinatra-sample >../../bin/jruby -S glassfish
    Starting GlassFish server at: 192.168.1.145:3000 in development environment…
    Writing log messages to: /Users/arungupta/tools/jruby-1.2.0/samples/sinatra-sample/log/development.log.
    Press Ctrl+C to stop.

    The main page is available at “http://localhost:3000/” and looks like:

    Clicking on “New Runner” gives …

    Enter the data, and click on “Submit” to show …

    Click on “Show All!” to see all the entries added so far …

    And after adding few entries the main page looks like …

    This application shows Create and Read from the CRUD, it’s fairly easy to add Update and Delete functionality as well but that’s an excercise left for the readers :-)

    You’ll hear all about it at Develop with Pleasure, Deploy with Fun: GlassFish and NetBeans for a Better Rails Experience at Rails Conf next week.

    Technorati: totd glassfish jruby sinatra crud

    Share and Enjoy:
    • Print
    • Digg
    • Sphinn
    • del.icio.us
    • Facebook
    • Google Bookmarks
    • DZone
    • StumbleUpon
    • Technorati
    • Twitter
    • Slashdot

    April 28, 2009

    TOTD #79: Getting Started with Sinatra applications on JRuby and GlassFish Gem

    Filed under: web2.0 — arungupta @ 6:00 am


    Sinatra is a DSL for quickly creating web-applications in Ruby with minimal effort. Like Rails and Merb, Sinatra is not an MVC framework and basically follows a flat-file structure instead. The framework define conventions such as location of static files and views, bootstrap, dev/production/test environment variables, filters, helpers, TDD, and much more.  Read Getting Started for complete details. Even though Sinatra is not a MVC framework but sinatra-gen may be used to generate new Sinatra projects.

    GlassFish Gem can easily run Rails, Merb, Sinatra, and any other Ruby framework applications based upon Rack. TOTD #70 shows how to run Rails applications and TOTD #53 shows to run Merb applications. This TOTD will explain how to run a trivial Sinatra application. A later blog will describe how to plug a generic Rack-based framework.

    Lets see how to get started with Sinatra using JRuby and GlassFish gem.

    1. Install Sinatra gem as:

      ~/tools/jruby >./bin/jruby -S gem install sinatra
      JRuby limited openssl loaded. gem install jruby-openssl for full support.
      http://wiki.jruby.org/wiki/JRuby_Builtin_OpenSSL
      Successfully installed sinatra-0.9.1.1
      1 gem installed
      Installing ri documentation for sinatra-0.9.1.1…
      Installing RDoc documentation for sinatra-0.9.1.1…

    2. Create a directory “sinatra-sample”, create a file “hello.rb” in that directory with the contents shown below:
      require ‘rubygems’
      require ‘sinatra’
      get ‘/hi’ do
        “Hello World!”
      end

    3. Run your sample using GlassFish gem as:
      ~/tools/jruby/samples/sinatra-sample >../../bin/jruby -S glassfish
      Log file /Users/arungupta/tools/jruby-1.2.0/samples/sinatra-sample/log/development.log does not exist. Creating a new one…
      Starting GlassFish server at: 192.168.1.145:3000 in development environment…
      Writing log messages to: /Users/arungupta/tools/jruby-1.2.0/samples/sinatra-sample/log/development.log.
      Press Ctrl+C to stop.

      And then the output is available at “http://localhost:3000/hi” and looks like:

    Neat and simple!

    You’ll hear all about it at Develop with Pleasure, Deploy with Fun: GlassFish and NetBeans for a Better Rails Experience at Rails Conf next week.

    Here is the order in which I’ll seek any help:

    The next blog will show how to create a Sinatra CRUD application and run it using GlassFish.

    Please leave suggestions on other TOTD (Tip Of The Day) that you’d like to see. A complete archive of all the tips is available here.
    Technorati: totd glassfish jruby sinatra

    Share and Enjoy:
    • Print
    • Digg
    • Sphinn
    • del.icio.us
    • Facebook
    • Google Bookmarks
    • DZone
    • StumbleUpon
    • Technorati
    • Twitter
    • Slashdot

    April 27, 2009

    GlassFish, NetBeans, and Project Kenai at Rails Conf 2009

    Filed under: General — arungupta @ 11:30 pm

    Did you know that …

    • GlassFish Gem is already used in production
    • GlassFish Gem can be used to run Rails, Merb, Sinatra, and any other Rack-based framework
    • Capistrano recipes are available for starting/stopping/bouncing the server
    • With GlassFish, standard Java monitoring techniques like JMX can be used for monitoring Rails apps
    • NetBeans provide a complete development environment for Rails applications

    There are many other similar nuggets that I’ll be covering in my Rails Conf 2009 session. Details are given below:

    Develop with pleasure, Deploy with Fun: GlassFish and NetBeans for a better Rails experience
    Tuesday, May 5th, 2009, 1:50pm
    Pavilion 1

    Register Today and avail a 15% discount using the code: RC09FOS.

    I plan to attend these sessions, lets see how many I can make :-) And of course, you’ll see me in the Exhibit Hall.

    And you’ll get to meet Project Kenai team, they form the foundation for Sun’s connected developer experience. Read about their participation here and meet them to learn about NetBeans and Kenai integration.

    And if you are interested in running with fellow attendees, follow @railsConfRunner.

    And it’s Vegas baby!

    JRuby and GlassFish is already used in production. Do you have a success story to share ? I’ll be happy to promote at RailsConf.

    Technorati: conf glassfish netbeans rubyonrails kenai railsconf lasvegas jruby

    Share and Enjoy:
    • Print
    • Digg
    • Sphinn
    • del.icio.us
    • Facebook
    • Google Bookmarks
    • DZone
    • StumbleUpon
    • Technorati
    • Twitter
    • Slashdot

    blogs.sun.com turns 5

    Filed under: General — arungupta @ 9:55 am


    blogs.sun.com – the blogging server for Sun employees has been chugging along for 5 years today!

    Congratulations to Linda and her team for running a stupendous show and providing all Sun employees with a great opportunity to engage with the community.

    blogs.sun.com Miles to go …
    Age 5 years 3 yrs, 8 months (73.3%)
    First Entry Apr 27, 2004 Aug 2, 2005
    Total Entries 132413 836 (0.6 %)
    Total Comments 145899 3174 (2.2 %)
    Total Bloggers 4385 1

    Some Analytics of Miles to go … …

    • 633,614 visits and 871,737 page views since Jan 5, 2007 according to Google Analytics
    • 20,088 cities from 210 countries according to Google Analytics
    • 747,907 visits since Jun 12, 2006 according to clustrmaps

    And here are some image snapshots …

    And here are the topics on which I’ve blogged more than 50 entries:

    I’d love to see similar statistics for blogs.sun.com :-)

    Follow bsc5years for all similar entries!

    Technorati: bsc5years bsc milestogo

    Share and Enjoy:
    • Print
    • Digg
    • Sphinn
    • del.icio.us
    • Facebook
    • Google Bookmarks
    • DZone
    • StumbleUpon
    • Technorati
    • Twitter
    • Slashdot

    April 23, 2009

    GlassFish asadmin CLI-driven Cluster Setup

    Filed under: General — arungupta @ 11:30 pm

    Here is simple script that:

    • Installs GlassFish
    • Creates a new domain using cluster profile
    • Create 2 instances in the cluster
    • Deploys a sample application to verify the cluster setup

    Everything in just one simple script!

    This script can be used on a virtual (Virtual Box, EC2 instance, etc.) or a physical image of an Operating System.

    echo A | java -Xmx256m -jar ~/Downloads/glassfish-installer-v2.1-b60e-darwin.jar -console
    cd $GLASSFISH_HOME
    chmod +x ./lib/ant/bin/ant
    ./lib/ant/bin/ant -f setup.xml
    cd $GLASSFISH_HOME
    echo ‘AS_ADMIN_ADMINPASSWORD=adminadmin’ > password
    echo ‘AS_ADMIN_PASSWORD=adminadmin’ >> password
    echo ‘AS_ADMIN_MASTERPASSWORD=changeit’ >> password
    ./bin/asadmin create-domain –user admin –passwordfile ./password –savelogin=true –portbase 5000 –interactive=false –profile cluster cloud
    ./bin/asadmin start-domain cloud
    ./bin/asadmin create-node-agent –user admin –port 5048 –interactive=false –passwordfile ./password cloud-nodeagent
    ./bin/asadmin start-node-agent –interactive=false –passwordfile ./password cloud-nodeagent
    ./bin/asadmin create-cluster –port 5048 wines
    ./bin/asadmin create-instance –port 5048 –nodeagent cloud-nodeagent –systemproperties HTTP_LISTENER_PORT=58080 –cluster wines cabernet
    ./bin/asadmin create-instance –port 5048 –nodeagent cloud-nodeagent –systemproperties HTTP_LISTENER_PORT=58081 –cluster wines merlot
    ./bin/asadmin deploy –target wines –port 5048 –availabilityenabled=true samples/quickstart/clusterjsp/clusterjsp.ear
    ./bin/asadmin start-cluster –port 5048 –interactive=false –passwordfile ./password wines

    After the script execution is complete, open up “http://localhost:58080/clusterjsp”. The page shows it is served from the “cabernet” instance. Enter some session data by adding values in the text box placed towards end of the page. Then stop the “cabernet” instance as explained in TOTD #67 after the string “OK, now show time!”.

    Now load the page “http://localhost:58081/clusterjsp” and it shows that the page is served from “merlot” instance. And the exact same session data is displayed towards the bottom of the page.

    It basically shows that the session data added in one instance is replicated to the “buddy instance” (“merlot” in this case) automatically.

    This scipt is tested on Open Solaris 2008/11, Windows Vista, and Mac OSX 10.5.x.

    The cluster and instance creation can be easily done using the web-based admin console as described here. But this TOTD also shows the power of scriptability for common GlassFish administration commands – asadmin is your hidden gem and learn more about it in this recorded webinar!

    TOTD #69 explains how to use Sun Web Server and Load-Balancer Plugin for the load balancer + clustering setup on Windows Vista. TOTD #67 explains the same steps for Apache + mod_jk on Mac OSX.

    Technorati: glassfish cli asadmin clustering highavailability

    Share and Enjoy:
    • Print
    • Digg
    • Sphinn
    • del.icio.us
    • Facebook
    • Google Bookmarks
    • DZone
    • StumbleUpon
    • Technorati
    • Twitter
    • Slashdot

    April 22, 2009

    Offshore monitoring of windfarms using GlassFish – MySQL Users Conference 2009 Day 3

    Filed under: General — arungupta @ 9:00 pm

    John Powell from eMapSite stopped by at the Whisper Suite in MySQL Users Conference earlier today to talk about his GlassFish issue. The possible workaround was suggested and then the discussion became interesting on how GlassFish is used for offshore monitoring of windfarms and process weather forecasting data. Hear all about it and watch a flashy demo of their product in this video:


    NetBeans, GlassFish, and MySQL is their development stack with a “very positive experience”!

    Stay tuned for the stories entry.

    And the complete picture album is available at:

    Technorati: conf mysqlconf mysql santaclara glassfish netbeans

    Share and Enjoy:
    • Print
    • Digg
    • Sphinn
    • del.icio.us
    • Facebook
    • Google Bookmarks
    • DZone
    • StumbleUpon
    • Technorati
    • Twitter
    • Slashdot

    LOTD #21: Production Deployment Tips for running Rails on GlassFish v2 using Windows

    Filed under: web2.0 — arungupta @ 2:26 pm

    SeaChange Affinity uses Rails and GlassFish as their deployment platform. One of their core developers posted tips based upon their experience so far and they are available at:

    Rails on GlassFish v2 using Windows

    Here are some of the quotes:

    Glassfish can really handle a heavy load

    and

    handling 400 simultaneous users under a supremely heavy load, the memory was holding great

    All previous links in this series are archived at LOTD.

    Technorati: lotd glassfish jruby rubyonrails windows

    Share and Enjoy:
    • Print
    • Digg
    • Sphinn
    • del.icio.us
    • Facebook
    • Google Bookmarks
    • DZone
    • StumbleUpon
    • Technorati
    • Twitter
    • Slashdot

    MySQL Users Conference 2009 Day 3 – Cloud Shootout

    Filed under: General — arungupta @ 11:52 am

    I arrived at the MySQL Users Conference just in time for the The Great Open Cloud Shootout.

    Kaj Arno was asking questions to the invited panelists shown in the picture above. Here is a partial discussion:

    What is cloud ?

    Thorsten Fully automatbale computing infrastructure, changes the way production scale deployments operate, saves time/cost, increases reliability
    Chander Elasiticity is an important aspect, Can “shoot for the moon without shooting foot”, accessing a pool of resources which is infinite from an individual/organization perspective
    Monty Much like electricity/network bandwidth, applying that same model to computing resources
    Jeremy Virtualization is an important piece
    Lew Not new technology, rather a new way of delivery. As a developer, provision the application through the code.

    Who is it for ?

    Monty It’s for you
    Thorsten Amazon launched, mostly for geeks. 2007 -> Amazon skeptical and RightScale gets VC funding, 2008 -> some common usage, 2009 -> Top-down from CIOs. Basically everybody, cross-organiation, vertical
    Mike Horizontal technology opportunity, starting to see mainstream applications including ISVs/primary line of business, interest/adoption is growing
    Chander Definitely growing for ISVs, makes backup sexy, “Even though running a backup company, expected to be entertaining”
    Jeremy Power outlets are shaped differently, technology has not matured enough. Next few years standardization will happen.
    Monty People will never notice it exists, but able to access the information
    Prashant Putting/Sharing the data on cloud

    Why use the cloud ?

    Monty All of a sudden facebook traffic, leverage a collective of people who are already investing in an effort
    Lew Cloud computing based on virtualization
    Mike More & more enterprises moving in the cloud, gain durability & resilience which was not an option because of a single data center
    Jeremy Legacy apps are easiest to move into cloud, they are better understood and can scale easily
    Prashant Cloud is the right approach/dream, not there yet. Traditional apps can be moved into cloud.
    Thorsten Flexibility in development and tests, DBA clone another slave server with exactly the same setup to test out schema changes
    Monty Spin up EC2 instances, run the tests and shut them down … everything in approx $1. Give it back to the cloud and make it more efficient for the world in general.
    Lew We are making it so affordable, cost can be 10% of what it was before.


    Cloud adoption barrier

    Chander Performance, a customer requested a refund where they were trying to shove a 1TB in an hour. US is 6Mbps, needs to significantly increase before it can be utilized.
    Thorsten Compute needs to move where the data is.
    Chander Most businesses will find bandwidth/redundancy limited. Customer always need to customer where not to use cloud and set expectations accordingly

    What apps will never move to cloud ?

    Lew Financially sensitive applications, owning your own data center
    Chander Trust and privacy, it’s more about education though. Encryption is going tobe a key.
    Jeremy Competitiion, unless other companies battle it out and making it easy to to migrate from one service to other, it’ll be difficult. Avoid vendor lock
    in.

    Are there cloud standards ?

    Mike Based on open industry standards,  no deep rooted concern in the user community
    Thorsten Way to operate across different clouds, API is not the most important level. What is a server ? Can I hibernate it, mount it, how much storage volume is allowed, cross-data center boundary are a better abstraction.
    Lew Very early to lock the standards, everybody is currently in a stage of experiementation
    Monty Potential downside to premature standardization, too early to jump to standards
    Chander Open standards are a definite key to success. S3 fostered innovation.
    Thorsten S3 is a good standard but not an open API. It will be doubly nice if it’s “free” or “open” or whatever the word is.
    Mike Standards dont really matter if the performance cannot be met. When innovating at a rapid rate, it’ difficult to make everybody agree upon standards.
    Lew At least publish the API where everybody can use them.
    Chander Showing backup to Sun cloud, Sun has S3 compatible APIs, also compatible to WebDAV.

    Cloud Business Opportunities

    Monty You can
    Mike Very unique and compelling business opportunity. Amazon Dev Pay: Buy infrastructure on demand, setup your software on AMI, set your own price and then customers can use it, “Software as a Innuity”
    Chander Traditional backup vendors will be worried.
    Prashant Database on the cloud
    Lew Seeing an explosion in the amount of data/compute required, accordingly analytics. Tremendous amount of opportunity when Cassandra & Drizzle are cloud-enabled.
    Mike More ISVs in the cloud.
    Jeremy How to do performance tuning and optimizations in cloud, do that for major cloud infrastructure.
    Monty Freedom to work from anywhere, don’t need to be physically at the datacenter, enables multinational consulting
    Chander When more clouds become available, it’ll be explosion which will happen later this year.

    How is cloud measured ?

    Jeremy CPU time in terms of use, storage centric clouds pay for integrity
    Lew Creating Data centers with loading docks.
    Monty Paying for CPU cycle, like mainframe model.
    Thorsten Cloud is like mainframe but very elastic.
    Chander Billing is not a challenge, storage clouds are better because of pricing, compute is challenging

    Databases & Clouds

    Thorsten Flexibility of moving to the next volume, master, slave makes is very refreshing
    Monty Start out thinking M x N problems, never think about one database instance in cloud, there will be X > 1

    And the shootout had to be shutdown because the timing estimates were slightly misjudged

    But all in all, an interesting discussion!

    Come meet us at the GlassFish booth in the Exhibit Floor. Or you can stop by at room #205 for the Whisper Suite for a more personal and 1-1 conversation.

    Technorati: conf mysqlconf mysql santaclara glassfish cloud

    Share and Enjoy:
    • Print
    • Digg
    • Sphinn
    • del.icio.us
    • Facebook
    • Google Bookmarks
    • DZone
    • StumbleUpon
    • Technorati
    • Twitter
    • Slashdot

    April 21, 2009

    MySQL Users Conference 2009 Day 2

    Filed under: General — arungupta @ 9:52 pm

    I presented on Creating Quick and Powerful Web Applications with MySQL, GlassFish, and NetBeans. The key messages conveyed during the preso are:

    • GlassFish is an open source community and delivers production-quality Java EE compliant Application Server.
    • GlassFish v2 is the Java EE 5 Reference Implementation and GlassFish v3 for Java EE 6. Read complete difference here.
    • Java Persistence API makes it really easy to create database-backed Web applications. It even creates MySQL-specific queries, when possible.
    • The web-based administration console and CLI are powerful GlassFish management tools that meets the need of any IT administrator.
    • NetBeans provides comprehensive and seamlessly integrated tooling for GlassFish. The goal is to make the Eclipse tooling at par with NetBeans.

    The slides are available here.

    And then notes from some of the sessions I attended:

    State of the Dolphin

    • 12+ million users, 70k downloads/day, 1100 MySQL Partners
    • Multiple platforms: LAMP, Windows, Mac, OpenSolaris, Solaris, RedHat, Suse, Ubuntu
    • Multiple Languages: php, Perl, Python, Ruby, Java, C, C++, C#
    • MySQL 5.1: 3 million downloads in 100 days
    • MySQL 5.4 announced: InnoDB Scalalbility, Sub-query optimizations, 59% faster than 5.1, 40% improvement in read/write test, 71% throughput increase
    • InnoDB: Fast index creation (add/drop indexes w/o copying the data), Data compression (shrink tables, to significantly reduce storage and i/o)
    • Embedded InnoDB (announced today): Proven high-performance and reliability and functionality of InnoDB, low-level but powerful non-SQL API for app programmers, operational characteristics needed for stand-alone apps where there is no DBA
    • Dr DBA was awarded “Acquirer of the Year: Oracle” :-)
    • MySQL Cluster 7.0: 99.999% availability, 4.3x higher throughput, 140k+ TPM and 4x less power and consumption than 6.3
    • MySQL Query Analyzer: Continuous query monitoring, find and fix problem SQL code, historical and real-time analysis, drill down into execution statistics

    InnoDB: Innovative Technologies for Performance and Data Protection

    • Dr Heikki Tuuri, was professor at Helsinki, founded Innobase, got acquired by Oracle
    • Performance and Data Integrity are basic features
    • Architected and written by one person
    • Full transaction support, Unlimited row-level locking, multi-version read-consistency, automatic deadlock detection
    • Innovative: adaptive hash indexes, insert buffer (performance benefits), doublewrite buffer, InnoDB plugin
    • Oracle/Innobase + Sun/MySQL

    Rethinking MySQL, Enter Drizzle

    • Goals
      • Pluggable/Infrastructure Aware
      • Community Developed
      • Multicore/Concurrency (load up 10,000 connections in db)
      • Focus on Web applications/enable others
      • Modernize codebase for manageability (currently C/C++, can we reuse STL and other libraries)
    • Philosophies
      • Have open and well-documented interfaces
      • Have transparent goals and processes, that are communicated publicly
      • Have fun and encourage collaboration
      • Remove barriers to contribution and participation for everyone
      • Enable contributors to build a business around Drizzle
    • Drizzle announced at OSCON last year
      • Translated into 30+ languages since then
      • 7% of developers are from Sun
      • 100+ contributors (>500 on the mailing list), even Postgres and Firebird developers \
      • Cirrus available now, Aloha next
      • Drizzle Developer Day 2009 scheduled this Friday
      • No patches are contributed back to MySQL Enterprise
      • Will be ready for production deployment Jun 2010
    • References
      • launchpad.net/drizzle
      • IRC #drizzle
      • Email

    High Performance Rails and MySQL

    • David Berube: Apress books on “Author Practical Ruby Gems”, “Practical Rails Plugins”, “Practical Reporting with Ruby on Rails”
    • Finding performance issues in Rails
      • Rails development log
      • eabe_db_tools: Ajax popup- displays query count, query each time for a each query on a page. Will be available on github next week.
      • mysql_slow_log
      • Is it a database problem: Firebug, YSlow, Ping, tracert, etc.
    • Let the database do the heavy lifting instead of Ruby: for example, don’t sort in Ruby
    • Deep eager loading: don’t load that is not required
    • Use built-int Rails grouping and aggregate functions
    • Caching: simple ootb caching, Cache Fu, MySQL triggers for DB function caching, Rails triggers for other caching

    Did you know 1.3 billion emails were sent as part of Obama’s election campaign – and all powered by MySQL ? Hear the details from Blue State Digital engineers who created the solution and maintained it:


    And you can always read the complete case study.

    Some pictures from earlier today …

    And then the evolving picture album is available at:

    Come meet us at the GlassFish booth in the Exhibit Floor. Or you can stop by at room #205 for the Whisper Suite for a more personal and 1-1 conversation.

    Technorati: conf mysqlconf mysql santaclara glassfish netbeans

    Share and Enjoy:
    • Print
    • Digg
    • Sphinn
    • del.icio.us
    • Facebook
    • Google Bookmarks
    • DZone
    • StumbleUpon
    • Technorati
    • Twitter
    • Slashdot
    Older Posts »

    The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.
    Powered by WordPress