Miles to go …

GlassFish provides support for High Availability by creating a cluster of server instances and session state replication. This enhances the scalability and availability of your application and is a critical piece of decision making critieria when selecting an Application Server. Clustering in GlassFish Version 2 provides comprehensive introduction to clustering, high availability and load balancing in GlassFish.

GlassFish provides out-of-the-box support for load-balancing HTTP(S), JMS, and RMI/IIOP traffic and front-ended by Sun Java System Web Server, Apache Web Server, and Microsoft IIS (more details here) using the Load Balancer plug-in. This plug-in however is not available for Mac OS X and a popular technique used on that platform for front-ending is to use Apache httpd + mod_jk. This is exactly what this TOTD (Tip Of The Day) is going to describe.

This TOTD is going to explain how to front-end a 3-instance GlassFish cluster with Apache httpd and mod_jk on Mac OS X.

This blog is using information from the following blogs:

• How to Loadbalance GlassFish Cluster with Apache Loadbalancer
• GlassFish Clustering in under 10 minutes
• How to install tomcat mod_jk on Mac OS X Leopard

And thanks to Vivek and Shreedhar for helping me understand the guts of GlassFish High Availability.

Without further ado, lets get started. The steps are slightly involving so strap your seatbelts!

1. First, lets create a 3-instance cluster following the screencast at GlassFish Clustering in under 10 minutes. Use the cluster name as “cluster1″ and instance names as “instance1″, “instance2″, “instance3″. The admin console will look like:

   

   Deploy “clusterjsp” and make sure it works using port hopping as explained in the screencast. Click on each instance to identify their associated HTTP port.

2. Define “jvmRoute” and “enableJK” properties on the newly created cluster as:
   

   ~/samples/v2/clustering/glassfish/bin >./asadmin create-jvm-options –target cluster1 “-DjvmRoute=\${AJP_INSTANCE_NAME}” Command create-jvm-options executed successfully. ~/samples/v2/clustering/glassfish/bin >./asadmin create-jvm-options –target cluster1 “-Dcom.sun.enterprise.web.connector.enableJK=\${AJP_PORT}” Command create-jvm-options executed successfully.

   These properties are required to enable “stickiness” for “mod_jk”. More details about how these properties are used internally is explained here.

3. Configure the above system properties for each instance in the cluster as shown:
   

   ~/samples/v2/clustering/glassfish/bin >./asadmin create-system-properties –target instance1 AJP_INSTANCE_NAME=instance1 Command create-system-properties executed successfully. ~/samples/v2/clustering/glassfish/bin >./asadmin create-system-properties –target instance1 AJP_PORT=9090 Command create-system-properties executed successfully. ~/samples/v2/clustering/glassfish/bin >./asadmin create-system-properties –target instance2 AJP_INSTANCE_NAME=instance2 Command create-system-properties executed successfully. ~/samples/v2/clustering/glassfish/bin >./asadmin create-system-properties –target instance2 AJP_PORT=9091 Command create-system-properties executed successfully. ~/samples/v2/clustering/glassfish/bin >./asadmin create-system-properties –target instance3 AJP_INSTANCE_NAME=instance3 Command create-system-properties executed successfully. ~/samples/v2/clustering/glassfish/bin >./asadmin create-system-properties –target instance3 AJP_PORT=9092 Command create-system-properties executed successfully.

   Note the value of “AJP_PORT” property for each instance, this will be used for configuring “mod_jk” later. You may have to restart the cluster in order for these properties to be synchronized for each instance. This can be easily done using the admin console as explained in the screencast above.

4. Install httpd: Mac OS X Leopard 10.5.6 comes bundled with Apache httpd 2.2, so that’s cool! Otherwise it can be downloaded from httpd.apache.org. However in the pre-installed version there are some intricacies with directory names that are explained below.
5. Lets install and configure “mod_jk” in “httpd”.
   1. The mod_jk binaries for Mac OSX are supposedly available at www.apache.org/dist/tomcat/tomcat-connectors/jk/binaries/macosx/. But installing the available module in httpd gives the following error:

      httpd: Syntax error on line 116 of /private/etc/apache2/httpd.conf: Cannot load /usr/libexec/apache2/mod_jk-1.2.25-httpd-2.2.4.so into server: dlopen(/usr/libexec/apache2/mod_jk-1.2.25-httpd-2.2.4.so, 10): no suitable image found.  Did find:\n\t/usr/libexec/apache2/mod_jk-1.2.25-httpd-2.2.4.so: mach-o, but wrong architecture

   2. The alternative is to build “mod_jk”. Fortunately it turned out to be a straight forward process because of this blog.
      1. Download latest Connectors source code (version 1.2.27). The file “BUILD.txt” (included in the source bundle) provide clear
         instructions to build and described below as well.
      2. Configure the build environment as shown:
         

         ~/workspaces/tomcat-connectors-1.2.27-src/native >./configure –with-apxs=/usr/sbin/apxs

         It shows the output as:

         . . . checking for target platform… unix no apache given no netscape given configure: creating ./config.status config.status: creating Makefile config.status: creating apache-1.3/Makefile config.status: creating apache-1.3/Makefile.apxs config.status: creating apache-2.0/Makefile config.status: creating apache-2.0/Makefile.apxs config.status: creating common/Makefile config.status: creating common/list.mk config.status: creating common/jk_types.h config.status: creating jni/Makefile config.status: creating common/portable.h config.status: executing depfiles commands

      3. Edit Edit “native/apache-2.0/Makefile.apxs” and add “-arch x86_64″ as described here. Please note that this string needs to be specified twice.
      4. Invoke “make” and “mod_jk.so” is generated in “native/apache-2.0″ directory.
   3. Copy the generated “mod_jk.so” as:
      

      ~/workspaces/tomcat-connectors-1.2.27-src/native/apache-2.0 >sudo cp mod_jk.so /usr/libexec/apache2/

   4. Load the “mod_jk” module in httpd by editing “/etc/apache2/httpd.conf”. Please note another similar file exists in “/etc/httpd/httpd.conf” but ignore that one. Add the following as the last “LoadModule” line:
      

      LoadModule jk_module     libexec/apache2/mod_jk-1.2.25-httpd-2.2.4.so

   5. Configure “mod_jk” by adding the following lines immediately below the previously “LoadModule” line:
      

      JkWorkersFile /etc/apache2/worker.properties # Where to put jk logs JkLogFile /var/log/httpd/mod_jk.log # Set the jk log level [debug/error/info] JkLogLevel debug # Select the log format JkLogStampFormat “[%a %b %d %H:%M:%S %Y] “ # JkRequestLogFormat set the request format JkRequestLogFormat “%w %V %T” # Send all jsp requests to GlassFish JkMount /*.jsp loadbalancer

      The key lines in this configuration are the first and the last one. The first line informs “mod_jk” about the location of “worker.properties” file (explained later). The last line instructs to redirect only JSP requests. This will allow the static content such as images, text files, and media to be served from “httpd” itself.

      Also create the log directory specified in the configuration as:

      sudo mkdir /var/log/httpd

   6. Create a new file “/etc/apache2/worker.properties”. Again, this is not in “/etc/httpd” directory. Use the following contents:
      

      # Define 1 real worker using ajp13 worker.list=loadbalancer # Set properties for instance1 worker.instance1.type=ajp13 worker.instance1.host=localhost worker.instance1.port=9090 worker.instance1.lbfactor=50 worker.instance1.cachesize=10 worker.instance1.cache_timeout=600 worker.instance1.socket_keepalive=1 worker.instance1.socket_timeout=300 # Set properties for instance2 worker.instance2.type=ajp13 worker.instance2.host=localhost worker.instance2.port=9091 worker.instance2.lbfactor=50 worker.instance2.cachesize=10 worker.instance2.cache_timeout=600 worker.instance2.socket_keepalive=1 worker.instance2.socket_timeout=300 # Set properties for instance3 worker.instance3.type=ajp13 worker.instance3.host=localhost worker.instance3.port=9092 worker.instance3.lbfactor=50 worker.instance3.cachesize=10 worker.instance3.cache_timeout=600 worker.instance3.socket_keepalive=1 worker.instance3.socket_timeout=300 worker.loadbalancer.type=lb worker.loadbalancer.balance_workers=instance1,instance2,instance3

      Read more about worker.properties format. Essentially this file is used to load-balance a 3-instance cluster and specify configuration values for each instance. Note that the value of “worker.instanceX.port” for instance X is exactly same as specified during instance configuration earlier.

6. Copy “tomcat-ajp.jar” from the “lib” directory of the latest Tomcat 5.5.x, commons-logging.jar (version 1.1.1), and commons-modeler.jar (version 2.0.1) to GLASSFISH_HOME/lib. This is done as:
   

   ~/samples/v2/clustering/glassfish/lib >cp ~/tools/apache-tomcat-5.5.27/server/lib/tomcat-ajp.jar . ~/samples/v2/clustering/glassfish/lib >cp ~/Downloads/commons-logging-1.1.1/commons-logging-1.1.1.jar . ~/samples/v2/clustering/glassfish/lib >cp ~/tools/commons-modeler-2.0.1/commons-modeler-2.0.1.jar .

   You may have to restart cluster in order for these JARs to be loaded by each instance.

7. An “httpd” instance is already running on port# 80 in my particular instance of Mac OS X. Instead of mangling with that, I decided to change the listening port for the new instance that will be spawn for out front-end. This can be easily done by editing “/etc/apache2/httpd.conf” and looking for lines similar to:
   

   #Listen 12.34.56.78:80 Listen 80

   And change
   “Listen 80″ to “Listen 81″.

That completes the configuration, phew!

Lets start “httpd” as:

sudo httpd

The “httpd” logs are available in “/private/var/log/apache2″. The following message indicates a successful start of the web server:

. . . [Thu Jan 29 11:14:16 2009] [warn] Init: Session Cache is not configured [hint: SSLSessionCache] [Thu Jan 29 11:14:16 2009] [warn] No JkShmFile defined in httpd.conf. Using default /usr/logs/jk-run time-status [Thu Jan 29 11:14:16 2009] [warn] No JkShmFile defined in httpd.conf. Using default /usr/logs/jk-run time-status [Thu Jan 29 11:14:16 2009] [notice] Digest: generating secret for digest authentication … [Thu Jan 29 11:14:16 2009] [notice] Digest: done [Thu Jan 29 11:14:16 2009] [warn] pid file /private/var/run/httpd.pid overwritten — Unclean shutdow n of previous Apache run? [Thu Jan 29 11:14:16 2009] [notice] Apache/2.2.9 (Unix) mod_ssl/2.2.9 OpenSSL/0.9.7l DAV/2 mod_jk/1. 2.27 configured — resuming normal operations

OK, now show time!

If everything has been configured properly as described above then “http://localhost:81/clusterjsp/HaJsp.jsp” looks like:

Enter session attribute as “aaa” and value of the attribute as “111″. After you click on “ADD SESSION DATA” button, the updated page looks like:

The highlighted part shows that the request is served from “instance1″ and recently added session data. Lets stop “instance1″ and see if the promise of high availability is fulfilled

Click on “Instances” tab, select “instance1″ and click “Stop”. The admin console looks like:

Notice “instance1″ is shown as stopped. Clicking on “Reload Page” on “http://localhost:81/clusterjsp/HaJsp.jsp” shows:

Aha!

Even though “instance1″ is not runing the session data is still available. And that is possible because of the seamless session failover from primary (“instance1″) to the replica partner (“instance2″). The highlighted part indicates that the request is now indeed served by “instance2″.

Here are some other useful links to consider:

• High Availability Adminstration Guide (online, pdf).
• GlassFish High Availability Reference Configurations.
• GlassFish High Availability Webinar.

Please leave suggestions on other TOTD (Tip Of The Day) that you’d like to see. A complete archive of all tips is available here.

Technorati: totd glassfish highavailability clustering loadbalancer mod_jk apache httpd mac leopard

Interested in contributing to open source ? Archana N has a recommendation for you:

Flame the sparks of passion in you,
Experience the world of Open Source anew.
Discover your potentials as they unfold,
Venture to contribute in the technologies untold.

Archana N is an ex-Sun Campus Ambassador from Amrita Vishwa Vidyapeetham, India. Recently she contributed on how to secure the existing EJB converter example from the Java EE tutorial using mutual authentication. She continue to work on developing RESTful web services using Project Jersey as her next contribution for GlassFish documentation.

The GlassFish Documentation team published her interview to recognize the efforts, read it here.

Thank you Archana for your contribution to GlassFish, keep the passion alive!

If you are interested in learning/talking about GlassFish at your university, spotlight.dev.java.net/start provides a comprehensive set of slides (with speaker notes), demos, screencasts, training materials and other resources to get you started!

Technorati: glassfish students campusambassador spotlight

Are you a student and would like to learn why GlassFish is meant for you ?

Create/join a student community at osum.sun.com and attend the GlassFish Webinar. This webinar will introduce GlassFish application server, show multiple demos that can be used to impress your colleagues/professors, and answer your questions around GlassFish.

The coordinates are:

Feb 3, 2009, 8:30am PT
Webinar Invite

This webinar will be conducted using Elluminate Live! You will be able to listen to live audio via your computer speakers or ear phones and be able to ask questions using chat/text.

Learn more about OSUM (pronounced “awesome”) here. Create your OSUM today and join over 900 student clubs from over 80 countries!

If you are interested in talking about GlassFish, spotlight.dev.java.net/start provides a comprehensive set of slides (with speaker notes), demos, screencasts, training materials and other resources to get you started!

Technorati: glassfish webinar osum students spotlight

As reported earlier, Alan Vargas, the Sun Campus Ambassador at CEUI (Centro Universitario de Ixltahuaca) presented on GlassFish. The message of “GlassFish v2 – Fast, Easy, and Reliable” and “GlassFish v3 – Modular, Embeddable, and Extensible” was clearly delivered in the talk.

Alan has published several useful blogs on GlassFish (in Spanish and English) recently:

• Plática de Glassfish en CEUI (GlassFish chat in CEUI)
• Historia de Glassfish (History of GlassFish)
• Qué es Glassfish? (What is GlassFish ?)
• Running Glassfish v2 UR2 on Windows 7
• Corriendo Glassfish v2 UR2 en Windows 7

The slides from his talk are available here. Here are some pictures from the talk:

The complete album is available here. Thanks to Alan Vargas for delivering a GlassFish talk!

Ronaldo Pass talked about GlassFish at the recently concluded Campus Party Brazil 2009. This is claimed to be biggest party of the Internet world in Brazil An attendance of over 6655, sort of, vouch for that! More details on Campus Party in the wikipedia entry.

There is a Campus Party community at OSUM, join here. Enjoy the images posted by the community on Flickr and videos @ YouTube.

Thanks to Ronaldo Pass for spreading the GlassFish wealth!

Please drop a comment on this blog if you have talked about GlassFish in your school/university/training. If you’d like to talk about it then spotlight.dev.java.net/start provides a comprehensive list of material (slides, demos, etc.) to get you started!

Technorati: glassfish students spotlight brazil mexico

A picture is worth thousand words! Now that Picasa 3.0 is available on Mac, here is a collage of pictures from the recently concluded Sun Tech Days in Singapore.

The complete photo album is available at:

And if you are interested in knowing all the action, read here!

Technorati: conf suntechdays singapore

Learn about GlassFish at Centro Universitario de Ixtlahuaca OSUM!

Date: Jan 26, 2009 Time: 10am-12pm PT Language: Spanish Sign up here!

Read more about OSUMs here.

Please drop a comment on this blog if you have talked about GlassFish in your school/university/training. If you’d like to talk about it then spotlight.dev.java.net/start provides a comprehensive list of material (slides, demos, etc.) to get you started!

Technorati: glassfish students spotlight

Follow up from Day 1.

Matt Thompson talked about cloud computing. Even though the term “cloud computing” is becoming more prominent now, it very well echoes Sun’s 25 years old mantra – “The Network is the Computer”. Even though there are some prominent players but the field is very much emerging. Matt touched upon different interpretations of a cloud – Platform, Infrastructure or Software as a Service (PaaS, Iaas, SaaS). A set of developer tools that allows you to manage your cloud is equally important for it to be successful. He then gave a preview of some of the open source developer tools (Kenai and Zembly) available from Sun Microsystems to assist in that direction.

After the keynote, some participants showed their “strange and unusual” talent to add a bit of levity to the otherwise intense technical conference. Even though there were 6 participants but it was less “strange and unusal” than the previous ones I’ve attended (Sao Paolo 2008, Hyderabad 2008, Beijing 2007). Here is a video of the show and judget it yourself:

A video of the GlassFish demo shown during Day 1 is now available:

The cloud camp in the evening had approx 50 attendees with an interesting discussion around the definition of cloud, security concerns, portability of applications across clouds, major players in the game, and much more. Free pizza and beer is always a good way to keep audience engaged until late in the night Sign up for a cloud camp near your city!

Here are some pictures from Day 2:

University Day today!

And finally the complete photo album in Singapore so far:

Follow the latest updates on twitter.com/arungupta.

Technorati: conf suntechdays singapore glassfish cloudcamp