Load Balancing ResponseMX over clustered Tomcats.
Load Balancing ResponseMX over clustered Tomcats.
What is load balancing and clustering. Well a lot cleverer people then me have wrote a lot of text on the different type of load balancing algorithms and clustering methodologies, so instead of regurgitating what my learned colleagues have wrote I’ll just briefly sum up.
Load Balancing is a method of distributing requests onto multiple servers seamlessly to the user doesn’t realise.
Clustering is slightly more in-depth. For static web sites load balancing is fine for distributing load, however for statefull applications running on app servers load balancing alone won’t work. This problem is load balanced servers work completely independently, so consecutive requests sent to different application servers will lose their session. Clustering allows multiple application servers to share their session data, so it is transparent to the user which server they are using at any time
Why would you want to do this ? well for two reasons, firstly to enhance performance, on sites with large volumes (two servers are better then one), giving you more capacity. Secondly for business critical applications distributed server gives fail over, I.e. if one server goes down you have a seconds server there ready to take the load (just like those skeletons in Jason and the Argonauts).
We’ve been doing some work on ResponseMX with MapXtreme to run over load balanced servers with clustered tomcats. ResponseMX uses session persistence quite heavily to store the state of its geographic maps. Clustered applications can work in a few ways, such as persisting its session data to a database, or broadcasting its sessions data to any other application servers, to do this we need all session data to serialisable, this is allowing data to be streamed to a string or suitable format to be wrote to disk. The changes required to ResponseMX will be included in the 3.0 release, however if you would like to get hold of a Beta version of this functionality in 2.6 please contact a member of the Dotted Eyes team who will be happy to oblige.
To set up our environment we shall be using 2 Tomcat 5 Application servers clustered both running ResponseMX and Mapxtreme. We shall also use Apache HTTP server in front on them to act as our load balancer.
1. Setting up Apache as a load balance (Many thanks to Pascal Forget and Matt Raible) for a great aticle on setting up Apache as load balancer http://raibledesigns.com/tomcat/)
1.1. Download and install Apache2.2
1.2. Download and install the mod_jk module into the modules directory of Apache
1.3. Add the following text to the Dynamic Shared Object (DSO) Support section of the httpd.conf file to install mod_jk
<br />
LoadModule jk_module modules/mod_jk.so<br />
JkWorkersFile conf/workers.properties<br />
JkLogFile logs/mod_jk.log<br />
JkLogLevel info<br />
1.4. Add the the following text to the httpd.conf file telling Apache what requests are for ResponseMX and to forward to the Tomcat application server.
<br />
JkMount /rmx26* loadbalancer<br />
JkMount /mapxtreme* loadbalancer<br />
1.5. Create a workers.properties file in the conf folder of your Apache installation, with the following text
<br />
ps=/<br />
# list the workers by name<br />
worker.list=tomcat1, tomcat2, loadbalancer<br />
# ------------------------<br />
# First tomcat server<br />
# ------------------------<br />
worker.tomcat1.port=11009<br />
worker.tomcat1.host=localhost<br />
worker.tomcat1.type=ajp13<br />
worker.tomcat1.lbfactor=100<br />
# ------------------------<br />
# Second tomcat server<br />
# ------------------------<br />
worker.tomcat2.port=12009<br />
worker.tomcat2.host=localhost<br />
worker.tomcat2.type=ajp13<br />
worker.tomcat2.lbfactor=100<br />
# ------------------------<br />
# Load Balancer worker<br />
# ------------------------<br />
worker.loadbalancer.type=lb<br />
worker.loadbalancer.balanced_workers=tomcat1, tomcat2<br />
2. Set up 2 clustered tomcat instances, for this example we shall be running two instances of tomcat on the same server. We shall also be using the in memory broadcasting clustering setup.
2.1. Download Apache Tomcat, I’ve tested this configuration on Tomcat 5.5.25 and Java 1.5
2.2. Install it twice in two separate directories.
2.3. Change the server.xml file to make sure tomcat is listening on 2 different ports to avoid those annoying port conflict errors. I’ve chosen to run mine on ports 9080 and 10080
<br />
<Connector port="9080" maxHttpHeaderSize="8192"<br />
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"<br />
enableLookups="false" redirectPort="8443" acceptCount="100"<br />
connectionTimeout="20000" disableUploadTimeout="true" /><br />
2.4. Change the AJP13 connector ports so they correspond the the ports in the workers.properties file in the apache installation
<br />
<Connector port="11009"<br />
enableLookups="false" redirectPort="8443" protocol="AJP/1.3" /><br />
2.5. Add the following cluster Element to the server.xml in the Engine element
<br />
<Cluster className="org.apache.catalina.cluster.tcp.SimpleTcpCluster"<br />
managerClassName="org.apache.catalina.cluster.session.DeltaManager"<br />
expireSessionsOnShutdown="false"<br />
useDirtyFlag="true"<br />
notifyListenersOnReplication="true"><br />
<Membership<br />
className="org.apache.catalina.cluster.mcast.McastService"<br />
mcastAddr="228.0.0.4"<br />
mcastPort="45564"<br />
mcastFrequency="500"<br />
mcastDropTime="3000"/><br />
<Receiver<br />
className="org.apache.catalina.cluster.tcp.ReplicationListener"<br />
tcpListenAddress="auto"<br />
tcpListenPort="4001"<br />
tcpSelectorTimeout="100"<br />
tcpThreadCount="6"/><br />
<Sender<br />
className="org.apache.catalina.cluster.tcp.ReplicationTransmitter"<br />
replicationMode="pooled"<br />
ackTimeout="15000"<br />
waitForAck="true"/><br />
<Valve className="org.apache.catalina.cluster.tcp.ReplicationValve"<br />
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;"/><br />
<Deployer className="org.apache.catalina.cluster.deploy.FarmWarDeployer"<br />
tempDir="/tmp/war-temp/"<br />
deployDir="/tmp/war-deploy/"<br />
watchDir="/tmp/war-listen/"<br />
watchEnabled="false"/><br />
<ClusterListener className="org.apache.catalina.cluster.session.ClusterSessionListener"/><br />
</Cluster><br />
3. Install Mapxtreme471 and ResponseMX on both tomcat instances.
3.1. Some configuration is needed on where the ResponseMX server will write its map images to, and where the client will request them from. The problem with having the application load balanced is you never know which server is going to get the request, and a request for a map from ResponseMX actually means two requests to the server, the xml map request telling the server what geoset to load, the view etc, which will respond with a URL to a map image, and the the second request to get that map image. However on load balanced and even clustered servers the image will only be wrote on that server, so if the second request goes to another server you’ll get a 404 error. The answer is to write the image to a shared directory (perhaps where your map data is being held). On my example I wrote them to a separate directory outside the ResponseMX application so the images could be requested straight from apache. The parameters that need changing are in the rmx.properties file
<br />
servername = rmx1<br />
image_url = http://<server name>/images<br />
image_dir = C:/Program Files/Apache Software Foundation/Apache2.2/htdocs/images<br />
The servername property needs to be different for each clustered tomcat instance. This parameter is appended the the image name to avoid naming conflicts between servers.
3.2. The server.xml in each ResponseMX application needs to be changed; the following line should be added to the web.xml file in the WEB-INF folder.
This tell tomcat that the session objects for this application should be shared with any other application servers in the cluster
4. Have fun with maps
4.1. Start both your tomcat instances and apache
4.2. Open a browser and hit the ResponseMX application through Apache on port 80 as opposed to going straight to tomcat (http://(server name)/rmx20 )
4.3. By looking at the ResponseMX log files you should be able to tell which tomcat instance is is receiving the requests, and if all is working it should be all of them.
Have fun clustering everyone.
