JBoss Clustering a Web Application

This article discusses the required steps to cluster a Web application using WildFly application server and the older server releases (JBoss AS 5).

Clustering allows us to run an applications on several parallel servers (a.k.a cluster nodes). The load is distributed across different servers, and even if any of the servers fails, the application is still accessible via other cluster nodes. Clustering is crucial for scalable enterprise applications, as you can improve performance by simply adding more nodes to the cluster.

Clustering Web applications with WildFly

WildFly clustering service is an on-demand service. This means that, even if you have started a cluster aware configuration, the cluster service won’t start until you deploy a cluster-aware application. Enabling clustering for your applications can be achieved in different ways depending on the type of application you are going to deploy:

To cluster a Web based application, all you need is to follow these steps:

  1. Start WildFly with an “ha” profile (or “full-ha”)
  2. Declare your application as “distributable” in the web.xml configuration file to have your HTTP session state replicated across the cluster:
<web-app>
  <distributable />
</web-app>

By default, Web applications rely on a distributed Infinispan Cache:

<cache-container name="web" default-cache="dist" marshaller="PROTOSTREAM" modules="org.wildfly.clustering.web.infinispan">
    <transport lock-timeout="60000"/>
    <replicated-cache name="sso">
        <locking isolation="REPEATABLE_READ"/>
        <transaction mode="BATCH"/>
    </replicated-cache>
    <replicated-cache name="routing"/>
    <distributed-cache name="dist">
        <locking isolation="REPEATABLE_READ"/>
        <transaction mode="BATCH"/>
        <file-store/>
    </distributed-cache>
</cache-container>

You can further specialize the way HTTP Session is managed through the distributable-web subsystem, which has been introduced in WildFly 17 to manage a set of session management profiles that encapsulate the configuration of a distributable session manager:

<subsystem xmlns="urn:jboss:domain:distributable-web:2.0" default-session-management="default" default-single-sign-on-management="default">

The HTTP Session Management is handled under the hoods by Infinispan therefore if you want to check its settings, you have to reach the infinispan-session-management attribute under the distributable-web subsystem. Example:

 /subsystem=distributable-web/infinispan-session-management=default:read-resource
{
    "outcome" => "success",
    "result" => {
        "cache" => undefined,
        "cache-container" => "web",
        "granularity" => "SESSION",
        "affinity" => {"primary-owner" => undefined}
    }
}

As you can see, there are several configurable attribute for the infinispan-session-management:

  • cache-container: This references the Infinispan cache-container into which session data will be stored.
  • cache: This references a cache for the related cache-container. If undefined, the default cache of the associated cache container will be used.
  • granularity: This defines the session manager mapping for the individual cache entries:
  • affinity: This attribute defines the affinity that a web request should have for a given server.

Configuring HTTP Session Granularity

The granularity attribute can have the following values:

  • SESSION: Stores all session attributes within a single cache entry. This is generally more expensive than ATTRIBUTE granularity, but preserves any cross-attribute object references.
  • ATTRIBUTE: Stores each session attribute within a separate cache entry. This is generally more efficient than SESSION granularity, but does not preserve any cross-attribute object references.

By default, WildFly’s distributed session manager uses SESSION granularity, meaning that all session attributes are stored within a single cache entry. While this ensures that any object references shared between session attributes are preserved following replication/persistence, it means that a change to a single attribute results in the replication/persistence of all attributes.

If your application does not share any object references between attributes, users are strongly advised to use ATTRIBUTE granularity. Using ATTRIBUTE granularity, each session attribute is stored in a separate cache entry. This means that a given request is only required to replicate/persist those attributes that were added/modified/removed/mutated in a given request. For read-heavy applications, this can dramatically reduce the replication/persistence payload per request. Here is how you can set the granularity to “ATTRIBUTE” for the default session manager:

/subsystem=distributable-web/infinispan-session-management=default/:write-attribute(name=granularity,value=ATTRIBUTE)

Configuring HTTP Session Affinity

The affinity attribute defines the affinity that an HTTP request has for a given WildFly server. The affinity of the associated web session determines the algorithm for generating the route to be appended onto the session ID (within the JSESSIONID cookie, or when encoding URLs). Possible values are:

  • affinity=none: HTTP requests won’t have affinity to any particular node.
  • affinity=local: HTTP requests will have an affinity to the server that last handled a request for a given session. This is the standard sticky session behavior.
  • affinity=primary-owner: HTTP requests will have an affinity to the primary owner of a given session. This is the default setting. Behaves the same as affinity=local if the backing cache is not distributed nor replicated.
  • affinity=ranked: HTTP requests will have an affinity for the first available member in a list that include the primary and the backup owners, and for the member that last handled a session. Behaves the same as affinity=local if cache is not distributed nor replicated.

The ranked affinity supports the following attributes:

  • delimiter: The delimiter used to separate the individual routes within the encoded session identifier.
  • max routes: The maximum number of routes to encode into the session identifier.

Here is how to set HTTP Session affinity to be ranked, with max-routes set to 2:

/subsystem=distributable-web/infinispan-session-management=default/affinity=ranked:add(max-routes=2)

Clustering Web applications with JBoss AS 5

The JBoss Application Server (AS) comes with clustering support out of the box.

The simplest way to start a JBoss server cluster is to start several JBoss instances on the same local network, using the run -c all command for each instance. Those server instances, all started in the all configuration, detect each other and automatically form a cluster.

For Web applications, a hardware- or software-based HTTP load-balancer usually sits in front of the application servers within a cluster. These load-balancers can be used to decrypt HTTPS requests quickly, distribute the load between nodes in the cluster, and detect server failures. The usual setup is to have a user session live entirely on one server that is picked by the load-balancer. This is called a “sticky” session.

Configuring JBoss to run in a cluster

So let’s see how to configure 2 instances of JBoss in a cluster. We’ll define two nodes nodeA and nodeB

Node Name Address
nodeA 192.168.0.1
nodeB 192.168.0.2

Now for every installation of JBoss go under the folder deploy\jboss-web.deployer (jBoss 4.X) or under the folder deploy\jbossweb.sar (jBoss 5.X and later) and pick up the file server.xml  line.

Configure on the 192.168.0.1 machine

<Engine name="jboss.web" defaultHost="localhost" jvmRoute="nodeA">

and on the machine 192.168.0.2

<Engine name="jboss.web" defaultHost="localhost" jvmRoute="nodeB">

Now create an HelloWorld web application and add a Servlet which simply dumps some session information:

public void doGet(HttpServletRequest request, HttpServletResponse response)

 throws ServletException, IOException { 
 
  response.setContentType("text/html"); 
  String dummy = (String)request.getSession().getAttribute("dummy"); 
  PrintWriter out = response.getWriter(); 
 
  System.out.print("Hello from Node "+ InetAddress.getLocalHost().toString()); 
 
  if (dummy == null) {  
   out.println("Dummy value not found. Inserting it....");  
   request.getSession().setAttribute("dummy", "dummyOk"); 
  } else {  
   out.println("Dummy is " +dummy); 
  } 
  out.flush(); 
  out.close();
}

Before packing the application add at the very beginning of the web.xml file

<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.4"
xmlns="http://java.sun.com/xml/ns/j2ee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee
http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">

   <distributable/>
</web-app>

Ok so now you’re ready to deploy the web application: where can I deploy it ?

The easiest way to deploy an application into the cluster is to use the farming service. That is to hot-deploy the application archive file (e.g., the EAR, WAR or SAR file) in the all/farm/ directory of any of the cluster member and the application is automatically duplicated across all nodes in the same cluster.

You can only put archive files, not exploded directories, in the farm directory. This way, the application on a remote node is only deployed when the entire archive file is copied over. Otherwise, the application might be deployed (and failed) when the directory is only partially copied.

If you want to read more about the Farming Service continue reading this article:
https://www.mastertheboss.com/en/jboss-application-server/146-jboss-farming-service.html

Ok, so now Jboss is ready to accept Cluster Web Applications, now we need at first something something sitting in front of our JBoss Cluster. We’ll use for our purpose Apache Web Server.

For a detailed explanation of all steps necessary to configure Apache-JBoss via modJK we suggest you reading the Wiki http://wiki.jboss.org/wiki/UsingMod_jk1.2WithJBoss. Now we’ll focus in particular on the part of the configuration relative to the cluster.

The first thing we need to have is redirecting calls to the mod_JK load balancer: this is done in file mod_jk.properties:

# Where to find workers.properties
JkWorkersFile conf/workers.properties

# Where to put jk logs
JkLogFile logs/mod_jk.log

JkMount /HelloWorld/* loadbalancer

Here HelloWorld is the Web Context of our sample application. Now let’s move to workers.properties file.

worker.list=loadbalancer,status

worker.nodeA.port=8009
worker.nodeA.host=192.168.0.1
worker.nodeA.type=ajp13
worker.nodeA.lbfactor=1

worker.nodeB.port=8009
worker.nodeB.host=192.168.0.2
worker.nodeB.type=ajp13
worker.nodeB.lbfactor=1

worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=nodeA,nodeB

# Status worker for managing load balancer
worker.status.type=status

As you can see we have the two nodes (nodeA and nodeB) with relative port and IP address.

Now launch the two instances of JBoss using the command run -c all. The first instance started will display this information when the second instance kicks in:

16:09:20,562 INFO [AjpProtocol] Starting Coyote AJP/1.3 on ajp-192.168.0.1-8009
16:09:20,578 INFO [Server] JBoss (MX MicroKernel) [4.2.0.GA (build: SVNTag=JBoss_4_2_0_GA date=200705111440)] Started in 1m:20s:438ms
16:10:28,343 INFO [TreeCache] viewAccepted(): [192.168.0.1:2300|1] [192.168.0.1:2
300, 192.168.0.2:2321]
16:10:28,421 INFO [TreeCache] locking the subtree at / to transfer state
16:10:28,593 INFO [StateTransferGenerator_140] returning the state for tree roo
ted in /(1024 bytes)
16:10:40,921 INFO [DefaultPartition] New cluster view for partition DefaultPart
ition (id: 1, delta: 1) : [192.168.0.1:1099, 192.168.0.2:1099]
16:10:40,984 INFO [DefaultPartition] I am (192.168.0.1:1099) received membership
Changed event:
16:10:40,984 INFO [DefaultPartition] Dead members: 0 ([])
16:10:40,984 INFO [DefaultPartition] New Members : 1 ([192.168.0.2:1099])
16:10:40,984 INFO [DefaultPartition] All Members : 2 ([192.168.0.1:1099, 192.168.0.2:1099])
16:10:45,187 INFO [TreeCache] viewAccepted(): [192.168.0.1:2309|1] [192.168.0.1:2 309, 192.168.0.2:2336]
16:10:47,453 INFO [TreeCache] viewAccepted(): [192.168.0.1:2313|1] [192.168.0.1:2 313, 192.168.0.2:2341]

Ok. In order to invoke our Servlet simply point to our Apache Web Server domain address

http://apache.web.server/HelloWorld/helloWorld

You’ll see that the balancer load balances calls and, once it has created the session, it will replicate the session to the secondary server. You can do experiments to test your cluster, switching off one instance of the cluster and verifying that the session stays alive on the other server.

Simply verify that the dummy values stays in the Session when you switch off the current instance of JBoss.

Dummy is dummyOk

Conclusion

JBoss clustering is designed to be simple to configure and simple to extend. It is one of the final pieces of the puzzle that makes JBoss a viable open source competitor to other enterprise J2EE offerings.

You can read more about Cluster Http replication in this tutorial.

Found the article helpful? if so please follow us on Socials