Configuring a Cluster of Artemis MQ servers

In this tutorial we will learn how to learn how to install and configure a cluster of Artemis MQ servers.

ArtemisMQ is the default broker for WildFly 10 and it's based on the former HornetQ messaging broker. As for HornetQ, you can either run a cluster of brokers embedded into the application server or as standalone JMS servers.

The advantages of running standalone Artemis MQ servers are mostly:

  • You have a wider list of communication protocols for the standalone ArtemisMQ server such as amqp, amqp or openwire
  • You can define a custom process sizing if you are running ArtemisMQ server standalone

So start by installing ArtemisMQ which is available from: https://activemq.apache.org/artemis/download.html

Next install the server by unzipping it.

Create JMS Servers

The ArtemisMQ distribution can host multiple brokers. So you need to create one broker to get started. It is recommended to create the broker outside of the extracted distribution. So if you were to create two ArtemisMQ servers in your $HOME/artemis folder:

$ ./artemis create ~/artemis/server1

You will be prompted to enter the user credentials for the server:

Creating ActiveMQ Artemis instance at: /home/jboss/apache-artemis-2.2.0/bin/server1

--user: is a mandatory property!
Please provide the default username:
admin

--password: is mandatory with this configuration:
Please provide the default password:

--allow-anonymous | --require-login: is a mandatory property!
Allow anonymous access?, valid values are Y,N,True,False
Y

Auto tuning journal ...
done! 

Perform the same steps also for the other server.

$ ./artemis create ~/artemis/server2

Configure Clustering with Artemis MQ

Now we will configure clustering. The basic way to configure clustering is via Server discovery which uses UDP multicast or JGroups to broadcast server connection settings.

In order to do that, we will be adding the Clustering configuration into the etc/broker.xml. Here is a sample Clustered configuration, which uses UDP broadcast groups and discovery groups to find out cluster members:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<configuration xmlns="urn:activemq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:activemq /schema/artemis-server.xsd">   

   <core xmlns="urn:activemq:core">

      <bindings-directory>./data/bindings</bindings-directory>

      <journal-directory>./data/journal</journal-directory>

      <large-messages-directory>./data/largemessages</large-messages-directory>

      <paging-directory>./data/paging</paging-directory>

      <connectors>
         <connector name="netty-connector">tcp://localhost:61616</connector>
      </connectors>

      <acceptors>
         <acceptor name="netty-acceptor">tcp://localhost:61616</acceptor>
      </acceptors>

      <broadcast-groups>
         <broadcast-group name="my-broadcast-group">
            <group-address>${udp-address:231.7.7.7}</group-address>
            <group-port>9876</group-port>
            <broadcast-period>100</broadcast-period>
            <connector-ref>netty-connector</connector-ref>
         </broadcast-group>
      </broadcast-groups>

      <discovery-groups>
         <discovery-group name="my-discovery-group">
            <group-address>${udp-address:231.7.7.7}</group-address>
            <group-port>9876</group-port>
            <refresh-timeout>10000</refresh-timeout>
         </discovery-group>
      </discovery-groups>

      <cluster-connections>
         <cluster-connection name="my-cluster">
            <connector-ref>netty-connector</connector-ref>
            <retry-interval>500</retry-interval>
            <use-duplicate-detection>true</use-duplicate-detection>
            <message-load-balancing>STRICT</message-load-balancing>
            <max-hops>1</max-hops>
            <discovery-group-ref discovery-group-name="my-discovery-group"/>
         </cluster-connection>
      </cluster-connections>

      <security-settings>
         <!--security for example queue-->
         <security-setting match="exampleQueue">
            <permission roles="guest" type="createDurableQueue"/>
            <permission roles="guest" type="deleteDurableQueue"/>
            <permission roles="guest" type="createNonDurableQueue"/>
            <permission roles="guest" type="deleteNonDurableQueue"/>
            <permission roles="guest" type="consume"/>
            <permission roles="guest" type="send"/>
         </security-setting>
      </security-settings>

   <addresses>
         <address name="exampleQueue">
            <anycast>
               <queue name="exampleQueue"/>
            </anycast>
         </address>
      </addresses>
   </core>
</configuration>

Perform the same settings also on server2. Just mind that you will need to provide an offset address for your connectors/acceptors:

  <connectors>
         <connector name="netty-connector">tcp://localhost:62616</connector>
  </connectors>

  <acceptors>
         <acceptor name="netty-acceptor">tcp://localhost:62616</acceptor>
  </acceptors>

Finally, as ArtemisMQ will start as well an embedded Web server, change as well the bootstrap.xml with an unique port address, such as 8261 for the second server:

   <web bind="http://localhost:8261" path="web">
       <app url="jolokia" war="jolokia.war"/>
   </web>

Now you can start both brokers:

"/home/jboss/apache-artemis-2.2.0/bin/server1/bin/artemis" run
"/home/jboss/apache-artemis-2.2.0/bin/server2/bin/artemis" run

If you have applied correctly the configuration tweaks, you will be able to read from the broker logs that the ClusterConnectionBridge managed to join the cluster nodes:

16:04:45,056 INFO  [org.apache.activemq.artemis.core.server] AMQ221001: Apache ActiveMQ Artemis Message Broker version 2.2.0 [0.0.0.0, nodeID=baae836e-8999-11e7-8be3-68f728f3b97d] 
16:04:45,317 INFO  [org.apache.activemq.artemis.core.server] AMQ221027: Bridge ClusterConnectionBridge@3f18a540 [name=$.artemis.internal.sf.my-cluster.3db9fe6c-864e-11e7-ad6c-68f728f3b97d, queue=QueueImpl[name=$.artemis.internal.sf.my-cluster.3db9fe6c-864e-11e7-ad6c-68f728f3b97d, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=baae836e-8999-11e7-8be3-68f728f3b97d], temp=false]@1ae67786 targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@3f18a540 [name=$.artemis.internal.sf.my-cluster.3db9fe6c-864e-11e7-ad6c-68f728f3b97d, queue=QueueImpl[name=$.artemis.internal.sf.my-cluster.3db9fe6c-864e-11e7-ad6c-68f728f3b97d, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=baae836e-8999-11e7-8be3-68f728f3b97d], temp=false]@1ae67786 targetConnector=ServerLocatorImpl [initialConnectors=[TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=localhost], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@585324508[nodeUUID=baae836e-8999-11e7-8be3-68f728f3b97d, connector=TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=62616&host=localhost, address=, server=ActiveMQServerImpl::serverUUID=baae836e-8999-11e7-8be3-68f728f3b97d])) [initialConnectors=[TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=localhost], discoveryGroupConfiguration=null]] is connected
16:04:45,658 WARN  [org.apache.activemq.artemis.core.server] AMQ222165: No Dead Letter Address configured for queue notif.596e53d5-899e-11e7-bba1-68f728f3b97d.ActiveMQServerImpl_serverUUID=3db9fe6c-864e-11e7-ad6c-68f728f3b97d in AddressSettings
16:04:45,659 WARN  [org.apache.activemq.artemis.core.server] AMQ222166: No Expiry Address configured for queue notif.596e53d5-899e-11e7-bba1-68f728f3b97d.ActiveMQServerImpl_serverUUID=3db9fe6c-864e-11e7-ad6c-68f728f3b97d in AddressSettings
16:04:45,856 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server started at http://localhost:8261
16:04:45,857 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia REST API available at http://localhost:8261/jolokia

Configuring the cluster using JGroup broadcast group

As for WildFly 10, you can delegate the broker cluster configuration to JGroups. With ArtemisMQ you can keep your JGroups configuration in a separated file as in the following example:

<broadcast-groups>
   <broadcast-group name="my-broadcast-group">
      <jgroups-file>test-jgroups-file_ping.xml</jgroups-file>
      <jgroups-channel>activemq_broadcast_channel</jgroups-channel>
      <broadcast-period>2000</broadcast-period>
    <connector-ref connector-name="netty-connector"/>
   </broadcast-group>
</broadcast-groups>

And here is the sample jgroups XML file that can be used:

<config xmlns="urn:org:jgroups"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/JGroups-3.0.xsd">
   <TCP loopback="true"
      recv_buf_size="20000000"
      send_buf_size="640000"
      discard_incompatible_packets="true"
      max_bundle_size="64000"
      max_bundle_timeout="30"
      enable_bundling="true"
      use_send_queues="false"
      sock_conn_timeout="300"

      thread_pool.enabled="true"
      thread_pool.min_threads="1"
      thread_pool.max_threads="10"
      thread_pool.keep_alive_time="5000"
      thread_pool.queue_enabled="false"
      thread_pool.queue_max_size="100"
      thread_pool.rejection_policy="run"

      oob_thread_pool.enabled="true"
      oob_thread_pool.min_threads="1"
      oob_thread_pool.max_threads="8"
      oob_thread_pool.keep_alive_time="5000"
      oob_thread_pool.queue_enabled="false"
      oob_thread_pool.queue_max_size="100"
      oob_thread_pool.rejection_policy="run"/>

   <FILE_PING location="../file.ping.dir"/>
   <MERGE2 max_interval="30000"
      min_interval="10000"/>
   <FD_SOCK/>
   <FD timeout="10000" max_tries="5" />
   <VERIFY_SUSPECT timeout="1500"  />
   <BARRIER />
   <pbcast.NAKACK
      use_mcast_xmit="false"
      retransmit_timeout="300,600,1200,2400,4800"
      discard_delivered_msgs="true"/>
   <UNICAST timeout="300,600,1200" />
   <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"
      max_bytes="400000"/>
   <pbcast.GMS print_local_addr="true" join_timeout="3000"
      view_bundling="true"/>
   <FC max_credits="2000000"
      min_threshold="0.10"/>
   <FRAG2 frag_size="60000"  />
   <pbcast.STATE_TRANSFER/>
   <pbcast.FLUSH timeout="0"/>
</config>

You can check for more Clustering examples in the Artemis MQ documentation: https://activemq.apache.org/artemis/docs/1.0.0/clusters.html

Related articles available on mastertheboss.com

How to configure a Queue in JBoss ?

This article has been moved here: JBoss JMS configuration

How to create a Queue with Jmx Console ?

  Bring up the JMX Console in your browser and look for the sect

JBoss JMS Queue example

The following article shows how to create a simple JMS Queue Prod

JBoss JMS Topic example

The following article shows how to create a simple JMS Topic Publ

JBoss HornetQ simple tutorial

HornetQ is an open source project to build a multi-protocol, embe

How do I configure a Queue/Topic to work in a cluster?

  JBoss AS 5 Just set the Clustered attribute to "true" in your

Follow us on Twitter