Colocated Symmetrical Live and Backup Cluster on Jboss EAP - With Parameterization

            For demonstrating the HA fail-over mode with parameters, this article will use two nodes, the configuration can however be extended as needed to numerous nodes. Traditionally for the Collocated failover mode setup, the full ha profile in the domain.xml will be replicated (after the hornetq server is itself copied over in the profile to create a back-up), this however will become tedious approach when there are say 4 or 5 cluster live and back up combinations.  Follow the below steps and life should be a little more easy

Step 1 : 

Dowload the jboss eap 6.4 version

Make two copies of the unzipped folder as master and node.

Step 2 :

Open domain.xml under the master/domain/configuration directory

Delete the profile default , ha , full

Navigate to the messaging subsystem under full-ha profile

Copy the hornetq-server and paste it right after the first hornet-server section

Provide a name to the second hornetq-server to differentiate it from the first , it can be any name , let us choose backup as the name.

Step 3:  Make changes to the  default hornet-server section  as below

           

            

The line no 14 shows the parameterization , the group name of the default server is now provided by the parameter groupa while the servers are configured to start. Also change the server-id of the in-vm connector and acceptor to a unique number on line numbers 24 and 35 ,this is because we are have to differentiate between the live and back up server ( which we are going to configure in next step )

Step 4:  Change the configuration the back-up server created by copying the original hornetq-server as below

            

                The line number 6 is changed to appropriately represent that this server will act as a back up . The backup group name is now represented by the parameter groupb .Also the server-id is now incremented to 2 to make it different from the live server running on  the same server . Also observe the changes on the socket-binding to messaging2 , this is to cater to the need that the backup server belonging to ${groupb} will come up when its master on a different server comes down . During such times this will help resolve the port-conflict that will arise with the groupa live server running on the current server. ( Jump to Step 4 to see the actual configuration of the socket-binding )

Step 4 :

            Move to the full-ha-sockets section of the socket-binding-group, create a new socket-binding named messaging2 and provide the port as 5446 ( you can opt to provide any number which would be different from the messaging socket-binding and which will not conflict with the other eap ports , this is a tested out port).

Step 5 :

            Move to the server-groups section and remove one of the server-group referring to the “full” profile . Change the server group name to “hornetqparamcluster”, this can be any thing or you can choose to leave the name as is, just remember the name.

Step 6:

Create a copy of the host-slave.xml file under the master/../configuration directory.

Step 7 :

Run the script ./add-user.sh under the master/bin directory ( this will be domain controller node ) , create a Management Realm user called admin .

Run the script once again and create a user called myhornetqcluster , during the interactive steps , as shown below.

         Copy the text <secret value="xxxxxx"/> 

Step 8 : 

•  Open the host-slave.xml file under the master/../configuration directory and add the below text under the management security realm node 

         <server-identities>
               <secret value="xxxxxx"/>
         </server-identities>
         where xxxxxx is the text copied in the Step 7.

• Move to the domain-controller section and remove the <local/> node and uncomment the <remote> node, add the attribute  username to the node with the value as “myhornetqcluster”  created in step 7.
• Repeat the above two steps for the file host-slave-node.xml
• Move to the servers section of the file host-slave.xml and change the group of the server to hornetqparamcluster .
• Add the system properties section to provide values to the ${groupa}  and ${groupb} parameters , provide the offset to 1000. Remove the second server configuration.
• Repeat the above step by editing the host-slave-node.xml but reverse the values of groupa and groub parameters , also provide the offset to 2000 , (remember we are running all the nodes on the single server ).

Step 9

• Start the domain controller  by running domain.sh under /master/bin as below

            ./domain.sh --domain-config=domain.xml --host-config=host-master.xml -b=192.168.56.101 

           -bmanagement=192.168.56.101

• Start the hornetq live server group by running the domain.sh under /master/bin in a separate window

         ./domain.sh --domain-config=domain.xml --host-config=host-slave.xml 

        -b=192.168.56.101 -bmanagement=192.168.56.101 

        -Djboss.domain.master.address=192.168.56.101 -Djboss.management.native.port=9993

• Start the hornet second live server group by running the domain.sh under /node/bin in a separate window

         ./domain.sh --domain-config=domain.xml --host-config=host-slave-node.xml 

        -b=192.168.56.101 -bmanagement=192.168.56.101 

        -Djboss.domain.master.address=192.168.56.101 -Djboss.management.native.port=9993

        Change the IP address to the address of your server.

          You should now have the hornetq servers in Colocated Symmetrical Live and Backup Cluster on Jboss EAP.

Fabric Deployment to Profile Git - Automation

Fabric maven plugin offers a wide variety of deployment options for Jboss Fuse. This article explores the deployment method pertaining to modifying the profiles on the Fuse profile GIT.

The two commands which we use to deploy the code in this article are fabric8:zip and fabric8: branch. Documentation on the plugin’s can be found here .

Scenario:

An enterprise project has the below bundles divided in to two profiles.

Common       :  DBCP bundle and MQSql Connector

Web Module : Has Business Related bundles, for e.g., Let us say

·      mvn:com.sundar.fabric8/webbackend/0.0.1-SNAPSHOT

·      mvn:com.sundar.fabric8/webfrontend/0.0.1-SNAPSHOT

·      mvn:com.sundar.fabric8/soapservices/0.0.1-SNAPSHOT

The goal is to generate a profile using fabric8:zip such that all the bundles from common are written to a profile common and the three bundles under the web module are written to web module profile.

Creating Profile Zips

1.   In addition to your regular code maven projects create a new project fabric8-deployment as a parent project.
2.  Create two modules common and web module with any packaging mode you wish (except pom), the main goal here is to generate profile zip files.
3. Modify the parent pom as below.
4. 
   
5. The distribution management section of the pom is needed. I have used a local repository for the example sake , it can be modified accordingly to the needs of the project.
6. In traditional fabric8:zip usage , the fabric8:zip plugin will be associated in the actual bundle builds and thus create a one-one to mapping for a profile and a bundle.
7. In this example the project which defines the fabric8:zip goal is marked to be ignore using the <includeArtifact> tag of the fabric8:zip goal which will then not write the project in to the profile created.
8. Now edit the common module pom to reflect as below.
9. Profile tag in the properties will be used to create the name of the profile.
10. ParentProfiles are mentioned if there is any associated parents for the profiles needed. (common examples are CXF projects might need to be associated with feature-cxf profile).
11. Bundles tag is where all the maven co-ordinates for the bundles and packages which go in to this project are mentioned separated by a space. The common profile in our example needed to have DBCP And MYSQL connectors, so these are added under the bundles tag.
12. Features tag is a place-holder in case any feature need to be added to the tag separated by spaces.
13. Edit the web module pom to reflect as below
14. Now run the fabric8:zip command on the parent project , you should see that the project zips are created individually and aggregated in to the parent target directory
15. You can run the mvn install command to automatically zip the profiles and deploy them to the locations specified in the distributionManagement part of the pom.

The generated profile will look as below

Creating the Project to Deploy the profile to the Git

     Once the profiles are created and installed to a accessible repository, we can now create the fabric8:branch  plugin based project to push these profiles to the repository.

1. Create a project camel-fabric-deployment and edit the pom as below
2. The distribution management section needs to point to the same repository where the profile zip files have been installed for the dependencies to be picked up properly.
3. Add all the profiles, created in the earlier step as dependencies to the project. Observe the classifier and type fields provided to specify the profiles.
4. Provide the branch name as the latest version .
5. Provide the oldBranchName as the current version in the git.
6. PushOnSuccess will determine if the repository needs to be updated on the successful compile or if it has not to be pushed.

Adding any Configuration files to the Profile.

           
We can add the configuration files to the profile within this framework by adding the profileConfigDir to the fabric8:zip goal configurations. The default value for this property is src/main/fabric8 , where in all the configuration files stored in this directory will be copied in to the zip file and subsequently pushed on to the profile via fabric8:branch goal.

            We can even have environment wise files created in each directory and over-ride the value of the profileConfigDir using the system variables. E.g., src/main/fabric8/${environment} and then run the command

Mvn –Denvironment=dev install

To generate the profile zip with only the files under the directory dev to move to the profile. This gives us a cleaner way of managing the property values from environment to environment.

Conclusion: Using the configuration properties provided by the fabric8 maven plugin we can tweak this deployment approach as needed by our environment. This approach has a pit-fall that the json file which describes the dependencies of each project (bundle), which generally gets created by the fabric8: zip goal will not be created. The ignoreArtifact property set to true will over-ride this functionality. Generally depending upon the setup of your project, this way can be safely adopted in to projects running fuse on ensemble in a common set-up.

For Sample Code Click Here

References: Thanks to Richard Stroop ,for clarifying my queries while working on the project.

Using AutoScaler in Fuse

Autoscaling is an easy repeatable way of spinning up fabric containers associated with the profiles . It can be particularly helpful while performing load tests and also to create the containers when new profiles are being deployed to the environment.

• Add the auto-scale profile to the root-container of the fabric (parent container ) not to the child containers.

Profile Requirements  can be defined  in two ways;from the web-console or the command line.

• Command Line : 
• fabric:require-profile-set command is used for defining the requirements for the instances that the profile is applied on 
• fabric:require-profile-set --minimum 1 --maximum 3 some-profile-id which then tells fabric that a minimum of one instance should always be associated with the profile in the fabric environment and also that a maximum of 3 instances can be associated at any given time.
• Web-Console
• Navigate to the Profiles page on the Hawt-io screen.
• Select the profile to which you want to add the requirements.
• Click the Add-Requirements link to the right of the screen.

• Provide the minimum and Maximum number of hosts to be associated with the profile.

Issues

Once the autoscaler is defined on the root container , the requirements on the profile will be auto-provisioned by the fabric environment . 

• But if for any reason the child containers are brought down , fabric will automatically create a new instance , this might prove to be a problem in case the child containers is being brought down for maintenance.
• If a container is down and is associated to a minimum requirements on a profile , the fabric will spin up a new container but will not attempt to start the shut-down container.

Issue With ObjectMessages 6.2.1 Jboss Fuse and AMQ

If you were working with Jboss Fuse 6.2.0 and migrated to 6.2.1 patch and working with Serialized Java Objects as messages in to AMQ , this post might be of interest to you.
I had a piece of code which was working fine in 6.2.0 version , the code was to simply drop a Java Serialized object in to AMQ  and then read it back.The class is as below

 package com.sundar.verify;  
 import java.io.Serializable;  
 public class MyBean implements Serializable{  
      /**  
       *   
       */  
      private static final long serialVersionUID = 1L;  
      private String id;  
      private String textData;  
      public String getId() {  
           return id;  
      }  
      public void setId(String id) {  
           this.id = id;  
      }  
      public String getTextData() {  
           return textData;  
      }  
      public void setTextData(String textData) {  
           this.textData = textData;  
      }  
 }  

with a camel route

 <camelContext xmlns="http://camel.apache.org/schema/spring">  
           <route>  
                <from uri="timer:foo?repeatCount=1&amp;delay=10000" />  
                <process ref="mybean" />  
                <to uri="activemq:personnel.records" />  
           </route>  
           <route>  
                <from uri="activemq:personnel.records" />  
                <log message="Body is ${body}" />  
           </route>  
 </camelContext>  

This was obsolutely working fine with Fuse 6.2.0 , but when the same code was migrated to 6.2.1 , I suddenly started to notice that the below exception was thrown

 Caused by: javax.jms.JMSException: Failed to build body from content. Serializable class not available to broker. Reason: java.lang.ClassNotFoundException: Forbidden class com.sundar.verify.MyBean! This class is not allowed to be serialized. Add package with 'org.apache.activemq.SERIALIZABLE_PACKAGES' system property.  
      at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36)  
      at org.apache.activemq.command.ActiveMQObjectMessage.getObject(ActiveMQObjectMessage.java:193)  
      at org.apache.camel.component.jms.JmsBinding.extractBodyFromJms(JmsBinding.java:126)  
      ... 23 more  

To prevent this from happening and for the deserilazation to work propertly add the below system property on the client side as below

-Dorg.apache.activemq.SERIALIZABLE_PACKAGES="java.lang,java.util,org.apache.activemq,org.fusesource.hawtbuf,
com.thoughtworks.xstream.mapper,<your-package-names>"  

and for the impatient

-Dorg.apache.activemq.SERIALIZABLE_PACKAGES="*"

In case if you are running the consumer on a fabric container , set the JVM props on the container as in the image below

Fabric Deployment Fuse

Deployments with Fuse And Fabric are very easy , there is a lot of automation available with in fabric which makes writing of scripts to manage devops a redundant process. In this post I am going explore two different ways of pushing changes to Jboss Fuse.

The versions of the products I am using are

1. Fuse 6.2.1 (bom version : 6.2.1.redhat-084). 
2. Fabric8 1.2.0.redhat-621084 (fabric maven plugin version : 1.2.0.redhat-133).

Using Fuse Profile Git - Manually

• Once Fuse Fabric Environment is setup on your local or server. 
• Go to http://<serverip>:8181. 
• Log in to Hawtio using your username and password 
• Navigate to the Containers , click on the root container navigate to Urls tab and copy the Git Url , it should look like "http://admin:admin@smunirAT-OSX:8181/git/fabric”. 
• Open your eclipse clone the repository , provide the username and password if prompted. 
• Create a new branch with a different version number
• Navigate to the working directory on the Git repo cloned , and import the fabric directory as general project to eclipse .
• All the profiles which are now the part fo the branch you checked out from git should now appear in eclipse.

• Edit the file io.fabric8.agent.properties file and change the repositories , features and bundles that you need to add / remove /modify on the profile

 #To add a repository to the profile use the below keyword   
 repository.  
 <repository-name>=mvn\:groupid/<artifactid/version/type/classifer  
   
 #To add a feature to a profile use the below keyword features  
 feature.<feature-name>=<feature-name>  
   
 #To add a bundle to a profile use the below keyword  
 bundle.mvn\:groupid/artifactid/version= mvn\: groupid/artifactid/version  
   

• Commit to the git repo and push to the remote repositories

Video for Demo Project




Look for The automated part of the above ....in my next post

Jmeter Jms Publisher with discovery url Java Sampler

Add the below Jars to lib/ext directory to the Jmeter installation

1. activemq-all-5.11.1.jar
2. fabric-api-1.2.0.redhat-133.jar
3. fabric-groups-1.2.0.redhat-133.jar
4. fabric-utils-1.2.0.redhat-133.jar
5. fabric-zookeeper-1.2.0.redhat-133.jar
6. guava-17.0.jar
7. jackson-annotations-2.4.3.jar
8. jackson-core-2.4.3.jar
9. jackson-databind-2.4.3.jar
10. mq-discovery-1.2.0.redhat-133.jar
11. mq-fabric-1.2.0.redhat-133.jar

Extend the class JavaSamplerClient.

Add arguments to the getDefaultParameters() method

Set the system properties zookeeper.url and zookeeper.password in setupTest() method.

Write the test to connect and send messages to the activemq broker.

For the complete code please click here

When fabric is used to create network brokers fabric assigns free ports so that they will not all run on default port settings , also this provides the flexibility to the users to add more brokers in to the mix later on or remove them when the load becomes lesser.

In this case the better option to connect to these network of brokers instead of using a fail over with all the ip's of the brokers or connecting individual brokers would be to use the discovery URL so to let the fabric manage the details.

Based on the below post from
https://github.com/FuseByExample/external-mq-fabric-client

There are three ways this can be done from Jmeter.Adding the System properties zookeeper.url and zookeeper.password while starting the Jmeter.

Editing the Jmeter.sh and adding the JVM Parameters

1. Edit jmeter.sh.
2. Append the below parameter to the JMETER_OPTS
   
   "-Dzookeeper.url=<zookeeperurl>  -Dzookeeper.password=<zookeeperpassword>".
3. Add the below jars to the lib/ext directory.
   1. activemq-all-5.11.1.jar
   2. fabric-api-1.2.0.redhat-133.jar
   3. fabric-groups-1.2.0.redhat-133.jar
   4. fabric-utils-1.2.0.redhat-133.jar
   5. fabric-zookeeper-1.2.0.redhat-133.jar
   6. guava-17.0.jar
   7. jackson-annotations-2.4.3.jar
   8. jackson-core-2.4.3.jar
   9. jackson-databind-2.4.3.jar
   10. mq-discovery-1.2.0.redhat-133.jar
   11. mq-fabric-1.2.0.redhat-133.jar
4. Run the script jmeter.sh and continue with the JMS publisher and JMS subsriber tests
5. The sample parameters are as below
   1. intialContextFactory=org.apache.activemq.jndi.ActiveMQInitialContextFactory.
   2. providerUrl=discovery:(fabric:<broker-group-name>).
   3. connection factory =ConnectionFactory.
   4. Destination = dynamicQueues/FOO.BAR.

Customizing the JMSPublisher Sampler with GuiControls

1. Copy the same jars from above step to lib/ext directory of Jmeter installation.
2. Write custom code to add a new sampler JMS Activemq Discovery Publisher.I went along the lines of JMSPublisherGui and PublisherSampler from the JMS protocol code , most of the code is copied from them with necessary modifications.
3. Create a class DiscoveryActivemqPublisherGui that extends AbstractSamplerGui and implements ChangeListener
4. Create a class DiscoveryPublisherSampler  which extends
   BaseJMSSampler  and implements TestStateListener
   to write the actual Sampler code.
5. In the initClient method of DiscoveryPublisherSampler write code to set the zookeeper properties to the system variables.
6. Copy the core part of the source code of Jmeter from here.
7. Add the property jms_discovery_publisher= JMS Activemq Discovery Publisher to messages.properties. This will be the label of the Sampler.
8. Use the pom.xml below to compile the code.

Replace the ApacheJMeter_core.jar with the jar obtained above.

For the complete code please click here