Advanced Java EE 7 build and test using Git, Gradle, Groovy and Arquillian

The current generation of tools for the development of Java enterprise applications has a lot to offer with regard to developer productivity. In this article I would like to show a Java EE7 hands on example that uses WildFly 8 as application server, Git, Gradle and Groovy for build management and Groovy, Spock, JUnit and Arquillian for test automation. Together with the Eclipse based Spring Tool Suite IDE (STS) they team up to make Java enterprise development productive like never before. The complete project named JTrack can be found on github at https://github.com/martin-welss/jtrack-ee7  

1     Basic Setup and Prerequisites

1.1   Java and Gradle

First, we need to install Java JDK 1.7 or JDK 1.8 and Gradle 1.12 or 2.0 which can be found at http://www.gradle.org/downloads.   The whole example works without IDE using only the commandline although project files for the Spring Tool Suite are included. I found it always very important to be able to build and test the complete project on the commandline so it can easily be built on every server or workstation of the continuous integration pipeline.

To install Gradle, just unzip the archive and set the environment variables accordingly. Suppose we have both Java and Gradle installed in /home/opt, our environment variables in .bash_profile or .bashrc should look like this (assuming a Mac or Linux system, Windows users please adjust the syntax accordingly):

export JAVA_HOME=/home/opt/jdk7 
export GRADLE_HOME=/home/opt/gradle 
export PATH=$JAVA_HOME/bin:$GRADLE_HOME/bin:$PATH

We can quickly check the setup with one command:

$ gradle -v

------------------------------------------------------------

Gradle 1.12

------------------------------------------------------------
Build time: 2014-04-29 09:24:31 UTC

Build number: none 
Revision: a831fa866d46cbee94e61a09af15f9dd95987421 
Groovy: 1.8.6 
Ant: Apache Ant(TM) version 1.9.3 compiled on December 23 2013 
Ivy: 2.2.0 
JVM: 1.7.0_60 (Oracle Corporation 24.60-b09) 
OS: Linux 3.15.7-200.fc20.x86_64 amd64

1.2   Database PostgreSQL

We then need to install and configure PostgreSQL: basically, all we need is a valid login into a database via TCP. Mastertheboss has of course the adequate tutorial: Configuring a datasource with PostgreSQL and JBoss/WildFly. The first section is sufficient.

1.3   WildFly Application Server

To install WildFly we use the project wildfly-git-install on github, which consequently externalizes the local configuration into a properties file. That opens up the path to manage all WildFly installations of our continuous integration pipeline from a central repository with all the power Git has to offer: you can find configuration differences with git diff, use branches, undo experimental changes and update the installation reliably with just one command: git pull origin. Besides, you get a complete history of who changed what and when. For this example we need the postgres_branch of wildfly-git-install. Here are the commands to get there:

git clone https://github.com/martin-welss/wildfly-git-install wildfly-git-install
cd wildfly-git-install
git checkout postgres_branch

Suppose you fired that command in your home directory, you need to add one more environment variable to .bashrc:

export JBOSS_HOME=$HOME/wildfly-git-install

1.4   system.properties

Finally, we have to setup the externalized local (or server-specific) part of the configuration using the file system.properties which gets loaded on wildfly start up. Please copy the given system.properties.postgres from $JBOSS_HOME to $HOME/system.properties and edit there the database properties according to your local setup:

cp $JBOSS_HOME/system.properties.postgres $HOME/system.properties

On my computer, the system.properties look like this:

exampleds.jdbc.url: jdbc:postgresql:example
exampleds.user: wildfly 
exampleds.password: wildfly
txn.node.identifier: node-test1 
jboss.bind.address: localhost

Of course this local file is not tracked by git which also makes sure that no passwords or other secret information finds its way into the central git repository. Now we can start wildfly:

./standalone.sh --server-config=standalone-full.xml -P=$HOME/system.properties

or with remote debugging enabled

./standalone.sh --debug --server-config=standalone-full.xml -P=$HOME/system.properties

2     The Gradle Build Script

First we need to clone the jtrack-ee7 project from github:

git clone https://github.com/martin-welss/jtrack-ee7 jtrack-ee7

Now take a look at build.gradle: Quite different from Maven and Ant, which both use declarative XML to describe the build, Gradle uses an expressive DSL and with Groovy: a powerful dynamic language to program the build process. To me it was a real revelation when I saw the first Gradle build script with true for loops and if statements. Since Groovy supports plain Java too, we programmers can use all our every day skills from Java development finally in the build process. I can now understand why Google chose Gradle as the build tool for the Android SDK. Gradle is easy to integrate into an existing infrastructure, because it can work with maven repositories to resolve dependencies and execute Ant Tasks as a built in capability. So here is how to specify the repositories:

repositories {
     
	mavenCentral()
     
	maven { 
		url 'http://repository.jboss.org/nexus/content/groups/public' 
	}
}

And here is a sample dependency:

dependencies {
     testCompile 'org.spockframework:spock-core:0.7-groovy-2.0'
     ...
}

2.1   The automatic unique build id

Did you ever wonder what exact version you are dealing with while examining a bug on a test system? Or did you ask yourself if the most recent deployment really went well? The infamous 1.1-snapshot from Maven won’t help, because there are maybe hundreds of deployable .war-files with that version. So here is the solution that really is a breeze to implement with Gradle and Git: With each execution of the build script, it generates automatically an unique build id that contains project name, git commit id, git dirty flag, timestamp and username of the user who triggered the build. A build id may look like this:

jtrack-ee7 98b0293 B20140807 1655 mw

If the commit has an associated git-tag, then the tag is also added to the build id. This build id is then automatically included in the META-INF directory of the .war-file and can be accessed from the running application to show it for example in the footer of the web pages. It is then clear that the version is based on git commit 98b0293 and I could switch to that commit by simply checking it out.

This is how jtrack-ee7 reads the build id:

snippet from file jtrack-ee7/src/main/java/itf/jtrack/web/JTrackApplication.java:

InputStream str = getClass().getResourceAsStream("/META-INF/build_id.txt");
BufferedReader br=new BufferedReader(new InputStreamReader(str));
buildId= br.readLine();

And here is how to define the new task builId in build.gradle that generates the build id and interacts with git:

task buildId << {  

     buildDir.mkdirs()

     // generate timestamp and user for build_id

     def build_id='B'+new Date().format('yyyyMMdd HHmm ')+System.properties.'user.name'

    
     try {

          // get git status and ref

          def gitref="git rev-parse --short HEAD".execute().text.trim()

          def gittag="git describe --tags --always $gitref".execute().text.trim()

          //if(gitref!=gittag) { gitref="$gittag $gitref"}

          def gitdirty="git status --porcelain".execute().text.isEmpty()?" ":"*"

          build_id=" $gittag$gitdirty "+build_id;

     }

     catch (Exception x) {

          println "Warning: no git executable, using simple buildId"

     }

     println "buildId: $build_id"
  

     // and write build_id to file

     new File("$buildDir/build_id.txt").withWriter { out -> out.println build_id }

}

2.2   Deployment

Of course Gradle gives us all the power for flexible and customized deployment. Since WildFly comes with the powerful jboss_cli command it would the obvious choice to do the deployment. But unfortunately jboss_cli writes deployments into the configuration file and that conflicts with the central goal of wildfly-git-install to externalize all local settings. So we fall back to drop-in deployment which leaves the configuration files untouched. WildFly generates marker files in the deploy directory that we can use to monitor the progress of the deployment.

So here comes the Gradle task that does the deployment: It copies the generated .war file to the deploy directory and monitors the marker files that end with €œ.deployed€ or €œ.failed€. Furthermore we use the callbacks doFirst() and doLast() to keep the process in the right order.

task deploy(dependsOn: war, type: Copy) {

     println "configure task deploy"

     from('build/libs/') {

          include '*.war'

     }

     into "$System.env.JBOSS_HOME"+"/standalone/deployments/"

     def target="${project.name}.war"

     def ok=new File("${System.env.JBOSS_HOME}/standalone/deployments/${target}.deployed");

     def nok=new File("${System.env.JBOSS_HOME}/standalone/deployments/${target}.failed");   

     doFirst {

          println "deploy doFirst"

          if(ok.exists()) { ok.delete() }

          if(nok.exists()) { nok.delete() }

     }

     doLast {

          println "executing wildfly deploy..."

          def deployed=false

          for(def i=0;i<80;i++) {

               if(ok.exists()) { deployed=true; break }

               if(nok.exists()) { break }

               sleep(500);

          }

          if(deployed) { println("deploy ok.") }

          else { throw new RuntimeException("Deployment failed, see Wildfly logs!") }

     }

}

Last but not least, our application needs some sample data for a thrilling user experience. That is what the task loadDB is for: it reads the database access login from system.properties and executes some SQL statements to fill the tables. Here is how it’s done:

task loadDB(dependsOn: deploy) << {

     // read system.properties

     sysprops.withInputStream {

          System.properties.load(it)

     }

     // satisfy classloader conditions for DriverManager

     configurations.jdbcdriver.files.each {

          Sql.classLoader.addURL(it.toURI().toURL())

     }

     DriverManager.registerDriver(Sql.classLoader.loadClass("org.h2.Driver").newInstance())

     DriverManager.registerDriver(Sql.classLoader.loadClass("org.postgresql.Driver").newInstance())

     // connect to database

     def sql= Sql.newInstance("${System.properties.'exampleds.jdbc.url'}",

                            "${System.properties.'exampleds.user'}",

                            "${System.properties.'exampleds.password'}")

 

     sql.execute("insert into users (name,email) values ('mick','dummyemail.com')

     ...

}

By using the system.properties we continue the concept of wildfly-git-install to externalize all local configurations to a local file, so this task can be run unchanged on any server of the build pipeline. Since task loadDB depends on task deploy, we can now execute task loadDB and finally have the application up and running:

gradle loadDB

Here is what you should see, when you point your browser to http://localhost:8080/jtrack-ee7/

groovy gradle tutorial wildfly

Please note the build id in the footer.

The application creates HTML5 with one constraint that comes from JSF: Facelets checks for well-formed XML documents, so every HTML-Tag must be properly closed. If you take a look at the backing bean WebConversation, you see that it is annotated with @ConversationScoped so the user can have several JTrack Tabs open at the same time (the rightmost icon in the toolbar opens a new Tab). All Session Beans (like UserManager or BugManager) expose their functionality as RESTful Webservices. By the way, if you look at the size of the generated .war file, you will notice it is only 49 KB small. There are no additional jar files included, all dependencies are declared as providedCompile in build.gradle which instructs Gradle to use them only for compilation and to not include them in the .war file (I have a soft spot for small, pure deliverables).

Continue reading the article in the next page where we will show how to use GroovySpockJUnit and Arquillian for test automation.


3     Automated Tests

Testing Java EE applications automatically has always been challenging because of the need for some kind of container to run it in. Here we take a look at two approaches that could make life easier: In the first section we use the Groovy and the Spock Framework to write (external) tests for the RESTful Webservices and in the second section we use Arquillian and JUnit to write tests that will be run inside WildFly so we can even test the methods of classes not reachable from the outside. Gradle has a default target to run all tests in the src/test directory:

gradle test

The test results can be found as usual in jtrack-ee7/build/test-results.

3.1   Groovy tests with Spock

Spock is a Test Framework written in Groovy, but compatible with JUnit. Since it is written in Groovy we can benefit from the potential for compact code. Here is how a test case for a RESTful Webservice looks like
(taken from file jtrack-ee7/src/test/groovy/itf/jtrack/RestfulSpec.groovy):

class RestfulSpec extends Specification {

    def "read userlist restful"() {

         println "starting test"

         setup:

              def jtrack = new RESTClient( 'http://localhost:8080/jtrack-ee7/items/')

         when:

              def response=jtrack.get(path: 'user')

         then:

              assert response.status == 200

              assert response.contentType == JSON.toString()

              assert ( response.data instanceof List )

              println response.data

         println "finished test"

    }

Each Groovy Specification consists of three sections

  • setup: prepare the context for the tests
  • when: specify the preconditions
  • then: check the results

We use the Groovy Module HTTP-Builder that gives us the ready to use RESTClient class and makes the processing of RESTful requests and responses very compact and expressive. Spock comes with more features to support the compact writing of tests, for example to specify test input values in a tabular form. Please take a look at http://docs.spockframework.org/en/latest/data_driven_testing.html#data-tables

3.2   Arquillian and JUnit

Arquillian is a JBoss project that enables unit testing inside a Java EE container like JBoss or WildFly. Arquillian can operate in two modes: embedded and remote. In embedded mode, the container is controlled (started and stopped) by Arquillian whereas in remote mode, Arquillian assumes an already running container to which it connects. In our setup, we use the remote mode to make sure the container is started with the system.properties as a commandline parameter.

The first example tests a simple method of a Session Bean. The example can be found in the file
jtrack-ee7/src/test/java/itf/jtrack/ArqTest.java

@RunWith(Arquillian.class)

public class ArqTest {

    @Inject

    ProductManager prodman;

    @Deployment

    public static JavaArchive createDeployment() {

         return ShrinkWrap.create(JavaArchive.class)

             .addClass(Product.class)

              .addClass(ProductManager.class)

              .addAsManifestResource(EmptyAsset.INSTANCE, "beans.xml")

              .addAsManifestResource("persistence.xml");

    }

    @Test

    public void firstTest() {

         System.out.println("starting tests");

         List<Product> prodlist=prodman.findAll();

         Assert.assertTrue(prodlist.size()>0);

         System.out.println("finish tests");

    }   

}

To make sure the test is run with the correct testrunner, we annotate the whole class with @RunWith(Arquillian.class). Then we can use injection to get us an instance of the ProductManager Session Bean which is our test target. In the method createDeployment() we must list all necessary classes and their direct dependencies to pack them into an archive which Arquillian deploys for the test. We add an empty beans.xml to enable CDI and a generic persistence.xml for database access. Now we can specify our test just like a standard JUnit test.

But Session Beans can be reached and tested from outside the container as Webservices as we did in the previous example. So how about JSF Backing Beans? Fortunately, that works the same way, please have a look at the File ArqManagedBeanTest.java where the WebConversation bean is tested.

4     IDE Support

All the new tools are not much fun as long as they are not adequately supported by an IDE. For this setup I used the Spring Tool Suite (STS) http://spring.io/tools which is based on Eclipse. It is free like Eclipse itself and backed by Pivotal Software, the company behind the Spring Framework and currently also the main sponsor of Groovy and Grails (the java counterpart of the famous dynamic webframework Ruby on Rails). And since Gradle is based on Groovy, STS has very good and mature Gradle Support. In fact, Groovy, Grails and Spring are moving more and more towards Gradle as build tool. Groovy scripts can be started just like Java applications from the toolbar as shell scripts or in the Groovy Console. If you install the complete Spring Features you get of course excellent tooling in the form of wizards, completion and checks for productive Spring development.

After downloading and installing, you need to further install the Gradle and Groovy features from the Spring Update Site. Here is how my setup looks like:

groovy gradle tutorial wildfly

Then you have a Groovy Editor with Code completion and a Gradle View with a task list like the following screenshot, which is quite similar to the list of Ant targets: just double click on one of the tasks and it gets executed.

groovy gradle tutorial wildfly

Together with the well-engineered Git support of Eclipse, STS is really fun to work with. The .project files for STS 3.6 are included so the only thing left is to setup the Classpath Variable JBOSS_HOME that points to the WildFly installation and you should be up and running.

We have come a long way: I still remember EJB 1.0 and the respective Application Server Reference Implementation by Sun which forced an unbelievable cumbersome GUI on us for each and every deployment. No scripting, no drop-in deployments. And XML everywhere.

Today with the right tools, Java EE development finally is more fun than pain. Here is my list of best features tools for state of the art Java EE development:

  • Git and WildFly together make it possible to manage all WildFly installations of the complete continuous integration pipeline from workstation to production server
  • Gradle leaves the limitations of XML behind and brings the power of Java and Groovy to the build process
  • The concept of the automatic unique build id can clearly identify every .war/.ear file or application and connects it with a git commit to the source code
  • Spock and Arquillian simplify test development and help to raise software quality
  • With STS there is an IDE that supports all the above tools

Auhor: Martin Welss works as a freelance Java expert and architect in Germany who specializes in state of the art Java Enterprise development, build and test automation and continuous delivery. Check his site at: www.it-focus.de

Found the article helpful? if so please follow us on Socials